How AI Security Became a DevSecOps Concern

While everyone is busy implementing ML in SecOps, we went further and started implementing SecOps in ML. But first things first. I am Svetlana Gazizova, I work at Positive Technologies as the Director of DevSecOps Process Development. By the way, we are familiar if you read my article about who are secure development specialists and where to study to become one.

What is MLSecOps

MLSecOps (or, as I've even seen, AISecOps) is a set of processes, practices, and technologies aimed at ensuring the security of the application development pipeline with machine learning (or artificial intelligence) models.

MLSecOps is needed to solve security issues of applications with ML and AI operations. In essence, it provides comprehensive protection of machine learning systems – those very “neural networks” that have become an integral part of our lives. Thus, almost 72% of companies use AI or ML modules in at least one business function. At the same time, according to HiddenLayer is an AI security platform developer, 77% of companies have detected leaks and information security incidents in AI.

So, MLSecOps helps to eliminate vulnerabilities, detect attacks (yes, yes, the stage Ops (let's not forget) and protects against leaks, maintaining the integrity and confidentiality of models and data. At the same time, the implementation of such practices helps the company not only detect and prevent anomalies, but also generally increase the availability of the applications being created and transfer the context to the quality layer. And security is always about quality.

The implementation of MLSecOps also concerns regulatory issues, so that it is not excruciatingly painful when checking the regulator: after all, we must guarantee trust in the models, and accordingly, in the applications used by the end user?

The use of models in applications is needed by someone. Now many functions are transferred to ML or AI – from the basic operating system, in order to speed something up and relieve specialists, to making management decisions based on the experience “accumulated” by the model.

Let's talk about problems

The complexity of MLSecOps is due to several factors:

Nobody knows about MLSecOps. The first rule of fight club, in short… It is extremely difficult to start spending resources and investing (even if not money, but knowledge) in the unknown. But it is necessary, because the “unknown” can already be actively exploited by attackers. Remember the expression: “If you stare long enough into the darkness, the darkness begins to stare back at you”?
Nobody knows anything about MLSecOps. Specialists need to have a very non-trivial set of skills: to be a bit of a security specialist, a bit of a DevOps specialist, and to be able to create models. Finding those who combine the skills of all three areas is something on the verge of fantasy.
The Complexity of MLSecOps. Yes, it is really difficult. Integrating MLSecOps into current development can be difficult due to the large number of elements of the already built process. This includes changes to the machine learning pipeline, testing, design, and development. Of course, all this will negatively affect the time-to-market and productivity of ML teams.
Evolving threat landscape. Attackers are also smart people and do not rely on outdated methods: new risks are constantly emerging that we must be able to take into account. Complex attacks may be ineffective to fight with classic methods. Therefore, we can only learn to constantly review and update the protective contour.
You need to be able to balance between security and performance. As discussed above, software security processes and checks often negatively impact team productivity and development speed. In general, there is no point in securing an application that does not perform its business function. It is like securing a computer that is turned off and has no internet access. Possible? Yes. Just not practical.

I hope I haven't demotivated you, because the risks of failing to implement security practices in ML applications are incomparable with the discomfort that may occur after reading this 🙂

What are the risks for business?

Data leaks. The presence of ML models in an application is always a signal that the software processes a solid piece of data that is continuously updated and remains relevant in the context of its task. Sometimes, users themselves contribute to this, “giving” AI sensitive information, among other things. Thus, ML becomes a desirable target for an intruder. At the same time, systems and applications are very susceptible to data leaks through exploitation of vulnerabilities in models. This includes unauthorized access to confidential information — user data, financial data, intellectual property.
Any leak is a blow to the reputation and a blow to the company's financial results.
FraudMachine learning systems are used in every industry. But what if a credit scoring system is compromised? How much could a company lose?
Failure to comply with regulator requirements. Yes, there is currently no regulatory document that obliges a company to use any means of protection against ML attacks. But we are obliged to protect applications: refer to any standard that you use in your work. It will definitely tell you what to do with application protection.
Disruption of operations. I have already said that ML is now used in both operational and management functions of companies – imagine what will happen if this opportunity is suddenly lost? Some of the company's processes will be interrupted – hence the financial losses from “downtime”. Are these risks comparable with the labor costs of implementation?

What does DevSecOps have to do with it?

Remember how many (or not?) years ago the concept of DevSecOps appeared in development? Development, security, operations, “eight DevSecOps”, “three pillars of secure development”, etc. If you look for articles that were written on this topic 5-7 years ago, you will see that they are all very vague: the same topic is discussed in a huge number of materials, only changing the form, especially without changing the content.

It seems that MLSecOps and DevSecOps schemes are similar, right? Well, or at least there is something in common…

How has the concept of DevSecOps changed now? If we look a little more closely, we will see that DevSecOps has “split off” from the standard concept of “pipeline security” and has also moved a little away from development.

Now it is something independent: DevSecOps dictates the rules of how to build a secure development cycle, changes the attitude of specialists to the company's cyber hygiene. Now, personally, nothing stops me from implementing secure development practices in companies where there is no DevOps pipeline – it can be built. And by the way, in companies with almost no cybersecurity either. I have already seen examples of how development became a trendsetter for the entire company: the emergence of secure development elements eventually led to the emergence of SOC, full-fledged incident monitoring, information security training, infrastructure hardening, etc.

In the case of the company I am talking about above, the implementation of secure development practices (DevSecOps) was impossible and unnecessary without DevOps and strong information security. Because what is the point of all this when your perimeter is not protected?

In fact, MLSecOps has the same pain points: without MLOps, you can’t do secure ML development. MLSecOps means integrating security practices and best practices into the existing process of developing and deploying machine learning models. This includes ensuring the security and privacy of the data used to train and test models, as well as protecting deployed models and the infrastructure they run on from malicious attacks. In short, all the good against all the bad — namely, against the risks we discussed above.

MLOps is the process of deploying machine learning models into production. It includes: automating the process of building and deploying a model, monitoring the performance and health of the model, scaling the infrastructure to handle large volumes of data and traffic.
The goal of MLOps is to make the process of developing and deploying models as efficient and reliable as possible: quickly and easily deployed into production and updated as needed.

In practice, MLSecOps and MLOps heavily influence each other and work together to ensure that machine learning systems are developed, deployed, and operated in a way that prioritizes security and reliability. And security and reliability are about quality. Just like DevSecOps 😉 Well, you remember.

Overview of MLSecOps and DevSecOps

Core security domains for ML and classic application development

If in classical secure development we see only blocks related to management, development and monitoring, then in matters of model security, great attention is paid to the stage of working with data and training the model, and only then do blocks from DevSecOps come.

When we work with data, we must ensure the security of the data collection stage, work with data sampling, and the dataset creation stage. When we ensure the security of the training stage, we must remember the stages of creating and testing the model algorithm, inference (applying the model), and analyzing the input data.

If you look at the MLSecOps architecture, you get the following picture: some practices are taken from DevSecOps, some are from the process of developing and operating models.

In my opinion, this is a pretty clear example of the connection between the “classical” secure development that we are already accustomed to and the secure development of machine learning models. It seems that DevSecOps is very complementary to MLSecOps – or vice versa. The important difference that exists now is the lack of any security regulations or testing during the development and operation of models, and vulnerabilities in them are and will be, because the world does not stand still.

And we are not standing still either! Want to learn more on the topic? Come in to our cyber horde.

I hope you found it interesting. Let's continue to uncover the mysterious layers of ML security and learn something new together!