A separate approach exists for the successful, productive operation of machine learning models: Machine Learning Operationsor abbreviated: MLOps. MLOps comprises three terms:
- Machine Learning
- Software Development
- Operations
MLOps learns from DevOps, but has special requirements
The combination of software development and operations has been well known for a long time and is known as "DevOps" has permanently changed the IT industry. The concepts established by DevOps form the basis for MLOps, where tried and tested methods of software development and software operation are used. The special requirements of MLOps include
- Reproducibility and traceability: Versioning is part of the basics of software development. But the same requirement also applies to machine learning: both the models created and the data used must be versioned. This is the only way to reproduce and track results.
- Scaling: Machine learning models must first be created (trained) and then offered as a service. Both require the dynamic provision of hardware resources. However, especially for training, it is often necessary to provide many resources for a limited period of time. This can only be achieved with a scalable solution.
- Monitoring: Of course, the availability of the machine learning services must first be monitored in the traditional way ("Is the service running at all?"). However, the service also continuously generates model predictions - the quality of which can decrease over time. If the use case allows, the prediction quality should therefore be constantly monitored.
- Reusability and standardization: From the data to a prediction, a number of steps have to be taken. Basically, these include Data collection, data preparation, training and prediction. Through standardization and a clear separation of the individual steps, it is more likely that individual components can also be used across projects.
MLOps Lifecycle: From proof of concept to scalable AI applications
How can these requirements be met? With the help of Pipelines a fully automated MLOps lifecycle is successfully covered. The complete lifecycle is shown in Figure 1:
The data is first collected and processed. This can now be used to train a model, whereby the exact data set with which the model was trained is also saved. The model itself is saved as Container packed and in a Container registry version. This model can now be rolled out automatically and is directly available for use as a service. The performance of the predictions, as well as the technical availability of the service, can be improved through appropriate Monitoring solutions can be monitored. If the prediction quality of the model becomes too poor, or if the newly collected data changes ("data drift"), an automated, new training of the model can be triggered and the life cycle begins anew.
Providers of MLOps solutions offer a good basis
A fully comprehensive MLOps solution is not always desirable or useful. Machine Learning Operations is a Graduated concept. Depending on the application, a needs-based concept should be designed and evaluated, which MLOps elements offer the greatest benefit.
There are a large number of providers for the technical implementation of the conceptual solution shown. Two technical solutions are shown here as examples.
Both Microsoft Azureas well as Amazon AWSoffer for the entire MLOps life cycle Cloud-native solutions on. Cloud providers use their familiar basic services and build their machine learning solutions around these services. This provides users with a user-friendly package that significantly simplifies many of the life cycle phases shown. The following table lists a Exemplary implementation in the cloud for both Azure and AWS:
| Lifecycle phase | AWS | Azure |
| Versioning | AWS Sagemaker, based on Amazon S3 object storage | Azure Machine Learning, based on Azure Blobstorage object stores |
| Deployment | AWS Sagemaker, based on AWS EKS (Kubernetes), or AWS EC2 (virtual machines) | Azure Machine Learning, based on Azure AKS (Kubernetes), or Azure VMs (virtual machines) |
| Pipeline creation | AWS Step Functions | Azure Machine Learning Pipelines |
Table 1: Exemplary Ml-Ops implementation in the cloud. Source: own compilation/Aug. 2021
The basis for artifact versioning is the Object memory the provider. The offerings for virtual machines or containers are usually used to deploy the service or for training. A graphical representation of a simple AWS solution is shown in Figure 2:
Of course, it is not absolutely necessary to operate MLOps in the cloud. Also On-premise solutions can be implemented. There are cloud-independent technologies for this, such as Kubeflow. Kubeflow builds on the Kubernetes ecosystem to train and deploy models using appropriate containers. It also offers a variety of components to map the individual lifecycle phases such as training, deployment, scaling, etc.
Machine Learning Operations is designed to meet the special requirements for the productive operation of machine learning models. Above all, this includes reproducibility and the automation of all processes. Through the creation special machine learning pipelines over the entire life cycle of a model, this is exactly what can be achieved - whether in the cloud or on-premise. With a professional and structured approach, your prototypically implemented AI use case or PoC into a productive, usable software solution.
To the doubleSlash MLOps range of services
Download whitepaper: Guide to the adapted process model
