MLOps and its tools
Gentle introduction about Machine Learning Operations
In early days , Data Scientist and ML Engineers focused mainly on algorithm development , data preprocessing , finding patterns in your data , making predictions and training model but they faced issues in managing and deploying models. They need a framework to deliver application with seamless integration to deploy their trained models and to make predictions.
Here comes in Machine learning operations to develop, deploy and monitor your workflow.
Build and deploy ML trained models with seamless integration
It provides a set of tools and frameworks that enable data scientists and ML engineers to easily build, train and deploy ML models in a scalable and repeatable way with seamless integration between your development cycle and overall operation processes.
What is MLOps?
Machine Learning Operations (MLOps) is a set of practices and techniques that combine machine learning and software deployment operations to manage end-to-end lifecycle of machine learning projects. It ensures that machine learning models can be developed, deployed, monitored and maintained efficiently. It mainly focuses on automation, collaboration and consistency to create more robust and scalable ML workflow.
Lifecycle
Data Engineering — This stage involves collecting data, establishing baselines, cleaning the data, formatting the data, labelling, and organizing the data.
Modelling — Now we come to the coding part, here we create the ML model. We train the model with the processed data. Perform error analysis, define error measurement, and track the model performance.
Deployment — Here we package the model, deploy it in the cloud or on edge devices as necessary. Packaging could be — model wrapped with an API server exposing REST endpoints, a docker container deployed on cloud infrastructure, deployed on server-less cloud platform, or a mobile app for edge-based models.
Monitoring — Once the deployment is done, we rely on a monitoring infrastructure to help us maintain and update the model. This stage has the following components:
- Monitor the infrastructure where we deploy — for load, usage, storage, and health. This tells us about the environment where the ML model is deployed.
- Monitor the model for its performance, accuracy, loss, bias, and data drift. This informs us if the model is performing as expected, valid for real-world scenarios or not.
Core Principles
With the surge in the usage of Machine Learning and AI in software products and services, everyone should be careful to follow MLOps best practices and use the right tools for testing, deploying, managing, and monitoring machine learning models in real-time.
Automation
Automation should be your priority if you want to use MLOps in your machine learning project effectively. The maturity of your machine learning process is determined by the level of automation you have. This, in turn, can increase the speed at which your machine learning models can be developed, trained, and deployed
Versioning
Versioning aims to treat machine learning training scripts, models, and datasets for model training as first-class citizens in DevOps processes and ensure that any changes made in the datasets and codebase are tracked correctly. The version control system saves and tracks different versions of the model. This makes it easy and seamless to revert to a previous version if needed
Testing
Testing is an essential aspect of the MLOps life cycle. MLOps supports a structured testing technique for Machine Learning systems based on the data pipeline, ML model pipeline, and application pipeline. As such, you should have methods in place for testing features and data, model development, and machine learning infrastructure.
Monitoring
Once a Machine Learning model has been deployed successfully, it should be monitored to know if it is performing as expected. In addition, the dependencies, data version, usage, and changes made to the model are monitored from time to time.
Pipelines
It has set of sequential steps triggered automatically design, deploy and manage model workflows
Data Pipeline -> Brings necessary data into your model
Environment Pipeline -> Choose right dependencies
Training Pipeline -> Train your model and improve performance
Testing Pipeline -> Test your model and cross validation
Deployment Pipeline -> Deploy your machine learning model in preproduction or production environment
Deploying ML models into Production
Model Requirements. The team, including management and all stakeholders, defines the business use case for the data and establishes the success criteria for measuring model performance.
Data extraction. The data scientists on the team select relevant data from a range of sources and integrate it for the ML task.
Data analysis. Team data scientists perform exploratory data analysis (EDA) to analyze the data available for creating the ML model. The data analysis process allows the team to understand the characteristics and data schema the model will expect.
Get Parameswaran R’s stories in your inbox
Join Medium for free to get updates from this writer.
Data preparation. The team prepares the data for the ML task, producing the data splits in their prepared format as the output. Data preparation involves applying feature engineering and data transformation to the model, and conducting data cleaning, in which the data scientist divides the data into sets for validation, training, and testing.
Model training. To train different ML models, the data scientist implements various algorithms with the prepared data. Next, to achieve the best performance from the ML model, the team conducts hyperparameter tuning on the implemented algorithms.
Model evaluation. The team evaluates the quality of the trained model on a holdout test set. This produces a set of metrics for assessing model quality as output.
Model validation. Next, the team confirms that the model’s predictive power exceeds a specific baseline, making it adequate for deployment.
Model serving. The team deploys the validated model to a target environment to serve predictions as part of a batch prediction system; as an embedded model for a mobile device or edge device; or to serve online predictions as web services or microservices with a REST API.
Model monitoring. The team monitors the predictive performance of the model to determine when to invoke a new iteration.
MLOps Levels
Google recognizes three levels of MLOps, beginning with a manual process level which involves no automation and is the most common, leading to ML and CI/CD pipelines that are both automated. Here are the three levels:
Level 0 (Manual Deployment Process)-> It reflects a manual process. At this level, the team may have a state-of-the-art ML model created by a data scientist, but the build and deployment process for ML models is completely manual.
Level 1 (ML Pipeline Automation)-> It reflects an automated machine learning pipeline framework that enables continuous testing (CT) of the ML model and continuous delivery of model prediction service. The team must add metadata management, pipeline triggers, and automated data and model validation steps to the pipeline to automate the process of retraining models in production using new data.
Level 2 (CI/CD Pipeline Automation)-> It reflects a robust, fully automated CI/CD pipeline system that can deliver reliable, rapid updates on the pipelines in production. An automated CI/CD pipeline system enables data scientists to explore and implement new ideas around feature engineering, hyperparameters, and model architecture rapidly, and automatically create new pipeline components, and test and deploy them to the target environment.
Tools
Tools allow you to manage model metadata and experiment tracking:
MLFlow
It is an open-source tool that helps you manage core parts of the machine learning lifecycle. It is generally used for experiment tracking, but you can also use it for reproducibility, deployment, and model registry. You can manage the machine learning experiments and model metadata by using CLI, Python, R, Java, and REST API.
Core functions -> Tracking , Projects , Models and Model Registry
Comprehensive tool used to manage entire process:
Kubeflow
Kubeflow makes machine learning model deployment on Kubernetes simple, portable, and scalable. You can use it for data preparation, model training, model optimization, prediction serving, and motor the model performance in production.
Key features:
- Centralized dashboard with interactive UI.
- Machine learning pipelines for reproducibility and streamlining.
- Provides native support for JupyterLab, RStudio, and Visual Studio Code.
- Hyperparameter tuning and neural architecture search.
- Training jobs for TensorFlow, Pytorch, MXNet, and XGboost.
- Job scheduling and work with major cloud providers.
DagsHub
It is a platform made for the machine learning community to track and version the data, models, experiments, ML pipelines, and code. It allows your team to build, review, and share machine-learning projects.
Key features:
- Git and DVC repository for your ML projects.
- DagsHub logger and MLflow instance for experiment tracking.
- Dataset annotation using label studio instance.
- Diffing the Jupyter notebooks, code, datasets, and images.
- Create a report for the project just like GitHub wiki.
- ML pipeline visualization.
- Reproducible results.
- Running CI/CD for model training and deployment.
- Provide integration with GitHub, Google Collab, DVC, Jenkins and external storage
AWS SageMaker
It provides framework to train and accelerate model development, track and version experiments, catalog ML artifacts, integrate CI/CD ML pipelines, and deploy, serve, and monitor models in production seamlessly.
