Oren Razon on key aspects of model observability, MLOps, and CI/CD for machine learning.
Subscribe: Apple • Android • Spotify • Stitcher • Google • AntennaPod • RSS.
Oren Razon is CEO and co-founder of Superwise1, a startup that builds tools to streamline observability for machine learning models. This episode provides a comprehensive overview of tools and best practices for deploying, monitoring, and managing machine learning models in production.
Some of the topic we covered include:
- The difference between monitoring and observability tools, particularly in the context of machine learning.
- Key features of a model observability solution.
- Learning from best practices, tools, and processes used in traditional software engineering (CI/CD, deployment modes like Canary and Shadowing; unit and integration testing.
- Responsible AI.
- The impact of broader trends on MLOps and model observability (e.g., multimodal models, reinforcement learning, large/foundation models).
Oren Razon on the Difference between Model Monitoring and Model Observability solutions:
Highlights in the video version:
- Introduction to Oren Razon
Difference between monitoring and observability
Machine learning, metrics, and models
Interpreting readouts from observability tools
Level of automation on the observability side
Injecting software engineering rigor into ML models
What do experienced practitioners do before deployment?
Key components of a model of observability platform
Metrics in ML depend on the model and use case
Training pipeline and modern observability tool integration
Integrating with ML Flow and all major model serving platforms
Teamwork and coordination
Observability should be agnostic
Multimodal models and diagnosing models
Root cause analysis and observability
Responsible AI
Related content:
- A video version of this conversation is available on our YouTube channel.
- Here’s what you need to look for in a model server to build ML-powered services
- Model Monitoring Enables Robust Machine Learning Applications
- Nikhil Muralidhar: MLOps Anti-Patterns
- Hamel Husain: Deploying Machine Learning Models Safely and Systematically
- “Navigate the road to Responsible AI”
[1] This post and episode are part of a collaboration between Gradient Flow and Superwise. See our statement of editorial independence.
[Image: Key Features of a Model Observability solution, from Superwise, used with permission.]