Gev Sogomonian on the role metadata plays in developing next-generation AI and LLM-backed applications.
Subscribe: Apple • Spotify • Stitcher • Google • AntennaPod • Podcast Addict • Amazon • RSS.
Gev Sogomonian is co-author of AimStack1, an open-source, self-hosted AI metadata tracker that logs all your AI metadata, such as experiments and prompts, and provides a user-friendly UI for comparing and observing them. It also offers an SDK for programmatically querying tracked metadata.
We also discussed their recently announced integration with MLflow. MLflow is a powerful open-source platform designed for managing the end-to-end lifecycle of machine learning projects, including experimentation, reproducibility, deployment, and maintaining a central model registry. However, despite its capabilities, effectively exploring and comprehending the results of tracked experiments can pose a challenge. This is where Aim comes in as a tool that offers a range of advanced features for in-depth exploration and comprehension of tracked experiments, delivering valuable insights and understanding of the results through a user-friendly interface.
Interview highlights – key sections from the video version:
- Building an open source experiment tracker
- Metadata as a foundation
- Integrations with rest of ML ecosystem
- Storing and managing metadata associated with ML
- MLflow
- AimStack user base and usage patterns
- Metadata and MLOps
- AimStack roadmap
- State of tools for ML and MLOps
- Data-centric AI
Related content:
- A video version of this conversation is available on our YouTube channel.
- Zongheng Yang: 2023 Running Machine Learning Workloads On Any Cloud
- Nikhil Muralidhar: MLOps Anti-Patterns
- Pardhu Gunnam and Mars Lan: Why You Need a Modern Metadata Platform
- Hamel Husain: Deploying Machine Learning Models Safely and Systematically
- FREE Report: 2023 Trends in Data, Machine Learning, and AI
- Specialized Hardware for AI: Rethinking Assumptions and Implications for the Future
- Percy Liang: Evaluating Language Models
If you enjoyed this episode, please support our work by encouraging your friends and colleagues to subscribe to our newsletter:
[1] Ben Lorica is an advisor to AimHub and other startups.