Shayan Mohanty on creating a data management system to power modern computer vision applications.
Subscribe: Apple • Spotify • Stitcher • Google • AntennaPod • Podcast Addict • Amazon • RSS.
Shayan Mohanty is the CEO of Watchful, a modern and interactive solution that places the control of data labeling back in the hands of data scientists, machine learning practitioners, and subject matter experts. This podcast focuses on a data management system (written in Rust) they built to support the level of automation and interactivity required to support Watchful.
Highlights in the video version:
- Description of the problem space that led Watchful to create a data management system
What are labeling functions?
Key requirements of the data management solution they needed
Why vector databases did not meet their needs
Components of the data management system that they might open source
Search, nearest neightbors, and vector databases
Uses cases outside of data labeling, for which their data management system might be suitable
Open source plans: what and when
Open sourcing the data model
Why they are considering open sourcing this system?
Related content:
- A video version of this conversation is available on our YouTube channel.
- Documentation (for developers to get started with Weaviate open source)
- The Vector Database Index
- Bob van Luijt of Weaviate: An open source, production grade vector search engine
- Ram Sriharsha of Pinecone: A new storage engine for vectors
- Frank Liu of Milvus: A Cloud Native Vector Database Management System
- Summer of Orchestration: conversations with co-creators of Prefect, Dagster, Flyte, and Orchest.
- New open source tools to unlock speech and audio data
- fastdup: Introducing a new free tool for curating image datasets at scale
- A Guide to Data Annotation and Synthetic Data Generation Tools
If you enjoyed this episode, please support our work by encouraging your friends and colleagues to subscribe to our newsletter:
[Image: Visual Labels by Ben Lorica, generated with DALL-E and Stable Diffusion Playground.]