Zongheng Yang on Sky Computing and SkyPilot.
Subscribe: Apple • Spotify • Stitcher • Google • AntennaPod • Podcast Addict • Amazon • RSS.
Zongheng Yang, is a researcher in the Sky Computing Lab at UC Berkeley, a multi-year research initiative that utilizes distributed systems, programming languages, security and machine learning to separate the services that a company requires from the choice of a specific cloud. He provides a detailed overview and update on SkyPilot, a groundbreaking intercloud broker that views the cloud ecosystem as a unified and integrated entity rather than a collection of disparate, largely incompatible clouds. SkyPilot enables users to run Machine Learning and Data Science batch jobs on any cloud, realize substantial cost savings, access the best hardware across clouds, and enjoy higher resource availability. Given the difference in pricing and hardware offerings between regions, SkyPilot has also become an important tool for users who rely on a single cloud provider.
Interview highlights – key sections from the video version:
- What is Sky Computing?
- Introduction to SkyPilot
- SkyPilot from an end user’s perspective
- The importance of being able to do things across regions (within the same cloud provider)
- What the current version of SkyPilot is able to optimize
- Data gravity
- Data gravity and the Skyplane project
- Support for Batch vs Streaming workloads
- Serverless technologies and SkyPilot
- SkyPilot: some example use cases
- SkyPilot roadmap
Related content:
- A video version of this conversation is available on our YouTube channel.
- FREE Report: 2023 Trends in Data, Machine Learning, and AI
- Specialized Hardware for AI: Rethinking Assumptions and Implications for the Future
- Percy Liang: Evaluating Language Models
- Jesse Anderson and Evan Chan: 2023 Trends in Data Engineering and Infrastructure
- Connor Leahy and Yoav Shoham: Large Language Models
- Machine Learning Trends You Need To Know
- Mark Chen of OpenAI: How DALL·E works
- Jack Clark: The 2022 AI Index
If you enjoyed this episode, please support our work by encouraging your friends and colleagues to subscribe to our newsletter:
[Image from Zongheng Yang, used with permission.]