Ken Liu on Balancing Privacy, Progress, and Responsibility in the Age of Large Language Models.
Subscribe: Apple • Spotify • Overcast • Pocket Casts • AntennaPod • Podcast Addict • Amazon • RSS.
Ken Liu, Ph.D. student in Computer Science at Stanford, is the author of Machine Unlearning in 2024. We explore the concept of machine unlearning, a process of removing specific data points from trained AI models. We discuss the historical context, popular approaches, and challenges associated with unlearning, such as data collection, evaluation metrics, and model interpretability. The episode also covers the practical implications and future directions of unlearning, including potential industry adoption, government mandates, and ongoing research efforts.
Interview highlights – key sections from the video version:
- Defining Machine Unlearning
- History of Unlearning and its Relation to LLMs
- Unlearning vs. Retrieval Augmented Generation (RAG)
- Popular Unlearning Approaches for LLMs
- Practical Challenges for Developers
- Exploring Unlearning Through Prompts
- Pipelines and Multi-LLM Systems
- Pretraining vs. Fine-tuning Unlearning
- Adaptability and Dynamic Unlearning
- The Critical Challenge of Evaluation
- Red Teaming and Domain-Specific Unlearning
- Unlearning in Academia and Industry
- Privacy-Preserving Techniques and Unlearning
- Privacy Techniques in the Post-Pretraining World
- Data Pruning and its Challenges
- Looking Ahead: The Future of Unlearning
- Interpretability and Explainability
- Predictions and Concluding Thoughts
Related content:
- A video version of this conversation is available on our YouTube channel.
- The Art of Forgetting: Demystifying Unlearning in AI Models
- Is Your Data Strategy Ready for Generative AI?
- Percy Liang → Evaluating Language Models
- Nestor Maslej → 2024 Artificial Intelligence Index
If you enjoyed this episode, please support our work by encouraging your friends and colleagues to subscribe to our newsletter: