Site icon The Data Exchange

ETL for LLMs

Ben Lorica

Brian Raymond on how data can be made AI-friendly with open-source building blocks that connect unstructured enterprise data to LLMs.


Subscribe: Apple • Spotify • Overcast • Google • AntennaPod • Podcast Addict • Amazon •  RSS.

Brian Raymond is the founder of Unstructured, a startup building open source data pre-processing and ingestion tools specifically for Large Language Models (LLMs). Unstructured is focused on building tools for transforming unstructured data, particularly from large organizations, into a format that can be effectively processed by NLP solutions and LLMs. The process is complex and time-consuming, often involving the transformation and curation of varied document formats and layouts, while ensuring a high-quality, clean data feed. Solving this problem is critical now more than ever, as it allows us to fully exploit the potential of LLMs, leading to more cost-effective, efficient, and high-performing AI systems.

Subscribe to the Gradient Flow Newsletter

Brian Ryamond will be speaking at the AI Conference in San Francisco (Sep 26-27). Use the discount code FriendsofBen18 to save 18% on your registration.


Interview highlights – key sections from the video version:

  1. The origin story of Unstructured: why focus on ETL for LLMs?
  2. Maximizing Efficiency in Machine Learning Models: Strategies and Challenges
  3. Unstructured: Architecture, Design, and Scalability
  4. Streaming data and external integrations
  5. Target personas: data scientists and beyond
  6. Injecting software engineering rigor into how we build data pipelines for LLMs
  7. Understanding the Challenges and Importance of Data Quality in LLMs
  8. His favorite Unstructured use cases to date

Related Content:

If you enjoyed this episode, please support our work by encouraging your friends and colleagues to subscribe to our newsletter:

Exit mobile version