The Data Exchange Podcast: Pardhu Gunnam and Mars Lan on why a metadata fabric is key to building data services on top of modern data stacks.
Subscribe: Apple • Android • Spotify • Stitcher • Google • AntennaPod • RSS.
Pardhu Gunnam (CEO) and Mars Lan (CTO), are co-founders of Metaphor Data1, creators of the first Modern Metadata Platform (MMP). As we noted in a previous post, a metadata fabric is the right foundation for data governance and data discovery solutions, data catalogs, and other enterprise data services. This insight resulted in several metadata systems being created within technology companies a few years ago. In fact, the team at Metaphor created one of the more popular systems – DataHub – while they were at Linkedin.
MMP’s are going to enable many important data services on top of modern data stacks. As an initial application of Metaphor’s MMP, we discuss how their metadata fabric makes data discovery truly self-serviceable.
Mars Lan
Metadata is such a cross cutting thing. You bring out the power of metadata by being cross cutting, not by being very vertical about it. So that’s why we spend a whole lot of effort thinking about architecture. How do we build a platform that is capable of onboarding, any kind of metadata you want, easily, and then present it back to the user in a way that’s very easy to build an application. … You need to have actual solutions that actually solve people’s problems. But because you have this solid foundation based on metadata, your solution will be sound as well. And then you can also change your solution, build your solution, evolve your solution over time, without having to tear out your foundation every time you do that.
Pardhu Gunnam
What Metaphor unlocks immediately out of the box is that it immediately gives you exposure to important rich insights within your data ecosystem. Like what data is the most useful, what is not, not just by simple queries and stuff, but bynbprofiling it with respect to who the users are, what kind of roles they have, and how are they related to you. … And there is no real place for capturing this information. That’s what Metaphor enables. … Metaphor enables people to share insights across the groups and improve your data literacy and within your company.
Highlights in the video version:
- Data Discovery, Data Lineage, and Democratizing Data
Collaborating around data, sharing data, trusting data, and discovering data
More people and jobs rely on finding and trusting data. Why did you decide to start Metaphor Data?
GDPR Compliance, Automation, and Human Component
Data Sets expand to Metrics, Dashboards, Machine Learning, Schema, and API
ML Flow for Data, Lifecycle Management, and Big Governance
Models and Broken Pipelines
Schema Evolution, Change Data Capture
Push Model and Supporting Both Forms of Ingestion
Metaphor Data tries to help companies understand their data to help solve their set of problems
Point Solutions and Metadata Fabric
Metadata’s real power is being cross cutting
Search and Data Discovery and Metadata Foundational Platform
Metaphor Data and DataHub
Difference between a discovery tool that uses this metadata mangement system vs. one that does not
Different types of metadata that makes your discovery tool much better
What do you have to do to implement Metaphor Data? What can it do and why is it the right platform for a company?
Metaphor Data: Unlocks immediatly out of the box, smooth integration, and rich insights into your data ecosystem
What are some of the engineering prioritites moving forward?
Homegrown Systems
Does Metaphor Data care about the underlying source transactional schema?
Modern data stack companies are homogeneous while bigger companies are very diverse
Are there plans to add another set of applications? If so, what would those be?
Data Quality and Pipelines
What kinds of companies should be thinking about a solution like Metaphor Data?
Is the metadata management system going to be a fundamental building block?
Do you still have to explain what metadata is?
Benefits of a better data discovery tool out of the box and foundational layer
Who is going to unlock the other use cases?
Metaphor Data is hiring. What are you hiring for?
Related content:
- A video version of this conversation is available on our YouTube channel.
- “The Growing Importance of Metadata Management Systems”
- Michel Tricot: “Modernizing Data Integration”
- Jeremy Stanley: “AI Begins With Data Quality”
- Ryan Wisnesky: “The Mathematics of Data Integration and Data Quality”
- “What is DataOps?”
- “The Emergence of Multi-cloud Native Applications and Platforms”
Subscribe to our Newsletter:
We also publish a popular newsletter where we share highlights from recent episodes, trends in AI / machine learning / data, and a collection of recommendations.
[1] Ben Lorica is an advisor to Metaphor.
[Photo: aerial photography of freight truck lot from Piqsels.]