Vasant Dhar on Valuation Bots, Systematic Investing, and Narrative Analysis
Subscribe: Apple • Spotify • Overcast • Pocket Casts • AntennaPod • Podcast Addict • Amazon • RSS.
Vasant Dhar is a Professor at the Stern School of Business and the Center for Data Science at NYU. He’s one of the creators of the Damodaran Bot, an AI-powered system designed to emulate the valuation analysis and investment insights of renowned finance professor Aswath Damodaran. This episode explores the transformative impact of AI in finance, covering applications such as generative AI, AI-powered valuation bots, systematic investing, and narrative analysis. It delves into the development of an AI valuation bot, discussing motivations, technical approaches, and challenges. The episode also examines key machine learning concepts relevant to finance, multi-agent architectures, and industry trends. Overall, it provides a comprehensive overview of AI’s current and future role in revolutionizing financial practices and decision-making processes.
Interview highlights – key sections from the video version:
-
- Early Machine Learning in Systematic Investing
- Proposal for a Valuation Bot in 2015
- Understanding Damodaran’s Approach to Valuation
- Project Collaboration with Damodaran and Initial Inputs to the Valuation Bot
- User Interface and Bot Functionality
- Challenges in Mimicking Damodaran’s Judgment and Storytelling
- Framing Unique Questions for Each Valuation
- Meta-Level Analysis of Damodaran’s Approach to Valuation
- Data Gathering and Report Generation
- Regulation, Subsidies, and Challenges in Fine-Tuning the Model
- Variance in Outputs and Reasoning Process of the Valuation Bot
- Using Retrieval-Augmented Generation to Limit Hallucination
- Fine-Tuning the Bot to Mimic Damodaran’s Style
- Measuring Success: Valuation Prediction and Explainability
- Backtesting the Bot and Challenges of Knowledge Leakage
- LLMs in Finance: The Future of Valuation Bots
- Leveraging AI Innovation for Valuation and Beyond
Related content:
- A video version of this conversation is available on our YouTube channel.
- Digital Mentors: Building AI Systems That Think Like Experts
- Financial Machine Learning
- Balancing Act: LLM Priors and Retrieved Information in RAG Systems
- The Financial Services Sector’s March into Generative AI
- Arun Verma → How machine learning is being used in quantitative finance
- Enhancing AI Retrieval Systems with Structured Data
- Joao Moura → Unleashing the Power of AI Agents
- Philip Rathle → Supercharging AI with Graphs
If you enjoyed this episode, please support our work by encouraging your friends and colleagues to subscribe to our newsletter:
Transcript.
Below is a heavily edited excerpt, in Question & Answer format.
Can you tell us about the Damodaran bot you’ve been working on?
I’ve been working with NYU professor Aswath Damodaran, who’s known as the “Dean of Valuation” on Wall Street, to create a bot that replicates his valuation approach. The idea first came to me in 2015, before transformers existed, but I approached him again when GPT came out, and he was enthusiastic. Damodaran is particularly interesting because he combines quantitative analysis with narratives – he wrote a book called “Narratives and Numbers” that explores how stories matter alongside cash flows in valuation. His blog gets 20-40 million views, and he’s widely followed and influential on Wall Street, making him an excellent “gold standard” for an AI valuation system.
How does the Damodaran bot work?
Users can query the bot asking it to value a specific company like BYD or Nvidia. The system then collects all the necessary data – balance sheets, income statements, cash flow statements, comparables, and estimates. But importantly, it also gathers information about industry trends, regulation, and market context. For example, when valuing BYD, it looks at the autonomous driving industry, battery technology, and electric vehicle market. It then applies Damodaran’s valuation methodology to produce a report.
The challenge has been getting it to “think” like Damodaran, especially in how he frames analyses. For example, with Nvidia, he started by asking whether AI is revolutionary or incremental technology, which then guided his market size estimates. For another analysis, he grouped Walgreens, Starbucks, and Intel together, framing them as “aging companies refusing to age gracefully” – this type of insightful framing is what we’re trying to replicate.
What technical approach are you using to build this system?
We’re using large language models, but simply giving the LLM all of Damodaran’s blogs and asking it to value a company produces unsatisfactory results. Instead, we’re trying to extract the meta-level questions and approaches he uses for valuation. We’ve broken the system into multiple agents in a multi-agent architecture, reminiscent of AI systems from the 1970s.
One of the challenges is controlling the system’s “attention” – determining what it should focus on first. Should it analyze the numbers first? Read news articles? Look at what other analysts are saying? The order affects the output. We’re also dealing with what I call “reasoning variance” – unlike traditional machine learning where I worried about model variance (stability in response to changes in training data), with LLMs you get variance in the reasoning process itself.
How well is the system performing?
For now, our goal is to get the machine to think like Damodaran rather than just produce the same numbers. We’re having him evaluate the bot’s reports and grade them. In parallel, I’ve been testing it on a random sample of S&P 500 companies to check stability and accuracy. So far, the valuations have been within plus or minus 50% of the market value, which is encouraging – I would have expected several hundred percent variance.
The bot produces comprehensive reports, but it still doesn’t have Damodaran’s level of skill, particularly in framing situations with the same expertise. It’s more of a version 1.0 that works but needs refinement.
What are the challenges in testing and evaluating such a system?
One interesting challenge is back-testing. I’d love to test how the bot would have performed historically, but this is complicated because the LLM has knowledge of events that wouldn’t have been available at earlier points in time. There’s “knowledge leakage” when applying today’s knowledge to past situations.
Another challenge is that we don’t know what the LLM operators are doing behind the scenes. Are we dealing with the same model today that we were yesterday, or has it been modified through reinforcement learning?
For long-term value investing, evaluation is inherently more difficult than with high-frequency trading algorithms where you can determine efficacy in a couple of days. With longer holding periods, assessment takes much more time.
How is Wall Street responding to this type of technology?
Most financial firms are exploring more obvious applications, like having AI write analyst reports. At one investment management conference, someone mentioned they’re unable to distinguish between reports produced by AI and human analysts. Others are looking at automatically summarizing markets or producing boilerplate reports.
I haven’t seen anyone else attempting what we’re doing – building a bot that’s both broad (can talk about anything) and deep (has genuine expertise). But now that people have seen what we’re working on, I expect others will try similar approaches.
Finance as an industry is very pragmatic – as long as something works and makes money, people will use it. This is already an industry where algorithmic approaches are widely deployed in areas like high-frequency trading.
What would you like to see from the companies building LLMs and AI tools?
I’m amazed at the pace of innovation in LLMs, vision systems, and other areas. Today’s developers have a tremendous advantage compared to 20-30 years ago when you had to build tools yourself. Now, there’s a powerful foundation to build upon.
I’m leveraging these tools not just for finance but also in completely different domains like olfaction, where I’m using computer vision techniques to predict what a mouse is smelling, potentially for disease diagnosis. These are tremendously exciting times – the toolkits from language, vision, and other domains are becoming so powerful that they enable us to build systems I couldn’t have imagined even a few years ago.