Building the Future of Finance: Inside AI Valuation Bots

Ben Lorica

Vasant Dhar on Valuation Bots, Systematic Investing, and Narrative Analysis

Subscribe: Apple • Spotify • Overcast • Pocket Casts • AntennaPod • Podcast Addict • Amazon •  RSS.

Vasant Dhar is a Professor at the Stern School of Business and the Center for Data Science at NYU. He’s one of the creators of the Damodaran Bot, an AI-powered system designed to emulate the valuation analysis and investment insights of renowned finance professor Aswath Damodaran. This episode explores the transformative impact of AI in finance, covering applications such as generative AI, AI-powered valuation bots, systematic investing, and narrative analysis. It delves into the development of an AI valuation bot, discussing motivations, technical approaches, and challenges. The episode also examines key machine learning concepts relevant to finance, multi-agent architectures, and industry trends. Overall, it provides a comprehensive overview of AI’s current and future role in revolutionizing financial practices and decision-making processes.

Subscribe to the Gradient Flow Newsletter

Interview highlights – key sections from the video version:

Related content:

 

If you enjoyed this episode, please support our work by encouraging your friends and colleagues to subscribe to our newsletter:

Transcript.

Below is a heavily edited excerpt, in Question & Answer format.

Can you tell us about the Damodaran bot you’ve been working on?

I’ve been working with NYU professor Aswath Damodaran, who’s known as the “Dean of Valuation” on Wall Street, to create a bot that replicates his valuation approach. The idea first came to me in 2015, before transformers existed, but I approached him again when GPT came out, and he was enthusiastic. Damodaran is particularly interesting because he combines quantitative analysis with narratives – he wrote a book called “Narratives and Numbers” that explores how stories matter alongside cash flows in valuation. His blog gets 20-40 million views, and he’s widely followed and influential on Wall Street, making him an excellent “gold standard” for an AI valuation system.

How does the Damodaran bot work?

Users can query the bot asking it to value a specific company like BYD or Nvidia. The system then collects all the necessary data – balance sheets, income statements, cash flow statements, comparables, and estimates. But importantly, it also gathers information about industry trends, regulation, and market context. For example, when valuing BYD, it looks at the autonomous driving industry, battery technology, and electric vehicle market. It then applies Damodaran’s valuation methodology to produce a report.

The challenge has been getting it to “think” like Damodaran, especially in how he frames analyses. For example, with Nvidia, he started by asking whether AI is revolutionary or incremental technology, which then guided his market size estimates. For another analysis, he grouped Walgreens, Starbucks, and Intel together, framing them as “aging companies refusing to age gracefully” – this type of insightful framing is what we’re trying to replicate.

What technical approach are you using to build this system?

We’re using large language models, but simply giving the LLM all of Damodaran’s blogs and asking it to value a company produces unsatisfactory results. Instead, we’re trying to extract the meta-level questions and approaches he uses for valuation. We’ve broken the system into multiple agents in a multi-agent architecture, reminiscent of AI systems from the 1970s.

One of the challenges is controlling the system’s “attention” – determining what it should focus on first. Should it analyze the numbers first? Read news articles? Look at what other analysts are saying? The order affects the output. We’re also dealing with what I call “reasoning variance” – unlike traditional machine learning where I worried about model variance (stability in response to changes in training data), with LLMs you get variance in the reasoning process itself.

How well is the system performing?

For now, our goal is to get the machine to think like Damodaran rather than just produce the same numbers. We’re having him evaluate the bot’s reports and grade them. In parallel, I’ve been testing it on a random sample of S&P 500 companies to check stability and accuracy. So far, the valuations have been within plus or minus 50% of the market value, which is encouraging – I would have expected several hundred percent variance.

The bot produces comprehensive reports, but it still doesn’t have Damodaran’s level of skill, particularly in framing situations with the same expertise. It’s more of a version 1.0 that works but needs refinement.

What are the challenges in testing and evaluating such a system?

One interesting challenge is back-testing. I’d love to test how the bot would have performed historically, but this is complicated because the LLM has knowledge of events that wouldn’t have been available at earlier points in time. There’s “knowledge leakage” when applying today’s knowledge to past situations.

Another challenge is that we don’t know what the LLM operators are doing behind the scenes. Are we dealing with the same model today that we were yesterday, or has it been modified through reinforcement learning?

For long-term value investing, evaluation is inherently more difficult than with high-frequency trading algorithms where you can determine efficacy in a couple of days. With longer holding periods, assessment takes much more time.

How is Wall Street responding to this type of technology?

Most financial firms are exploring more obvious applications, like having AI write analyst reports. At one investment management conference, someone mentioned they’re unable to distinguish between reports produced by AI and human analysts. Others are looking at automatically summarizing markets or producing boilerplate reports.

I haven’t seen anyone else attempting what we’re doing – building a bot that’s both broad (can talk about anything) and deep (has genuine expertise). But now that people have seen what we’re working on, I expect others will try similar approaches.

Finance as an industry is very pragmatic – as long as something works and makes money, people will use it. This is already an industry where algorithmic approaches are widely deployed in areas like high-frequency trading.

What would you like to see from the companies building LLMs and AI tools?

I’m amazed at the pace of innovation in LLMs, vision systems, and other areas. Today’s developers have a tremendous advantage compared to 20-30 years ago when you had to build tools yourself. Now, there’s a powerful foundation to build upon.

I’m leveraging these tools not just for finance but also in completely different domains like olfaction, where I’m using computer vision techniques to predict what a mouse is smelling, potentially for disease diagnosis. These are tremendously exciting times – the toolkits from language, vision, and other domains are becoming so powerful that they enable us to build systems I couldn’t have imagined even a few years ago.