Why Build Your Own AI Model When Frontier Models Already Exist?

Ben Lorica

Joel Hron on Open Models, Proprietary Data, and the Economics of Enterprise AI.


Subscribe: Apple • Spotify • Overcast • Pocket Casts • YouTube •  AntennaPod • Podcast Addict • Amazon •  RSS.

In this episode, we talk with Joel Hron, CTO of Thomson Reuters, about the company’s decision to develop its own frontier AI model, Thomson, using open-weight models and decades of proprietary content and expertise. We discuss the economics of building a specialized model, the data and training infrastructure required, and how Thomson works alongside commercial frontier models in production. We also explore AI agents in legal and tax workflows, the challenges of verification and human oversight, and why enterprises may increasingly turn to open-weight models for greater control over their AI systems.

Subscribe to the Gradient Flow Newsletter

Interview highlights – key sections from the video version:

Jump to transcript

Related content:

Support our work by subscribing to our newsletter📩

Transcript

Below is a polished and edited transcript.

Ben Lorica. All right. So today we have Joel Hron, CTO of Thomson Reuters, which you can find at Thomson Reuters.com. And for our listeners and for myself, I guess there’s Reuters, the new service, but then the parent company is Thomson Reuters, which also sells software. I think mainly legal. Is that correct? Yeah.

Joel Hron. So I would say primarily sells software. I think you know certainly publicly we’re we’re well known for Reuters, the news and media brand. But as a business, that’s that’s really a relatively small portion of the company, like less than 10% really, of the of the company overall.

And we are primarily a software provider in the legal, tax, compliance, audit, and Risk segments, and I would say with across those segments, you know, probably in the top three software providers in the world, in each of those segments, legal being the business biggest, as you said, but very material presence in tax and compliance more broadly as well.

Ben Lorica. And by the way, for our listeners, it’s Thomson without a P, for you Americans. All right. So, so the reason I have Joel here is they’ve announced this really fascinating frontier model, specifically tuned for their domain and use cases, and so I want to go through and just talk through because I think a lot of people are, you know, at the point where they want some level of customization and control, and so I think there’s going to be a broad lesson here, applicable to many of you listeners. So first of all, Joel, what was the pivot point? At what point did you folks decide?

All right, so maybe we should just do our own thing, do our own frontier model. I know that you there was an acquisition of a startup in the UK, right? So, so maybe there was some talent acquisition there, but it has to be more than that, right? So,

Joel Hron. Yeah, I mean, this has been a long journey. Perhaps you know more than two years now. I would say we did. We certainly didn’t just like wake up one morning and say, let’s let’s go do this and turn around in a couple weeks and ship a model. I would say you know this the starting point for this was really driven by a recognition of what was and is and is going to continue to drive the flywheel of AI in the future, and I think as we look at that, we see sort of three dimensions of that.

We see compute, certainly, which everybody talks quite a lot about as the main scaling factor, but the other two components are really data and expertise, and Importantly, I think the type of data and the type of expertise that is driving the frontier of models is really data and expertise that is out of distribution of the normal internet of data. That has, you know, somewhat been exhausted to a degree, and the things that are really driving the frontier of capabilities in future iterations of models are really like expert knowledge and expert judgment.

You see, you know, great leaps in things like science and coding and things like that. And it’s not just, you know, more code that’s being written. It’s like harder problems, harder coding problems, harder security problems, like harder judgment and reasoning problems that are really pushing these models to new frontiers. And within the domains of law and tax and compliance and risk, as I said, you know, TR has always been and is probably most well known as the leading content and expertise provider in those industries. And so, we employ more lawyers and tax professionals than almost anybody else in the world.

And we really saw those assets as being real differentiators on those two dimensions of the AI flywheel, and we felt like the best way to really harness them into something that was durably valuable was through the training of a model directly, and that’s what brought us here. Certainly, it’s been a long journey, and we can talk about that. But that was that was the root for. Philosophy or strategic thinking, if you will.

Ben Lorica. But on the other hand, obviously there must have been an inflection point because if you were starting in your journey and you were just starting to use AI, then you would have a bunch a bunch of use cases, several use cases, and then you know while you’re still kind of feeling your way around, you might use one of the commercial frontier models, use their API endpoints, right? So prove out the use case. So you must have gotten to the point where you were using a lot of AI.

Joel Hron. Yeah, we do. I mean, we have really like an agentic AI product within every one of those verticals today, whether it be law or tax or audit or risk. Like we have a variant of like an agent-native AI product in the market today, and I would say, by and large, even still to this day, we use a plurality of commercially like closed source, if you will, models within those products. I would say that’s still like the majority of the token consumption of those products today.

Now that might change over time as we evolve, Thomson, but still, like that, that’s true today, and we continue to leverage those third-party frontier models, like where it makes sense to do so. In fact, our journey of kind of developing Thomson really started from like a pretty basic place of like just using the fine-tuning APIs that are available by those providers, you know, using simple things like LoRA fine-tuning of those models, and what we found was that those models became overly narrow in terms of their capability.

As we did just sort of basic fine-tuning, we would get really good at the one or two evals that we had sort of set as priorities for that fine-tuning exercise, and every other capability of the model would degrade quite dramatically.

This sort of catastrophic forgetting problem that’s kind of well talked about in the AI community, and that was sort of unavoidable when using some of these basic fine-tuning APIs, and so it was really through that process several years ago that we said, “Look, we need if we’re going to do this, we need a much more like advanced and purposeful methodology for not just like post-training the model, but really thinking about training throughout the full life cycle of mid training, continuous pre-training, as well as fine-tuning and reinforcement learning, and so we built we built a team like through an acquisition, like you said, through Safe Sign, and continue to hire into that team to build up the capabilities from a technical perspective that were necessary to kind of develop those techniques, and we continue to evolve that model over the last I’d say 18 months, and evolving that model, you know, really involves building our own datasets.

So taking both our raw content as well as our editorial content and sort of munging it into a form that is useful for AI development.

It involved building a tremendously broad but also deep evaluation suite that helped us understand the performance improvements we were getting on the model and the dimensions that we cared about, but also the more general dimensions that would speak to whether the model was sort of suffering from this catastrophic forgetting problem or not, and building the training methodologies to exercise those things in a useful way, and so that’s really what the last 18 months or so has been like.

I would say we’ve also continued, you know, speaking to the first sort of dimension of the flywheel that I mentioned earlier: compute data expertise.

You know, our bet at the early stages was that you know while compute is important, like the open-source community would maintain, if not lag by a few months, at least maintain pace with the frontier of intelligence, and that through that open-source or open-weight model development, we would spend significantly less compute, in that we wouldn’t have to sort of pre-train from scratch some of these like frontier capabilities, and we would just be able to sort of build on top of those open-weight models, and fortunately, I would say that’s largely held true over the last couple years, and we’ve certainly benefited from that, and continue to benefit from that today.

Ben Lorica. All right. So for our listeners, just a bit of a bit of background. So as Joel noted, they didn’t. Pre-trained from scratch. In fact, they took Alibaba Qwen, a checkpoint of Alibaba, and then I guess they first realigned it using their using constitution, right? So and then. then, as Joel mentioned, they have their own datasets that they wanted to make sure they fed into the training process billions of tokens, as best I can tell from reading the paper, and then they did a bunch of post-training on top of that to make sure the Thomson model excelled in some in their domain and also agentic workflows.

So Joel so Alibaba when I have to admit, it’s been a while since I’ve tried Qwen. I’ve kind of shifted over on the open-weight side. I’ve shifted over to GLM and Kimi. Those are the those are the new fashionable things. So, so at what point did you? So, so first of all, Qwen. I guess it must be true then that the license of Qwen allows you to do what you did, right? So, and then, what was the time frame when you took the checkpoint? Was that 12 months ago or?

Joel Hron. No, so the first checkpoint of Qwen is Qwen 3.5, which was back in Probably like end of May, beginning of June, when we started this last training run.

Ben Lorica. Okay,

Joel Hron. And at and at that point of time, you know that was really state of the art as far as open-weight models is concerned. Like Kimi k3 and GLM five three and such did not come out until after that. So, you know, you know, one of the things that like we really focused on when we built this like training pipeline, which you which you described, is like model realignment, continuous pre-training, and then a various stages of post-training with RL as well as agentic RL environments. We really focused on building that pipeline to be adaptable to future variants of models that would come out.

So, you know, we’re actively looking at GLM 5.3 flash today, for instance. Other open-weight models from companies like Thinking Machines, NVIDIA, et cetera, have been released since June. So, I would say just in the last couple of months, we have a lot more optionality in terms of model starting points, even than we did in May or June. And so, and so, Qwen 3.5 was really the best version on our benchmarks at that time, but you know today, you know in our current you know training environments, we’re using different versions of different models. And

Ben Lorica. I think I read this somewhere. The cost as you compute it, I’m not sure if this is just the compute cost was about $$450,000. Obviously, that doesn’t that doesn’t include the employees.

Joel Hron. No, no. So $450,000 is the is the final training run compute cost. You know, if you looked at sort of like the full life cycle, yeah. So if you looked at the full life cycle of the last two years of development across you know our people as well as compute. You know we’ve probably spent somewhere around $40 million in total, like compute and personnel costs, which is still pretty modest in the in the scheme of things.

But $450,000 is really like the training cost for the final run, given the model size, kind of you know that we’re in, which is you know Qwen 3.5 is about just shy of 400 billion parameters, 397 billion parameters, I think, and so

Ben Lorica. $40 million. So that’s sort of $40 million is not an investment that the typical enterprise would take lightly, right? So obviously, as you know, these neo labs are getting seed rounds. Certainly, certainly not

Joel Hron. One that an enterprise would take lightly. But I would argue that the majority of them are probably spending multiples of that in token costs annually as well. So, so yeah, I mean, there’s there’s a cost to this, and I’ve I’ve sort of used the analogy before.

Ben Lorica. It’s a bit of a the way you described it. You had some sunk costs just to get up to speed, but now let’s say you take one of the latest Kimis and start from scratch from using your pipeline from there. That’s not going to be $40 million, right? So that’s going.

Joel Hron. Absolutely, yeah. I mean, like, and we hopefully are building our infrastructure and our code in a way that we could do that in a matter of weeks, not months and years. You know, adapt to the latest open-source models, and so I think our goal is to build a pipeline that is cheap to run and easy to adapt to future variants of open-source models, and we can continue to very quickly and cheaply layer on our intelligence to those variants. So, I think I think that’s an accurate statement.

That like you know, it does take some cost to get up and running, but I think once you do, you build some equity into something that’s a little bit more sustainable and durable,

Ben Lorica. But then as you as you hinted, there’s a bit of you have to invest in a team as well, right? So, so can you can you describe for our audience like if you were to advise someone in another company, right? So what sort of team will they need to get this off the ground?

Joel Hron. Yeah, you know. So our team today is between you know science and engineering is probably around 30 people, maybe a little bit less even. I would say like we built this team in house at the same time. Like there are there are a lot of companies out there that are building like these teams as like consulting services in many ways. I think you can look at other companies who have done this via partnership or you know, and so that’s an option as well. I don’t know that you need like we have, you know, several trillion tokens of content within TER across all the industries.

So like, we’re dealing with quite a large amount of data, quite a large number of experts, and so you know the team of people that we have is probably larger than what maybe most other enterprises would need to have, but you know that’s sort of the order of magnitude that we’re thinking of. But

Ben Lorica. Then some of the people, as you mentioned, are in house because, as you describe it, right? So a lot of the work has to do with these data pipelines.

Joel Hron. Absolutely, right? Yeah, data pipelines and also like

Ben Lorica. These are the people already at the R, right?

Joel Hron. Yeah, and also like building not just the data pipelines, but building the RL environments that represent what our customers are trying to do with these models in the first place, and that’s where you know I mentioned our experts in law and tax and compliance, we have more than 3,000 experts in those domains, and so a lot of the work is not just like you know that group of a few dozen people building the model, but it’s really building environments that allow those 3,000 people to do their work and us to harness like the judgment and interpretation and expertise in their minds into data that drives the flywheel of the model.

So I think that’s probably the most important thing to get right is really what is what is the IP that your business has, and how do you harness that IP and knowledge and judgment into data that you can leverage, whether it’s for training a model or not. Like that, that is the data that is unique to your business and what you should be using to leverage your differentiation in the market.

Ben Lorica. Can you explain to our listeners, Joel, why that first step was important? This whole notion that you would take a public constitution and do some realignment of the open-weight model.

Joel Hron. Well, yeah, I think a couple things. There, you know, certainly in an enterprise like I said ourselves, we deal with like large customers and government customers, etc. and there was an open question as to like, okay, how do you take an open-weight model that you can’t validate the source from China?

Ben Lorica. From China.

Joel Hron. Well, it may be from China, it may be from anywhere in the world. Yeah, like these labs. But you know, you know,

Ben Lorica. There’s a bit of a paranoia about. There

Joel Hron. Is a paranoia, and so I think we very abstractly said, how do I take any model in the world where I have the weights and realign those weights to values and principles and ethics that like we can stand behind as a company? So I think that was sort of the first like novel, like academic challenge, perhaps, but also commercial challenge that we wanted to solve. But I think the second is really rooted in the business that we’re in. You know, you talked about Reuters. Like Reuters is a 150 plus year old business that’s rooted on trust principles that are like more than.

150 years old about how to ethically practice journalism and how to be truthful and honest and objective, and all of those things are also true when you think about the law or you think about financial services or you think about other industries like that. This aspect of being truthful and factual and honest-that like these were principles that were important to us that we wanted to make sure made their way into both the like personality of the model as well as the functionality of the model, and so that’s why we really focused on that as like a first a first stage of the training pipeline.

Ben Lorica. And then as far as the data in the second step, right? So as you mentioned, you have a lot of domain-specific data in legal, accounting, auditing. How hard was it to take that data and make it consumable?

Joel Hron. Yeah, I mean this is hard. This is a training

Ben Lorica. Process. And by the way, I guess you might as well answer this question to what’s the nature of the data? Is it multimodal?

Joel Hron. Okay, yeah, yeah. So, so I mean, certainly this is this is all of the work, or not? I shouldn’t say all, but this is the majority of the work in terms of in terms of the data, I would say it really comes in three shapes. And first of all, in terms of like the modality of the data, we’re talking primarily textual data today. You know, Reuters for sure does have like quite a large archive of both you know image and video content. But you know, our focus was primarily textual in this case, given that we were legal is

Ben Lorica. Legal is mainly textual, and then auditing and accounting is somewhat structured, right?

Joel Hron. Structured and numerical, yeah. And certainly, we did train on a lot of tax content as well, but that tax content was primarily like I would say regulatory in nature in terms of like, what does the tax law say, and how do you interpret the law? Not to say we won’t evolve that. I think there’s a lot of numerical and quantitative sort of use cases that we have in front of us, but our primary focus at this point in time has been textual, and we’ve been I would say more biased towards the legal domain at this point in time, as well, but maybe

Ben Lorica. The source data is not in PDF files. There, what’s the format of the source?

Joel Hron. It can vary. Sometimes it’s PDF form files. Sometimes it’s like XML systems that like our data repositories run on. So it can be a variety of different of different shapes. I would say you know we use the data in a number of different ways. So first of all, we use it in terms of continuous pre-training, and we’ve talked about this statistic. We’ve we’ve only leveraged I would say less than 10% of our raw content for continuous pretraining up to this point. We’ve we’ve worked with a company called Datology, you know, on this like continuous pretraining problem.

They’ve been a really good partner of ours there, and so we use the raw content really in CPT. The experts that I mentioned, 3,000 you know legal and tax experts, you know they have worked for TR for decades, and their primary job has been really organizing the law. They add metadata attributes, head notes, you know citation linkages, things like this, to the content to enrich it and make it easier to search and easier for lawyers to use for things like research and stuff like that. So we also are able to use a lot of the metadata that has been curated for many years to create like synthetic. So how did you

Ben Lorica. How did you take the universe of data and select the 10% Was it a random sample?

Joel Hron. No, we use our evaluations to help inform where we think the best 10% of data is going to come from to really optimize the model healthcare, you

Ben Lorica. Had specific use cases, so you were able.

Joel Hron. Correct. Yeah, I mean, like I mentioned, we built pretty robust evals across both you know legal tax as well as general domains, and so we try to be selective about our data. You know, the 10% figure is one that we’re working on. We think we can we can grow that perhaps and include more data over time. Certainly, that continuous pre-training phase is one of the more expensive like training phases of the model. So we try to be selective through that process. But we also create you know.

Fine-tuning data and RL data based on like preference judgments and other activities that are done by like our real legal experts that are employed within the company, and we create various tasks for them to help us curate that data.

And then the last step is really, I think, quite unique to TR, like I said, one of the one of the core ways that people buy software from TR is you know through our content products like Westlaw, Practical Law, and if you think about what those products are, they are really like search systems on top of like wide arrays of content and information, and as we’ve built our agentic products, what we’ve done is we’ve sort of decomposed those existing, you know, content software products into basically agentic tools.

In other words, like we’ve decomposed these software products into tools that an agent can use instead of a software UI that a human would use, and we’ve also trained Thomson to be able to use those tools, you know, more natively to accomplish legal work. So you mean

Ben Lorica. You mean if the task involves retrieval, then you already have the systems where Thomson’s

Joel Hron. Using our retrieval system to help like solve the problems, and that system

Ben Lorica. Is highly optimized. Could be a knowledge graph, could be a database, highly optimized for whatever question needs to be answered.

Joel Hron. Correct, and just like you know when OpenAI trains you know GPT or Anthropic trains Claude, they give it web search and they give it a harness with you know grep and glob and other like OS level tools, and the model learns how to use these much better. The same goes for Thomson. Like Thomson learns how to use our native content systems more intelligently and better through that training process, and we see that as a real potential source of differentiation for Thomson in the long term.

Ben Lorica. By the way, I don’t know if you’re comfortable sharing, but what’s the what’s the stack for the data pipelines and some of the training that happens after you get the checkpoint, and then you have the you have to build the data pipelines. Then you have to continue training. Is that built in open-source projects like Ray or what?

Joel Hron. Yeah, I mean, I can’t share too much there. I’ll share what we shared in our technical report, which you can find on Hugging Face. But we’ve partnered with Lambda Labs on the training infrastructure, and so we have our GPU infrastructure built there. They’ve been a really good. I wonder what they’re

Ben Lorica. Using Ray for distributed computing.

Joel Hron. Yeah, perhaps I’m not sure. I’m sure. I’m sure one of my engineers would love to geek out with you on that one day. But we use we use Lambda Labs on the infrastructure side for training. We use we use we’re partners with AWS as well as GCP and Microsoft and use a number of their services for like hosting and serving the model at inference today. And I’m trying to see what else I can tell you that. So in terms

Ben Lorica. Of in terms of the post-training, you mentioned RL, and you built your own RL environment. So anything else you want to share on the post-training?

Joel Hron. Yeah, yeah. I guess like really the two main components of that are what I said before, like traditional reinforcement learning, like DPO associated to like human expert judgment that is being applied over the course of many tasks, as well as this agentic RL loop where we sort of put the model inside of the harness with access to our tools and trained it in a loop. There, I think both of those stages are quite important.

Ben Lorica. So, so I’m not a user of any of your software, but I imagine there’s a chat interface. I chat, I type a prompt, and then based on what I have in the prompt, you have some routing mechanism. Sometimes it goes to Thomson, sometimes goes to something else, right? So is that?

Joel Hron. Yeah. So this is this is interesting. So we have Counsel, which is our product that we sell into the market, and we have Thomson, the model. Now, CoCounsel is a, as I said, like an agentic system that is built to solve sort of end to end all types. Legal questions and work, and workflows, and CoCounsel itself is actually built on the Claude Agent SDK, and we heavily leverage not just Claude, but also OpenAI models and Gemini models underneath the hood, as well as Thomson.

And so, as you say, you know, within that stack, there’s effectively like a sort of orchestrator agent that you know sort of handles the main dialog with the end user, and that operates inside of like a code like harness in a sandbox environment, and then that gets delegated then to other models or sub agents or systems to do that otherwise.

Ben Lorica. So you folks own that harness and that delegation step.

Joel Hron. We use Claude’s agent SDK today. You know we certainly have like variants that we’ve we’ve sort of tested on other harnesses as well as ones we’ve built, but today CoCounsel is primarily running on the Claude Agent SDK. So,

Ben Lorica. What percentage? I don’t know if you’re comfortable saying this. What percentage of inference is now going to Thomson?

Joel Hron. Yeah, I don’t. I don’t know that I can give you a precise figure on that, but I would argue, you know, between five and 10% at the moment. So you know, again, which is which is not

Ben Lorica. Which is not bad considering you both just are in the first inning of this thing. Yeah,

Joel Hron. Yeah. So the primary tasks that get routed to Thomson today are what we call tabular analysis as the feature. It’s really bulk document review. So anytime a user needs to sort of like analyze hundreds or thousands of documents and ask lots of questions of these documents, think of something like an M&A due diligence or something like this. You know, we use Thomson to power that like high volume review. The model is really good at it. It’s really well suited for like high volume sorts of activities, and so that’s the primary place today. And we’re continuing to sort of expand that over time.

I also like don’t believe that number needs to be 100% in order for this to be successful. Like I actually doubt that it will ever be 100% You know, like I think there are still places where, like, the frontier of intelligence, like, will never compete with, nor, nor, nor should we aspire to. You know, but we don’t have to be at 100% of token consumption for this, like, still to be really valuable and useful. And that’s kind of how we think about

Ben Lorica. It. But I suspect it’ll get higher than five to 10%

Joel Hron. Yeah, I would certainly expect it’ll get higher for sure. Like I would be, it definitely will. Yeah,

Ben Lorica. Yeah, yeah. So in terms of the Thomson now, so Thomson, is it fair to say, Joel, that one way to think of Thomson as it’s Qwen, but you know, obviously post-trained and customized for primarily legal at this point. Is that correct?

Joel Hron. Yes, but with one caveat, I would say Thomson is not like just Qwen customized for legal because the

Ben Lorica. Reason I describe it that way, I’m just trying to think in my head of the size. Yeah,

Joel Hron. Yeah, the

Ben Lorica. Size of the model.

Joel Hron. Yeah, yeah, that’s the appropriate size. But I think the important part for us is that like tomorrow Thomson could be GLM five three specialized for legal, and the next day it could be Nematron, and the next day it could be inkling, and like we’re gonna keep sort of evolving the, the sort of root model of Thomson as the open-weight community evolves, and keep that flywheel moving. That’s our main principle. But in terms of size, yeah, that like sort of four or 500 billion parameter range is kind of where we see the appropriate like Pareto frontier to be for the work we’re trying to do. Do

Ben Lorica. You imagine Joel that you folks will also start training smaller models for you know very specific things like if you determine hey for this highly specific thing we can get by with 30 billion parameters. I guess some of these models already do this kind of mixture of experts anyway, right? But it still seems like for certain things that are highly repetitive, that you know people are going to do a lot of. Maybe it’s overkill to have a model that can speak 2,200 languages.

Joel Hron. Well, so yeah, and we actually do this today already. So we do deploy SLMs into the stack, and Thomson like could be a great like source of distillation for SLMs even. So you guys then if you look at your portfolio,

Ben Lorica. I don’t know if you’re comfortable answering this, but what percentage of inference is going to some variant that open weights, not just Thomson?

Joel Hron. Oh, I don’t know if I could give you a good estimate of that, but like think about all the things that happen underneath an AI system, like just think about a search stack. A search stack is a series of re-rankers, each of which are like, you know, basically SLMs that are like doing some job of re-ranking, as an example.

Ben Lorica. Right.

Joel Hron. And so, like, there’s just a long tail of places where an AI model shows up that isn’t necessarily, but

Ben Lorica. It seems like the your usage of frontier models will decrease over time. I mean, just like a lot of the people I talk to, as they become much more adept at doing the things like you describe, it’s not that it like you said it’s not going to go away, but it might. It the trend line is pointing in the downward direction.

Joel Hron. Yeah, it’s hard to say. You know, at the same time, our usage of frontier models increases in the absolute, right? So even though maybe as a percentage of the whole it decreases, like the consumption of the models and the absolute increases because the demand is so high and the number of use cases that we can address is so high. So, I think I think those two go hand in hand. Like certainly, as a percentage of our token consumption, like we’re trying to grow Thomson relative to our reliance on entirely third-party models. Yes.

Ben Lorica. So Thomson. So Thomson does not yet excel in your other areas of paths and audits. It’s mainly legal.

Joel Hron. Mainly legal. And again, like our sort of bias in terms of training has been Around legal, primarily, not to say we, you know, we did use a lot of tax content and Reuters content in the in the pre-training phases of Thomson, but like in terms of the RL environments, it’s been more targeted at legal.

Ben Lorica. The reason I ask is because I have this very naive thinking around the tax domain, which is basically, I have the generally accepted accounting principles, I have the tax code, so I can I can create some sort of verification tool, right? Kind of like the same thing that why math is getting disrupted by you have lean or in the case of code you can compile or run the code. So in the tax thing, it seems like I have GAAP and then I have the tax code. I can create something that I can verify, right? Am I wrong? Yeah,

Joel Hron. I mean quantitative problems are traditionally more verifiable, and you know this is one of the interesting things about TR. Like, we have arguably the best tax engine in the world. Which, if you think about a tax engine, a tax engine is like a calculator for tax. It is the verification tool of tax. Now, the I think the nuance here is that it’s not just like one plus one equals two kinds of problems in tax. There’s a lot of like human judgment as to okay, you had this expense as a business. Well, how does that expense need to be treated? Like the tax law, so

Ben Lorica. That’s the GAAP not enough.

Joel Hron. There’s judgment as to like how should I treat this? How should I codify this particular item? There’s maybe certain opportunities that you could have to say, hey, look, I’m gonna run this under an LLC instead of like as a main part of the business, and like that would you know force the tax treatment differently, so there’s a lot of like interpretation and judgment. I’d say that goes into how a tax a tax return gets prepared.

With that said, we have like a really novel product that we actually just shipped not too long ago called Ready to Review, and it effectively does exactly what you just described, which is for an individual, let’s say it’s a 1040, take your like personal documents, whether it’s like your 1099s, your W-2s, K-1s, whatever it may be, and compute the tax return like in one shot with AI as well as like other models used for like extraction of data from the documents and things like that, and it’s a really good example of what you’re saying, where you’ve got basically AI sort of in this loop, like extracting data, putting it through a return, verifying outputs, and solving that problem iteratively.

Ben Lorica. Yeah, it’s crazy because especially for people who are salaried, it seems like they shouldn’t even have to file taxes, right? So in other countries, they don’t file taxes, right? So basically, the government just sends them here’s your tax return. Yeah, we have all your we have all your information basically, right?

Joel Hron. Yeah, I think our tax code is more complicated than anywhere else in the world, which makes that hard to do. But, but yeah,

Ben Lorica. Yeah, yeah. By the way, I have a pet peeve, which is basically the use of the term continual learning. In your guys’s case, continual learning means I took Qwen and then a checkpoint and then I added my own data and then I kept training. Right, so that’s one usage. But the more common usage actually is the following, which is basically I have a model and I use it in production and then I capture some signal. So yeah, this was wrong. This was not wrong. This was wrong. This was right, and so on. So then it then you can adjust the model so that you know over time without retraining it. So what does the model mean?

It could it just mean adjust the context, right? So the database where the rag hits, or the harness, or tools and the adapters, and then at some point maybe the weights, right? So anyway, yeah.

Joel Hron. So for us, we mean the former, not the latter. To be clear, like our customers are actually quite sensitive to how their data is used to improve a model or train a model, and so are we for what it’s worth. And so one of the one of the other benefits of Thomson, which we hadn’t talked about, is around security of the model and the data and how that ultimately comes together. And that’s one of the things that’s very important for our customers as well. And so when we say continual learning, we don’t mean this like continual retraining of the model on usage data.

What we mean is the fact that we can take an open-source model, apply our own data to it, and add to its knowledge, not degrade its knowledge in some areas and improve it in others. Like it’s really speaking to this point of catastrophic for continual

Ben Lorica. Learning doesn’t have to violate privacy. It just means that within a customer, right? So the model just learns over time without retraining from scratch. All right, so let’s see. I guess. What are the next steps at this point? I mean, next step could be taking what you have now and forging ahead, or yeah, you can take another open-weight models, one of the ones you listed, and then start the process over. What’s more likely to happen?

Joel Hron. Yeah, I would say both. You know, so we are. That’s four. That’s

Ben Lorica. $$450,000 each, right?

Joel Hron. They are, yeah, yeah. But I mean, we don’t we don’t go through full training runs on every open-source model just to figure out which one’s best. Like we run our evals like you know, sort of. You can

Ben Lorica. You can abandon midstream,

Joel Hron. Right? And even before we start training, we know which candidates are most likely to be successful, and so we will sort of pick one of those and retrain a variant of Thomson relatively soon. I would I would expect, but at the same time, we’ll also sort of forge ahead on the development of Thomson as it relates to more use cases in CoCounsel legal the product, and so like I said, like you know, drive that token consumption number up from like five to 10% to higher and higher as Thomson starts to become the best model for other use cases.

So that’ll be the second sort of area of focus, and then third, we’ve got a lot of interest like direct inbound from customers who are building a lot of their own a lot of their own tech in house who have interest in using the model directly for their purposes as well. So, I think we’ll we’ll be exploring with them what like serving of the model looks like and how they want to consume it and things like that. So those are probably you. You would be

Ben Lorica. You’d almost be like an API endpoint for your customers.

Joel Hron. Yeah, I mean, a lot of customers are quite interested in using the model in this way. So, so definitely something we’re exploring.

Ben Lorica. And in terms of, I guess one other direction as you mentioned earlier, is to just increase the data.

Joel Hron. Correct. Yeah, as well. I mean, that’s another experiment that we’ll run is try to understand where the point of diminishing returns there is, and figure out what’s optimal. Yeah,

Ben Lorica. Yeah. Whether at some point you hit some sort of saturation.

Joel Hron. Yeah, I would expect we will. I mean, we’re not 100 of data is unlikely, and that sort of continue to drive the model. So, it will be something less than 100. It’s hard to predict what that is, but we’ll we’ll try to figure that out.

Ben Lorica. And by the way, just to get our give our customers a feel for in the legal case, in the legal domain, is it mainly search retrieval? What are they doing?

Joel Hron. Well, I mean, if you look at CoCounsel today, it’s everything. So, like,

Ben Lorica. And like you said, with the accounting thing, legal also. Is there some subjectivity in legal as well.

Joel Hron. Oh, tons!

Ben Lorica. Yeah,

Joel Hron. Tons. Like, I mean, just by virtue of the fact that like you litigate,

Ben Lorica. Yeah,

Joel Hron. Would suggest that there’s subjectivity. Meaning, like, what is a litigation? It’s a it’s a case where two arguably very experienced attorneys or teams of attorneys are arguing the same circumstances from two polarly opposing points of view. So, like, absolutely, there’s subjectivity there because, like, you’re interpreting the law and you’re trying to use historical legal judgments and cases and statutes and regulations

Ben Lorica. To

Joel Hron. Build that argument.

Ben Lorica. For those of us who aren’t familiar with the legal domain, it’s basically so. So the AI is it basically like a first year associate lawyer at this point?

Joel Hron. Yeah, I mean we’ve tried to use those analogies and like say, oh, this is like a paralegal. Now it’s a first year associate. Now it’s a senior associate. You know, I’ve had customers say that they feel like CoCounsel operates at like a senior associate kind of level now, but it handles work across the spectrum of legal. So, so that could be like preparing legal research memos. It could be drafting first versions of briefs for motions to dismiss and things like that. We released a product called Brief Builder a couple weeks ago that supports in this product or in this process.

It could be things like negotiating a contract relative to like a playbook of terms I’ve re agreed on. It could be developing the playbook in the first place. It could be, you know, pouring through thousands of contracts in an M&A deal, you know, trying to find like key points of risk.

Ben Lorica. But then it’s still obviously highly recommended that a lawyer reads it at the end, right? Because just in case there’s hallucination.

Joel Hron. Well, absolutely, and it’s not just that the lawyer reads it at the end, but like also that the lawyer engages in like the process by which the AI took to sort of get to the end in the first place, and like I think the way that we build the product in the first place really sort of elucidates to the human expert like how that process was reached, and I think at the end of the day, like just like in if it’s an engineer shipping code with a coding agent, like the engineers are accountable for what goes out into the world, you know, and so the appropriate level of sort of oversight and judgment needs to be applied there.

Ben Lorica. So, so the lesson in coding and math is the following, which is we’re in an era where generation is cheap, right? So you can write code or you can prove theorems. The verification is doable in both domains. You’ve got lean and you’ve got compilers and what whatnot in coding Acceptance means in math you write a paper, hopefully in a way that human mathematicians can understand and sign off on. Maybe the right now that’s not quite there. The papers tend to be impossible to read, but I’m assuming that’ll get there, but then the final step is, you know, the credit, right?

So who does the mathematician give get credit? So one proposal from mathematicians is, if Ben or Joel created the paper before we give them credit, maybe they should give a talk in front of mathematicians because what if they’re incapable of giving a talk about the so-called paper that they wrote, right? So I wonder in the legal case. So generation is cheap is getting cheaper. Some sort of validation, maybe not. There’s no way to formally verify these things, right?

Joel Hron. That’s the that’s the crux of legal. Like verification and validation is the hard hardest part of building any. I’m

Ben Lorica. Assuming you have heavily hyperlinked citations in whatever you generate, right? We

Joel Hron. Yeah we absolutely do, but like verification is. Just verifying does this case exist? You know, I make you know an AI writes a claim, and it says this case supports this claim. That could be a legitimately real case, but like nothing to do with

Ben Lorica. The claim,

Joel Hron. Or that, or it has everything to do with it. But the claim is like maybe just slightly hyperbolic

Ben Lorica. To

Joel Hron. What the case said, or it extended sort of some judgment in like a little bit too far-reaching of a way. Like those, that’s the nuance of judgment that legal experts provide, and that’s where AI systems are tough today. And that’s what we focus on when we build our products, is really sort of like trying to do that as well as we can. But that’s the that’s the part where I think the human judgment really comes into play, and why we why we really you know encourage our human interest to get involved.

Ben Lorica. I guess playing devil’s advocate, what if you get so good at it, Joel? Maybe I start checking out, tuning out, right? So, in other words, you know, I read 100 of the briefs coming out of this model. They’re all correct. The 101st, maybe.

Joel Hron. Yeah, I mean, I hope that doesn’t happen. I mean, you certainly see it sometimes, but like, I think I think the risks, you know, to lawyers, like, and what we say is like fiduciary professionals is just, you know, really like too high to like suggest them checking out. But I would say like their mind does move towards like higher order things, which is like framing of arguments differently or like you know, kind of like hardness testing, or like you know trying to challenge their ideas more aggressively before they bring it to court.

Like, I think that you know my belief is that it elevates their thinking and allows them to like be a lot more like strategic in how they prepare a brief, rather than just focusing on the writing of all the words that need to go into the brief in the first place. And I think that’s what we hear from our customers is that’s that’s how they benefit from it for sure.

Ben Lorica. So it seems like a lot of the things that a first year associate is doing, you guys are able to do now and well. And you know, just like in coding or in math, right? So usually you give your grad student or your junior developer some easy projects so that they can learn. So, but we’re taking that off the table. So it’s almost like it’s almost like we need to figure out a way to train the next generation. If a lot of the a lot of the ways they used to learn will get automated, right?

Joel Hron. Yeah, I get asked this question a lot, and I do think this is a problem, kind of like across all industries, as these sort of like easier tasks get automated. Like, how do I train junior people to sort of like climb the stack? And one thing I would I would sort of like offer as a point of view here is that while you might be taking off some like grunt work that a first year associate or like a entry level engineer would have done before, like think about how many more reps they are getting now than they ever would have gotten before.

Like before, for an entry level associate, maybe they do like one big deal every six months because they’ve got to read through all the contracts. Now they could literally sit on a half dozen or a dozen deals in a matter of six months, and they see such a broader range of contexts than they could have before, so much faster. So they might be losing something, but I think at the other end they are gaining something, and they’re gaining a lot more repetition and a lot more breath than they would have gotten, you know, and hopefully doing that much faster.

Certainly that needs to be balanced, but I think I think there’s a positive aspect to this too.

Ben Lorica. All right, so in closing, I’m going to run this idea buy again, buy again because you didn’t seem to buy it, but I’ll I’ll give it another try. Which is the following, Joe? I think open models will win in the following sense that they will garner more of the compute. Obviously, the frontier models will continue to prevail, especially in complex tests. Right. So if you’re doing scientific research or proving really advanced math theorems, frontier models are. But over time, more and more of the routine work is going to go over to open-weight models.

Joel Hron. I think I believe the way you framed that statement is that more of the routine work will go to open-weight models as it becomes easier to orchestrate and manage them for sure.

Ben Lorica. And that and. We don’t know. There might be a breakthrough. I’ve been talking to people who’ve been working on post-transformer architectures. We don’t know. Maybe at some point the models won’t be as gigantic. We can run them on laptops. Yeah, I

Joel Hron. Mean, Jeff. Dev’s quite interesting. I don’t know if you saw Jeb recently. Like that’s another example of like a sort of

Ben Lorica. Basically, basically, Bert. Yeah,

Ben Lorica. And then just not the generation

Joel Hron. Side.

Ben Lorica. And then machine learn, and then the AI people rediscovered machine learning, basically. Yeah,

Joel Hron. Yeah. I mean, it’s basically just like not everything. Non generative tasks.

Ben Lorica. Yeah, yeah. Not everything needs a chatbot.

Joel Hron. So, so I mean, I think I think there’s a lot of like opportunity, you know, to be to be had here. But the one thing I would, the one like sort of wrinkle I would put into your statement is that like, just because open-weight models become more of a default for routine tasks doesn’t mean that like Frontier Labs don’t win. Like I think there’s still like a lot of very complex problems and complex orchestration problems that need to happen with those models.

Ben Lorica. A lot of the open-source community is not just the models, right? You’ve got the harnesses, you’ve got serving software like vLLM, you’ve got distributed computing frameworks like Ray. So it seems like the momentum is the you’ve got pieces in place. So in fact, when I talk to people about this, because I was around for the.com, right? So when you were during the.com, I mean, obviously, people, everyone will was using Solaris and Oracle, right? So, but because the perception was Linux wasn’t good enough and the open-source databases weren’t good enough, they were several years away.

But then a few years later, they took over. But now with the open-weight models, we’re not even talking about they’re several years behind. Right. So there are few. months behind, right?

Joel Hron. Correct, and this is why I think the harness layer is important as well, just because you know I think the Frontier Labs benefit from co-training their models within their harnesses, but like you know, for an enterprise, like optionality is really important, and being able to pick the best models for right jobs is important, and like I think I think the open-source community around harness development is really important. I applaud OpenAI for open-source and Codex Harness as well. The

Ben Lorica. Title of your paper, Sovereign AI, is another reason, right? So you have control over your roadmap and your, and your destiny, right? So, yeah, yeah. And with that, thank you, Joel.

Joel Hron. Thank you, Ben.