Ben Lorica and Evangelos Simoudis on Anthropic’s $800 Billion Bet, SpaceX’s AI Ambitions, and Regulating AI Labs.
Subscribe: Apple • Spotify • Overcast • Pocket Casts • YouTube • AntennaPod • Podcast Addict • Amazon • RSS.
Ben Lorica and investor Evangelos Simoudis discuss the opportunities and risks revealed in Anthropic’s S-1, from explosive revenue growth and improving economics to mounting compute commitments, competition from open-weight models, and dependence on coding. They also examine SpaceX’s emerging neocloud business and whether access to cheap capital and growing demand can sustain it. Finally, they debate proposals for regulating frontier AI labs, including restrictions on AI self-improvement, independent testing, and mandatory disclosure of security incidents.
Interview highlights – key sections from the video version:
Related content:
- A video version of this conversation is available on our YouTube channel.
- Anthropic’s IPO: The Business Is Real. The Hard Part Comes Next
- SpaceX: The $2.1 Trillion Compute Landlord
- Previewing the Debate Over Frontier AI Regulation
- Previous conversations with Evangelos Simoudis can be found here.
- Evangelos Simoudis blog
Support our work by subscribing to our newsletter📩
Transcript
Below is a polished and edited transcript.
Ben Lorica. All right, so we’re back with my friend Evangelos. His blog is at corporateinnovation.co. We are recording this on the morning of October 30, 2026. Episode notes, including all the resources we may cite, are on The Data Exchange Media. Please hit the subscribe button if you’re on YouTube, and subscribe wherever you get your podcasts.
So, topic number one. This is a big week in AI. Anthropic released their S-1, right? And 134…
Evangelos Simoudis. Megabytes.
Ben Lorica. Yeah, yeah. And what I did was basically look through it on the day of, and just pull out things I liked and things I didn’t like. But I haven’t really read much, so this is the first time I’m going to bounce some ideas off someone else.
So, let’s start with things I like. First one, Evangelos, is the growth. Growth seems hard to fake.
Evangelos Simoudis. Exceptional. Very good. I agree.
Ben Lorica. And I don’t think there’s much to argue about. The growth numbers are out of this world, right? They’re insane.
Evangelos Simoudis. We also need to be watching how that growth continues. But I agree with you. Great, great numbers.
Ben Lorica. And the second item I liked, which, in retrospect, I kind of just put in my post, but I don’t know if you’ve had a chance to vet what I said, is that the unit economics seem to have improved. At least in the S-1, they said it used to be 40 cents of revenue per dollar spent on compute, and that figure was now $2.08.
Evangelos Simoudis. Yeah, I agree. I think we will need to see where the gross margins go, because that was another very nice surprise, if I can call it that. And I think as people scrutinize what’s involved, what’s included in that gross margin, and how they can continue that, it could be another nice surprise if these numbers stick.
Ben Lorica. So, for listeners who may find this surprising, in light of the fact that you’ve got these open-weight models, isn’t there price pressure? How come the unit economics got better?
Evangelos Simoudis. I don’t think that we have seen it yet. Remember, the numbers that they report go through the first half of 2026. I’ll speak now for what we’re seeing in the enterprise. I don’t think that the open-weight models have had the effect yet that we expect to see going forward.
Ben Lorica. And then, I don’t know about you, but I still detect a reluctance on enterprises’ part toward Chinese open weights.
Evangelos Simoudis. That is true. I will say that there is no question that there is a big movement afoot to decrease costs. I think the first step is, what can we do with smaller proprietary models that cost less and give us the expected response, the expected performance?
But I also think that a number of them are starting to experiment with open weights, including Chinese open weights, which they want to have more on-premises, as opposed to having data moving out.
Ben Lorica. So, item three that I like. It seems like the cash burn is not as worrisome as I would have thought. They have raised $141.1 billion, which is insane, but they still hold $131.7 billion in cash and short-term investments.
Evangelos Simoudis. Yeah, I think that they have done a very good job in cash management and pushing some of their expenses toward warrants and debt, which I think is an interesting approach. With interest rates rising, and if they continue to rise, that may come back to bite them. But they do have enough cash, and that’s where the performance of the company will play a big role going forward.
Ben Lorica. So, the next item I filed under things I like, but it’s a bit of a mixed bag. It seems like adoption is now broadening, right? No single customer accounted for more than 10% of revenue. The top five customers’ share fell from 36% to 21%, and the number of customers that spend at least $100,000 grew from 950 to close to 12,000.
But the thing that is a bit weird here is that 48% of revenue still flows through the cloud hyperscalers. I guess it’s not weird. That makes sense. They’re also happy to compete with them.
Evangelos Simoudis. Exactly. I think that’s both a like and a dislike, in a sense. There’s a significant risk there. Look, the customer counts impressed me. I have to say that I was not expecting this type of broadening.
What I’m watching for, vis-à-vis what we just spoke about, is whether these customers will continue spending the same amount, number one, or whether the use of smaller models, the use of open-weight models, will decrease that.
Number two is whether that consulting operation Anthropic has set up will enable them to continue penetrating the lower strata of the market.
So again, these are things both to like and not like. And as you may already have mentioned in your piece, Meta and Microsoft, which were big users, are starting to pull back on their use of Claude. So that could also be a concern going forward.
But again, the numbers they posted in terms of customer counts are extremely impressive.
Ben Lorica. Yes, and then let’s see. What’s next on my list is that revenue per employee has also grown quite well, right? Over the last 12 months, that went from $1.4 million per employee to $6.6 million, which, as you just mentioned, I guess the more forward-deployed engineers they have, might that number slow down or even decrease?
Evangelos Simoudis. Actually, what I thought was impressive about this number is, first of all, I think it sent a clear signal to more conventional enterprises of what an AI-first company can achieve. So that’s a big point there.
The second one, I think it sent a signal to software companies, including SaaS companies, which definitely have better economics per employee than traditional enterprises. But the kind of number that Anthropic has posted is just the next level up.
So that is going to call into question what companies like ServiceNow and Salesforce and Workday and the like can do. And also, obviously, if you go further down to more traditional software companies, SAP and so on, that will have an impact. They’ll have to rethink how they improve their numbers because now there is a bar to cross.
Ben Lorica. And so, before I go to the things I don’t like, are there things you like that are not on my list?
Evangelos Simoudis. No, actually. To me, it’s interesting how software development has pulled the business along. It tells you what we can do with AI and the right AI. We’ll see how it translates to other tasks, but again, so far, so good. Everything has been extremely impressive. Go ahead.
Ben Lorica. So I guess there’s one other thing on my list that I like, which is a bit of a mixed bag. I’m just going to mention it here before we dig down later in my list of negatives.
Basically, for a long time, my impression was that a lot of the revenue came from API calls, right? On Amazon. But I guess increasingly, a lot of the revenue is going to Claude Code.
So at least there’s a sign that maybe there’s some notion of a product coming to fruition, coming alive somehow, which gives them maybe an opportunity to go beyond Claude Code. Maybe there will be Claude Legal, Claude Finance, and, you know, I think they’re…
Evangelos Simoudis. Trying for that.
Ben Lorica. That would be the…
Evangelos Simoudis. Yeah, I think…
Ben Lorica. It’s a bit of a nascent signal that I don’t want to oversell.
Evangelos Simoudis. Look, there’s no question that the product-market fit in software has been unbelievable, and it opened the eyes of many of their competitors and the whole world in general.
Whether they will be able to, and this is a statement not only for Anthropic, but we have OpenAI coming right behind them on the IPO scene, whether they will be able to create product-market fit for other areas, you mentioned legal, it could be finance, it could be financial operations, in other words, insurance processing, that remains to be seen.
And it’s also something that you mentioned in your piece, which I really like, which was that in software, success makes you want to do more with it, right? We want to write more software. That does not translate to every other process.
And so even if they find product-market fit in another area, it doesn’t imply that we will see the same kind of curves that we have seen with software.
Ben Lorica. The token burn.
Hey, before I go to my list of negatives, we briefly mentioned that revenue per employee is growing, and then you mentioned SaaS. What happened to the SaaS apocalypse?
Evangelos Simoudis. Actually, based on the quarterly results that companies are starting to announce for the third quarter, we’re starting to… I won’t call it SaaS apocalypse two, but there are certainly some worries.
So if you look at ServiceNow, if you look at Salesforce, and we’ll see where we come out by the end of the reporting season, I think some of the worries have come back. Maybe not to the extent that we were thinking about, or that the community was thinking about early on, but certainly it has not gone away.
Especially because of the thing that we just mentioned here, that there might be other domains where you can find good product-market fit, even if you may not find the acceleration that you found with software development.
Ben Lorica. But is it just my pet peeve, Evangelos, that the media kind of went nuts on the SaaS apocalypse as a headline?
Evangelos Simoudis. I think they did. We’re always trying to find blemishes and trying to knock things down.
But look, there is no question that many companies are starting to re-examine what percent of a packaged application they are actually using, and whether they can replicate certain features on their own without having to pay these huge licenses.
Now that they’ve gotten the bug that they can develop software very quickly and very painlessly, if I can call it that, that puts pressure on the SaaS application providers.
Ben Lorica. By the way, speaking of this topic, I can’t name people, but I talked to a very high-level person in one of these publicly traded SaaS companies. And the comment was, you have all these people on LinkedIn saying, “We’re getting off your software because now I’ve discovered I can do this and that.”
And then when you look up whether the person is still using the software, yes, in fact, they’re using it quite heavily.
Evangelos Simoudis. They are. And look, the fact is, and I say this a lot to our portfolio companies, these applications, whether they’re on-premises or SaaS applications like Workday, ServiceNow, Salesforce, they hold a lot of data now.
And so even if you don’t use the business logic, if I can call it that, part of the application, they’re not going to give you that data that they have now in their warehouses for free.
And we see Salesforce already starting to make moves in that direction, charging for access to the data even if you’re not using their application.
So that’s why I’m saying that I don’t think we’re going to have a SaaSpocalypse in the way that it was envisioned, but we’re definitely seeing certain implications from the success of the AI tools.
Ben Lorica. All right. So, things I don’t like. The bear case, right?
First of all, the commitments for compute. $800 billion over the next 10 years. At least they don’t use the word trillion, because one of my pet peeves in this era, Evangelos, is people throw around the word trillion right and left, as if a trillion is such a trivial thing.
Evangelos Simoudis. Yeah, yeah. Well, I mean, we’re starting to use the U.S. debt, so why not?
Look, I agree with you, and I think what our listeners need to consider here is the fact that, on the one hand, we have these commitments. On the other hand, we don’t know what will be deliverable from the data center creators. So there is a lot of fudging. There is a lot of gray area there.
Ben Lorica. But one caveat, based on my reading, is that most of this can’t be canceled?
Evangelos Simoudis. I think probably something like 60% to 65% is noncancelable, but there is a piece of it that is cancelable.
Ben Lorica. What, 65% of $800 billion? Yeah. Okay.
Evangelos Simoudis. I mean, it’s still a big number. I’m not trying to dismiss it. I’m just saying that I agree with you, but if you want to be very precise, there is a little bit of a silver lining.
Ben Lorica. Yes. All right.
So the next thing I don’t like, I guess, is the training treadmill, right? Because basically, I think it’s still expensive to train these models, and you are under the gun to keep releasing them.
Because imagine you’re Anthropic or OpenAI and say, “You know what? We have enough models. We’re going to take the next year off, or the next six months off, and just help our customers use our existing models.”
I don’t think that’s going to be good for their brand or marketing, right? So, in other words, they’re on this treadmill where they have to incur all of these expensive costs around training.
Evangelos Simoudis. Yes, and…
Ben Lorica. By the way, before you continue there, if you talk to anyone in enterprise, they’ll tell you quite frankly, “Look, if they don’t release another model for the next six to 12 months, we have more than enough to do.”
Evangelos Simoudis. Exactly. I think that what you’re seeing is that the enterprise is not switching models as rapidly.
There is interest, as we talked before, in decreasing the cost, and I think these companies have been doing a good job in decreasing the cost per token.
But I don’t think that we will see, especially as we move from experimentation to deployment, this type of model switching that has been assumed. Obviously, every application gets updates, but it’s not going to be at the pace that is anticipated.
So I think that will slow down, and the question then becomes, what’s the moat for these companies?
Ben Lorica. Right, right.
And so again, the third item I don’t like is that prices are heading down, and you can see this through all sorts of third-party data, right?
The Chinese labs are producing tokens at an industrial scale, releasing their models as open weights, and I think in the next six to 12 months we’ll have respectable Western open-weight models.
Nvidia is starting to buy new labs to beef up Nemotron. Reflection has an open-weight model. Thinking Machines will probably continue to release open-weight models. And then I know of some initiatives that are going to do kind of a full open-source model initiative.
So I think the pressure from some notion of open is real. And people always say, well, the open weights are just distilling the…
Evangelos Simoudis. I think that’s a big issue.
Ben Lorica. The proprietary models, right? But then as long as the proprietary models keep improving, the open-weight models are going to improve.
So the analogy that I’ve come to like is water skiing, right? A water skier is being pulled by a speedboat, so the water skier can go as fast as the speedboat, or maybe even faster, but can never get ahead of the speedboat.
Evangelos Simoudis. Yeah, that’s right.
Ben Lorica. Right, that’s…
Evangelos Simoudis. Right.
Ben Lorica. So, you know, prices are heading down. I think that’s going to continue. Am I right to consider that a negative?
Evangelos Simoudis. Yeah, I think it’s kind of what I was saying before. We will have to see how their revenue growth, starting with Anthropic, but also going to OpenAI, which will follow quickly behind them on the IPO market, continues.
I think we’ll have to see whether that revenue growth continues to be as impressive as it’s been through the first half of 2026.
Ben Lorica. So another negative is coding is disproportionately carrying the business, right? Some estimates place it at 70% to 80%.
And you’re a former VC. Isn’t it an adage in the VC world that your first customers are usually not representative of the broader market?
And in this case, as you point out, coding is expensive. People start coding. They write more code. They use more tokens. But a lot of knowledge work is compressive. I want to summarize a talk. I want to read a bunch of papers, but only give me bullet points, right?
Evangelos Simoudis. Yeah. So look, I think that, as I said before, identifying coding as a use case was brilliant. I don’t know if it was an accident or if it was deliberate, but it was brilliant product-market fit, and that has allowed not only Anthropic but also other companies to be successful.
Identifying additional such use cases will be very important for the future. Otherwise, I think we will see some very unexpected outcomes.
Ben Lorica. So on the coding front, I guess because, as you said, they identified the market, now that’s more competitive, right? Everyone is going into coding. I mean, OpenAI has a lot of revenue in coding as well.
I happen to use OpenCode with OpenRouter, so I can switch models. But the general impression I get is that most people settle on Claude Code, and maybe they don’t really move off it quite as easily as I would think.
But that’s just anecdotal data on my part.
Evangelos Simoudis. Yeah. Actually, I would put it both during experimentation… I mean, if you look at enterprises that are experimenting with AI, and most of them do, including many of these 12,000 that they list as customers, during that phase there is a lot of switching between models, between tools.
But once you start moving forward and you start going to deployment, you’re making some decisions. Then the switching, whether it is at the model level or the application level, goes away because the enterprise cannot afford it.
So this is something that I think this operation by Anthropic and these consulting partnerships they have could be extremely beneficial for them.
Because if an organization adopts their tools for real, it will be very difficult then, and we’ve seen it over the years in information technology, to dislodge them. And that then becomes their moat.
Ben Lorica. There is one upcoming trend, which will happen in the next few months, early next year probably, and that’s that a lot of people are working on shared memory, shared context.
One of the things is that when you use a coding tool and then try to switch to another, you lose a lot of the context.
Evangelos Simoudis. Yeah, a lot of context.
Ben Lorica. A lot of context and memory. But a lot of people have noticed this and are upset about it.
So I know of at least one open-source project that I’ve actually been quite impressed by, which is going to release something early next year, and this will solve your switching costs, I think.
Evangelos Simoudis. Yeah. So again, I think we need to keep monitoring what’s happening in the broader community, as opposed to just in one of the new labs. Go ahead.
Ben Lorica. All right. So one thing that seems to be a trend whenever these tech companies go public is that the co-founders carry control through a class of shares.
In this case, the seven co-founders control 50.1% of the vote, which I thought… Their brand is, I guess, “We’re going to shepherd AI to safety, so we’re the right seven people to do that. So we should have control,” right?
Evangelos Simoudis. I agree with your concern. I mean, the way I put it is that the public shareholders buy economics, whereas they do not buy control, and that is very different from what we’ve seen in…
Ben Lorica. In the past.
Evangelos Simoudis. In the past, exactly.
Ben Lorica. Yeah, yeah, yeah.
What about, so I brought this up earlier, but partners who are also rivals, right?
Evangelos Simoudis. Exactly. I agree, I agree.
Ben Lorica. Particularly Amazon, Microsoft.
Evangelos Simoudis. Yeah, exactly.
Ben Lorica. Right. So I…
Evangelos Simoudis. Agree.
Ben Lorica. That’s a negative. All right. So do you have any negatives?
Evangelos Simoudis. I would say the dilution component to me is important because they have certain milestones that, if they’re reached, they are able to issue more stock. And also they’re assigning stock to philanthropy, and that can lead to significant dilution.
Again, I’m not against philanthropy. I’m just saying that from a shareholder perspective, that can lead to significant dilution.
And again, you can put it as a negative because we always think of dilution as a negative thing. It’s definitely, I would say at the very least, very different from what we’ve seen in past IPOs.
Ben Lorica. All right. So, second topic is a quick one. In my last newsletter, I posted a cheeky graphic where I said SpaceX is becoming a neocloud.
Evangelos Simoudis. Ben, one more thing. I’m sorry. I apologize for breaking the rhythm. Sales and marketing rose by 26 times. Revenue rose by 14 times. To me, that was also a negative. Let’s move on.
Ben Lorica. All right. So topic number two: Is SpaceX becoming a neocloud?
Well, according to their earnings, that seems to be the case, right? AI contributed 431 in operating profit, but they spent $16.4 billion.
And so, what does this mean in terms of things moving forward? I identified three things. Let’s see if you agree with any of these, Evangelos, or probably you’ll agree with all of them, right?
One is, for this to work out for SpaceX, they need to have continued access to cheap capital.
Evangelos Simoudis. Yes.
Ben Lorica. And in fact, I don’t know if you noticed, but Elon tweeted something where there will be a bifurcation in neoclouds. Those who can access cheap capital and those who can’t.
And those who can access cheap capital usually have another business, meaning in his case, Starlink and all of that. And then the pure neoclouds are screwed, basically, is what he…
Evangelos Simoudis. Yeah, I agree with that observation. But I will also say the following: SpaceX is getting into too many businesses. I mean, you saw that they bought spectrum. Now they want to be a telco.
Ben Lorica. And they may become a car company.
Evangelos Simoudis. Exactly. So to a certain extent, I think they are spreading the risk compared to, say, Anthropic, where they have one business and they have to succeed.
Ben Lorica. Or I guess sticking with the neoclouds, compared to CoreWeave.
Evangelos Simoudis. Or compared to Corlanda or any of those guys. But on the other hand, each of these businesses requires significant capital.
I mean, TerraFab needs $100 billion. To set up a proper telco, you need, I don’t know, $30 billion to $50 billion. These are billions, right?
So again, I don’t envy him for having to make certain decisions. Yes, it spreads the risk, but also he’s getting into businesses that are very capital-intensive.
Ben Lorica. And by the way, I think in his S-1 he identified the addressable market as $40 trillion.
Which, by the way, I just heard an interesting stat. From 1990 to 2025, the two largest software companies in history, Microsoft and Apple, combined, their total revenue was not even $10 trillion.
But Elon somehow is going to get to $40 trillion, right? So whatever…
Evangelos Simoudis. Works.
Ben Lorica. All right. Number two that needs to go right. You need to keep growing with solvent customers.
But right now, their neocloud customers are Anthropic, not profitable; Reflection AI, not profitable.
So you need to get to enterprise customers who are willing to pay, lock them in for long-term contracts. But it’s a commodity business. It’s not like you can charge premium pricing, right?
Evangelos Simoudis. Look, I believe that SpaceX got into the neocloud business not by design but by accident. They had the capacity. Good for them that they managed to have the capacity to create that architecture around Colossus, and all of a sudden they came to market and said, “We have that capability.”
Ben Lorica. And the reason they have capacity is he wanted to build models, right?
Evangelos Simoudis. Exactly, exactly.
And now there are a lot more. By the way, a lot of companies, even Meta now is talking about, “Hey, we have capacity too because we have access to this data center.”
So I’m very concerned about where that is going. On the one hand, I’m concerned about whether we will be able to fulfill all of the promises that data center developers have made and bring up the capacity.
But on the other hand, there is quite a bit of capacity that is becoming available, and whether it will continue to be filled profitably, that remains a question.
Ben Lorica. Here’s another source of capacity that I’m imagining, and tell me if I’m crazy.
Look to next year. You have all these new labs that raised so much money, that bought compute in advance to lock it in. Now they have to show growth to raise their next round. Well, I can show growth. I can sell my capacity.
Evangelos Simoudis. Yeah, exactly. And that’s exactly what SpaceX has done, in my opinion, with their data center.
Ben Lorica. So, all in agreement. Do you think this will happen? These new labs may share capacity next year?
Evangelos Simoudis. I actually think, to me, what is more important, and I say this as an investor, is exactly what you said just before that.
Now these companies will come to their next round of funding, and they will need to show that they have been meeting the claims that they were making before.
And I think a lot of investors, both existing and prospective, are going to be disappointed at what will be posted. And I think we will have a crisis around that.
Whether Nvidia will be able to backstop everything, I do not know. But we will definitely have a…
Ben Lorica. But right now, VCs only value growth. So here’s how I would play the game, right?
I’m a new lab. I have all this capacity. To show growth, I’ll sell my capacity, but then I’ll also release a small model to show, “Hey, look, we built a model, and this is just the first. We’re going to build more, but in the meantime, look at our revenue.”
Evangelos Simoudis. Yeah.
Ben Lorica. Right.
Evangelos Simoudis. Yeah, I do not know. I guess I’m getting too curmudgeonly to make optimistic statements these days.
Ben Lorica. Which, by the way, leads me to the third thing that has to go right, which plays into everything we’ve just been talking about.
The spot prices for compute capacity in these neocloud data centers have to continue to go up.
Evangelos Simoudis. Continue to go up, yeah.
Ben Lorica. And what do you think?
Evangelos Simoudis. Whether they will continue to go up? I mean, a lot of it depends. Right now…
Ben Lorica. Right now it’s going up, right?
Evangelos Simoudis. Right now it’s going up because there is more demand for all of this experimentation than there is supply.
I expect that next year, toward the end, if everything goes as planned, we will start seeing much more equilibrium than we can today.
Ben Lorica. All right, topic number three.
Obviously, the elections in the U.S. are on Tuesday, and it’s widely anticipated that the Democrats will gain control of at least one house of Congress, maybe both.
And if that’s the case, then discussions around what to do about AI may grow more intense.
And even if they fall short, and let’s say they don’t win either house of Congress, the belief is that in the 2028 election cycle, AI and how to regulate AI will be some sort of litmus test on the Democratic side of the nomination process for president.
So with that said, I’ve started to gather some proposals that are up in the air, let’s put it that way, from the Democratic side of the House.
I’m going to run them by Evangelos just to get his quick reaction, and then I’ll react too.
The first set of proposals centers around self-improvement and the development of models.
Proposal one: Maybe we should pause recursive self-improvement. Yeah, pause recursive self-improvement.
Evangelos Simoudis. I think that I agree with that. I think we need, at the very least, to study it a lot more and try to understand it a lot more than we do today.
Ben Lorica. Wow! I am shocked because my reaction to that proposal is that this seems a bit crazy in the following sense.
First of all, you have to define for me precisely what recursive self-improvement is, because…
Evangelos Simoudis. That’s what I said.
Ben Lorica. Because if I’m using AI to code, is that recursive self-improvement? So someone has to…
Evangelos Simoudis. It depends what you’re coding, though, right?
Ben Lorica. Yeah, yeah. If I’m coding the model, right? Or is it only if I let the AI run wild and create itself?
Evangelos Simoudis. Yeah.
Ben Lorica. You know what I mean? But if you’re restricting me from using AI to even work on the next-generation model, that’s impossible. So how is that even enforceable?
Evangelos Simoudis. Well, actually, it comes down to what I said and what maybe you better elaborated, which is that we need to define what we mean by recursive self-improvement.
Because today we’re starting to use that term, but there are many interpretations of the term. And the more we understand it, I think we need to refine those interpretations.
So that’s why I made the statement that I did.
Ben Lorica. Yeah, yeah. I’m a little wary of that.
The second thing around self-improvement is this: Use only vetted models to develop successors.
Evangelos Simoudis. That makes sense.
Ben Lorica. Yeah, yeah. So here’s my pushback to that.
Who’s going to certify the models, to vet them? First of all, that seems to be a foundational step.
And I may be using an internal model that hasn’t been released to the public yet to do self-improvement. Is that allowed, or am I only supposed to use public models that have supposedly been vetted?
And by the way, we don’t have any vetting process right now anyway.
Evangelos Simoudis. Yes. So I think, Ben, as we find with every new technology and government involvement, the people in the government who want to be involved are behind where the technology is. That’s an adage that everybody accepts.
So how are you going to do this type of definitional work that we talked about, recursive self-improvement? How are you going to do vetting work? It will really depend on who is doing it and how they got there to be doing it.
And I think that if we assume that some bureaucrat who is in some government organization is anointed to do this vetting, and then receives all sorts of side input from people, call them lobbyists, call them hangers-on, whatever, I think it’s going to be a complete failure.
So I think what we are seeing here, based on what you just said up to this point, is that conceptually these ideas make sense.
But who is going to be doing it and how? What kind of detail is going to be provided around these proposals? That’s probably more important than the proposal itself.
Ben Lorica. All right. Next one is wait before using new models for internal R&D.
So the wait could be anywhere from three to six, nine months, right? It would have to be models that have been deployed three, six, nine months out to the world, and then you can use them for your own…
Evangelos Simoudis. That’s crazy. I think that’s crazy.
Ben Lorica. Yeah, yeah. And also, even let’s say for the sake of argument you do that. Just because you deployed it to the public and nothing happened, that doesn’t mean that when you use it for internal research…
Evangelos Simoudis. Exactly. Right.
Ben Lorica. It’s safe, because it’s a different use case.
Evangelos Simoudis. Yeah.
Ben Lorica. I guess this one has already been proposed, which is coordinate slowdowns with competitors, but that’s not really, I don’t think that’s a prisoner’s dilemma scenario, right?
Evangelos Simoudis. It is, and I think that I’ve seen this, or I’ve listened to several debates along these lines, and nobody believes that it can lead to anything successful.
Ben Lorica. All right. So then the next set of proposals is around testing.
I guess give independent evaluators employee-level access. I think that’s already happening. I think Anthropic is already doing this for certain agencies, right?
Evangelos Simoudis. Yeah. But again, it also depends. It’s what I said before: Who is doing it, and how did they get there to be anointed in that position?
So it’s not only the permission aspect, but it’s also the characteristics of that person or persons who are doing the evaluation.
Ben Lorica. Yeah, yeah. I think Anthropic is working with a nonprofit evaluator.
But I think, as you pointed out, government agencies will struggle to hire people because basically the compensation is bad and it’s more bureaucratic. The tools you have to use inside the government are not as good.
So not many people will want to work in the government, right?
One idea that I’ve heard, I’ve been in a meeting where people are proposing that what we need to do is encourage the emergence of private testers equivalent to the credit rating agencies.
But as you know, they can be captured as well, right? In the global financial crisis, the rating agencies were paid by the banks whose securities they rated.
There’s another model, and I’m not sure if you’re familiar. There’s an outfit in Chicago called Underwriters Laboratories.
Evangelos Simoudis. Yes, I do.
Ben Lorica. I think if you buy a fridge or something, there might be a seal, right?
Evangelos Simoudis. That’s right.
Ben Lorica. I don’t know much about them, but they have a good reputation. I don’t know what the relationship is with the companies that they’re rating.
Evangelos Simoudis. Yeah. No, actually, and in Europe, you have corresponding organizations. Actually, they’re better organized on that front than we are.
These organizations have tried to make a leap, in a sense, a jump from what they have been traditionally testing, which is devices that have consumer exposure, to testing other technologies.
They have not been as successful, but I think you’re on the right track there.
Ben Lorica. But the point is that they’re part of this long tradition around test engineering.
Evangelos Simoudis. Yes.
Ben Lorica. That the AI community may find it worth listening to them.
So another proposal is share models with international safety institutes in other countries. That’s not going to happen under the current administration.
Evangelos Simoudis. I don’t think it’s going to happen in general. It may happen partially, but that will make it die quickly. I think that’s really hard to implement.
Ben Lorica. So in financial services, they have this notion of stress testing.
Evangelos Simoudis. Yeah.
Ben Lorica. So that’s another proposal, to come up with an analog for AI models, where there’s some routine, regular stress testing happening.
Evangelos Simoudis. I think that makes sense.
Ben Lorica. All right. So then, very quickly, a bucket around training security.
One is lock down your training environments. I guess this is because of the Hugging Face incident. I don’t know exactly what the specific proposal is there.
I know that OpenAI just announced that they changed the way they train models, which is they make a copy of the entire internet, and then the model training is only happening in this kind of sandbox copy of the internet.
Evangelos Simoudis. It’s a larger sandbox.
Ben Lorica. Yeah, yeah. So the agent is free to do all sorts of internet stuff, but in this copy of the internet.
Evangelos Simoudis. Yeah. Look, I will also say that it would be a good first step if the government requires these companies to fully disclose all of these incidents.
Because I’m very concerned about the fact that we are being drip-fed more breakouts of these agents, and I think that there are a lot more.
I’m not trying to start a conspiracy rumor, but I think there are a lot more problems than we know about. And it would be even more unfortunate if there are problems that the AI providers do not know about.
But I think the new labs know a lot more than they’re letting out, and there are reasons for that as well.
But I think if there is somebody who could require them to make full disclosures, I think that would be a particularly good first step.
Ben Lorica. Yeah, in fact, disclosure is one of the proposals.
And then I’ll rattle off quickly because I know you have to go.
Protect whistleblowers is another one. Reward internal escalation, right? So if someone internally in the lab sees something, they should not be punished. In fact, they should even be rewarded for raising alarms, and so on and so forth.
Evangelos Simoudis. Provided that it’s real, right?
I mean, I think the gentleman from Anthropic who resigned created a lot of hubbub, but there wasn’t a lot of meat behind what he said.
So to me, at this time when we have everybody on edge, both employees, because their jobs may be affected, and consumers who hear rumors about humanity’s demise and all, I think it will be important, particularly for the labs and for the people who take these steps, whether they’re whistleblowers or whatever, to be very serious and very cautious about what they’re causing.
Ben Lorica. And with that, thank you, Evangelos. Again, episode notes on The Data Exchange Media.