Site icon The Data Exchange

Reasoning Doesn’t Start With Language

Ben Lorica

Andrew Dai on Visual Reasoning, CAD Automation, and the Limits of Scaling LLMs.


Subscribe: Apple • Spotify • Overcast • Pocket Casts • YouTube •  AntennaPod • Podcast Addict • Amazon •  RSS.

Ben Lorica speaks with Andrew Dai, co-founder and CEO of Elorian AI, about why today’s frontier models still struggle with complex visual reasoning and why simply scaling language-centric architectures may not solve the problem. Dai explains Elorian’s approach to building specialized multimodal foundation models, spanning data, architectural changes, reinforcement learning, and eventually pre-training, with applications in video, robotics, engineering, and CAD. The conversation also explores visual chain-of-thought, why generation is different from understanding, and why AI-native CAD could become the visual equivalent of the vibe-coding boom.

Subscribe to the Gradient Flow Newsletter

Interview highlights – key sections from the video version:

Jump to transcript

Related content:

Support our work by subscribing to our newsletter📩

Transcript

Below is a polished and edited transcript.

Ben Lorica. All right. So today we have Andrew Dai, co-founder and CEO of Elorian AI, which you can find at Elorian AI. Elorian’s taglines on the website are “Building the foundation of visual thinking” and, in further description, “We’re enabling models to think through spatial relationships, physical constraints, design intent, and abstractions to reach complex, creative solutions that work in the physical world.” And with that, Andrew, welcome to the podcast.

Andrew Dai. Thanks.

Ben Lorica. So I know that you’re—I think I read somewhere that you may not yet be pre-training a frontier model, but you might be post-training existing frontier models. But the bottom line is there’s some notion of a foundation model or frontier model involved, but it’s not for text; it’s for visual reasoning and visual intelligence.

And so, Andrew, just to start off, for our listeners who aren’t steeped in this topic, for visual reasoning and visual intelligence, what are the typical tasks? If you were to go to one of these benchmarks, what are some of the things they do? I’m assuming object detection is one of them, right? Give us an idea of what these tests are.

Andrew Dai. Sure. Yeah, object detection, object recognition, segmentation—these we see as traditional computer vision tasks. And if you think about it, these tasks are actually very natural for animals or for babies or kids, right?

If you are a horse or a rabbit, you need to recognize what object is coming at you. If it’s a lion, then you need to run. If it’s maybe another rabbit, then you don’t need to run. It’s very fundamental to survival.

Same with object segmentation and object detection. All of these traditional computer vision problems can actually be traced back to very basic visual capabilities that you need for survival.

But what sets humans apart from animals, and adult humans from baby humans, is that we can do very complex visual tasks. And by complexity, I mean, when you are buying a coat that, say, has 12 buttons on it, you can count how many buttons there are. Or if there’s a crowded room with a lot of people, you can count how many people there are.

But currently, these models—if you show them, say, 13 people in a room—they’re not going to be able to count that accurately. We test this quite regularly with models like Fable, ChatGPT, and Gemini. So the fact that Fable can’t solve it means that scaling isn’t working for these complex visual tasks.

Other complex tasks include things like understanding a technical drawing or a floor plan: Can a wheelchair move through a floor plan? Or a complex graphical design problem. A lot of these problems require human-level visual reasoning.

Ben Lorica. So just to level-set, in this discussion, are we going to be talking only about static images and not video?

Andrew Dai. We are also working on video.

Ben Lorica. Okay. So video has this additional task where there’s some sort of prediction that you might want to do, right? Predict the next frame, or if I raise my hand, what am I about to do? There’s a certain prediction happening.

Our listeners might be surprised to hear that a natively multimodal model like Gemini can’t even do basic counting. Is that what you’re saying? So if I show Gemini a picture with, I don’t know, 20 Rubik’s Cubes in it, it will fail?

Andrew Dai. Yeah, that’s right. It’s actually very easy to test. You can go to any restaurant with a lot of place settings and a long table and ask it how many people can fit around the table. Or take a picture of an office building and ask it how many windows there are.

Nine times out of 10 that we’ve tried, it doesn’t give you the right answer.

Ben Lorica. And when you say they’re not capable of doing it, these are basically the out-of-the-box Gemini models. But if someone were to post-train this model for a specific task—let’s say I’m actually going to deploy this in a factory where I’m going to count defects, and I’m going to do some post-training—do you think they can do that with one of these frontier models?

Andrew Dai. Well, we are doing post-training, right? So I think it requires a lot of knowledge about how these models work.

We are doing post-training, but we’re also changing the architecture to be more specialized for multimodal. And with a bunch of these changes, we are able to get to state of the art on several of these complex visual reasoning benchmarks.

So it is possible. But to really have a very big impact on these visual capabilities, you do require pre-training. So we expect to be pre-training models next year.

Ben Lorica. So in the text area, for the last few years, fine-tuning—supervised fine-tuning—has largely been solved, right? You can go to a service, upload your sample prompts and desired responses, and come back after lunch with a fine-tuned model.

Are there services like that for visual fine-tuning, supervised visual fine-tuning?

Andrew Dai. Supervised visual fine-tuning? I don’t know. I haven’t looked.

Ben Lorica. But it’s interesting. So you said that even if there is such a service, it’s not enough because you will need to mess around with the architecture.

Andrew Dai. Yeah, that’s right. Basically, the current architectures are very suited for coding and language, and multimodal data is just fundamentally different. You need to treat it differently than just sequences of tokens.

Ben Lorica. So, Andrew, what about the models that were purposely built—not specifically, but somewhat customized—for visual tasks? I’m thinking of the video generation tools like Sora and things like that.

I’m assuming the frontier labs have invested a little bit in visual reasoning, right? For example, if you’re Facebook, you have Instagram. You would imagine they have some sort of frontier model that’s better at visual reasoning than the standard ones, right?

Andrew Dai. Yeah, it is better. But if you look at the products they’re building, they’re very generation-focused. So Google Veo, Genie, Nano Banana—they are all very focused on generation. And the same with models like Seedance and ChatGPT Image. They’re very generation-focused.

But generation and understanding are two different kinds of problems.

We’ve had generation models for a long time, right? We had GANs, if you remember those.

Ben Lorica. Yeah, yeah.

Andrew Dai. And the same for the text world. We had GPT coding models four or five years ago. If you asked any random person, “Can these models code?” they would probably say yes. But if you asked a software engineer, “Can these models code?” they definitely would not have said yes four years ago.

That is the difference between generation and understanding.

With generation, it’s very easy to generate something that looks real. But understanding is completely different. You have to really understand the details of something you generated to fool an expert, like a film director or an artist.

If you show any video generated by a model today to a film director, they would still say, “Oh, there are so many things that are wrong.” And that’s because these models don’t understand what they’re generating, just like four years ago the coding models had no idea what they were writing in code.

Ben Lorica. Now, there are a class of startups that have been around that focus on video—Twelve Labs and startups of that ilk—where the focus is on some sort of video understanding.

So I have 5,000 hours of surveillance-camera video, and I want to be able to type a prompt and find something. I think I actually talked to someone at Twelve Labs. They built embedding models and things like this.

So I’m assuming that, for video at least, there are some—I’m not sure. I don’t know that they actually have a frontier model. I know they have embeddings. But there’s something going on with video as far as video understanding, right? There are solutions out there.

I don’t know how good they are, honestly, because I haven’t tried them. But I’m assuming they’re better than video search from Google, right?

Andrew Dai. I haven’t heard that these alternatives have been better than Gemini for video understanding. I expect they’re probably not using frontier models. They’re probably not building the same kind of vision-language models or large-language-model-type designs, which have lots of parameters. They take a lot of compute to train and serve.

Ben Lorica. They have a way of embedding, so they have more information about what’s in the video. My understanding is, let’s say you have a video of you, Andrew, giving a talk—they’ll even understand what’s in your slides, right?

Andrew Dai. Yeah. Embeddings, I think, are one kind of approach and definitely a very cost-effective and cheap approach.

But we’re focusing more on complex video reasoning, like asking complex questions rather than just generating a simple embedding.

Ben Lorica. No, no. I mean, this is the whole point of these startups. I want to be able to look through thousands of hours of video to find something, right? Whatever the application is—security, national intelligence, that kind of thing.

The question is, I actually don’t know how good they are. Obviously, if the video has a lot of audio, then you can transcribe it, and maybe you get something through the transcript. But my understanding is they go beyond that, in the sense that the video could have no sound and they’ll still be able to find the time the person opened the door, something like that.

But I’m not sure how much of that is in the public domain and whether there are competitive benchmarks for video understanding. I don’t know what the state of the art is. Do you?

Andrew Dai. Yeah, there are some video benchmarks out there. Those are some of the ones we’ve evaluated and compared to Gemini, and we’re leading on those.

From what we can tell, we’re not aware of what Twelve Labs’ performance on those benchmarks is, but we plan to release our performance later this year.

Ben Lorica. So your approach right now is that you’re still sort of in the learning stage, in the sense that you’re not pre-training. You’re post-training one of the open-weights models, I assume?

Andrew Dai. That’s right.

Ben Lorica. But you’re post-training it with a purpose, right? Because my understanding is you don’t necessarily want to be a research lab that doesn’t ship anything useful. You want to ship something that’s useful right away.

So what are the limitations? What are the things that post-training has shown it’s responsible for, and where do you think it hits a wall?

Andrew Dai. Post-training does let you change the data distribution quite a bit and make some tweaks to the architecture, but it doesn’t let you make larger changes.

Ben Lorica. Would you be comfortable sharing some of the specific tasks that you’ve targeted for post-training?

Andrew Dai. Some of the tasks—I would say video understanding is one thing we’re looking at. Some of the counting problems I mentioned are also quite interesting because that’s where these models fail.

Ben Lorica. And what’s the suite of techniques for post-training that you use?

Andrew Dai. Data is a big one, so being very careful about data. I was a data area lead for Gemini, and so we are essentially a data-first company.

We’re very careful about getting new data, cleaning up data, getting the right data vendors to give us visual reasoning data, and generating our own data. So that’s really critical.

And then we’re making changes, like I said, to the architecture to specialize it for multimodal understanding. And we’re also using RL as well. So, the full suite.

Ben Lorica. But then, the types of things you’re attacking—are they real-world problems, or are they aimed at specific public benchmarks?

Andrew Dai. It’s a bit of both. There are a lot of real-world use cases that aren’t well matched by a benchmark. But we’re also working on improving the model across certain benchmarks as well, so that it has a broad range of capabilities rather than just being used for one application.

Ben Lorica. The reason I ask is, it’s one thing for you folks to continue post-training and post-training, but it’s just you. It’s not hitting an end user or someone who will actually depend on the results.

Do you have early users who are actually giving you feedback that, “Hey, this is great, but it’s actually not enough for us”?

Andrew Dai. Yeah. To build a product, it’s really important to talk to potential customers early to figure out what their use cases are and what they need from a model or from the endpoint. So we are talking to some companies that we can potentially be working with.

Ben Lorica. So the type of post-training you’re doing is not for made-up problems, I guess is my question. It’s for problems that real companies have to tackle.

Andrew Dai. Yeah. We need to get this model in the hands of real businesses.

Ben Lorica. So how much data are we talking about in terms of post-training?

Andrew Dai. Post-training, you don’t need a ton of data. I can’t say the exact numbers, but we are getting significant amounts of data from a bunch of data vendors that I know from Gemini. And yeah, there are a lot of data vendors out there right now.

Ben Lorica. And then, as you hinted very early, data is not enough. You have to fiddle with the architecture, right?

You take the open-weights models. Are you comfortable telling us which ones—what the base models are that you’re using?

Andrew Dai. I don’t think we want to disclose that just yet.

Ben Lorica. But basically, the base models you’re using—are they text models, or are they models that already have somewhat of a visual reasoning capability?

Andrew Dai. Yeah, they already have visual understanding capabilities, and we are essentially making those better.

Ben Lorica. And so when you get to the point where you have to tweak the architecture, this is not something a regular enterprise will be able to do, obviously, because they don’t have the personnel.

So what would the offering be at some point? You work with a company, there will be a data component—maybe they provide you some data, maybe you acquire data for their specific task—but then you’ll need to send the equivalent of forward-deployed engineers to tweak the architecture?

Andrew Dai. We don’t want to customize a model for each company. We’re still here to build frontier models—very general models that are very widely capable.

And to do that, the models have to be quite large, and so it doesn’t quite make sense to customize this large model for each business.

Ben Lorica. So the goal is, through tweaking the architecture, you have enough learning so that this tweaked model is enough to serve as the foundation model for a lot of downstream applications.

Andrew Dai. Yeah, for a lot of downstream multimodal applications, essentially. Kind of like how coding models are used for a whole range of coding tasks, from SVGs through to building websites and all that kind of thing.

We’re here to do the same for multimodal: building a multimodal foundation model that can be used for all kinds of multimodal use cases, probably a lot that we’ve never even thought of.

Ben Lorica. So I’ve read that the types of use cases you’re looking at are real use cases, like engineering diagrams. What about use cases in robotics? Because that requires visual intelligence as well, right?

Andrew Dai. Yeah. Robotics requires visual intelligence to work in the real world. Basically, whenever you work in the real world, you need vision. You can’t just do it with different kinds of code.

So yeah, that’s a compelling use case. And we’re talking with some robotics companies as well about how we can collaborate, because what they’re seeing is that vision is a really big bottleneck for robotics AI and the robot brain.

Ben Lorica. So you alluded to earlier that at some point you anticipate that what you’re doing—post-training and tweaking the architecture—may not be enough, and then you might have to pre-train a foundation model from scratch.

Man, that’s going to be expensive.

Andrew Dai. Yeah, it’s expensive.

Ben Lorica. You’ll need to raise another round of funding to do that, right?

But I guess what I like about your approach is that you’re not doing it from the get-go. You want to understand first exactly what you need to do when you get to that point, right?

Andrew Dai. Yeah. We want to understand, from the point of view of the technology and the data, what’s a good setup.

But not just that. We want to understand what the potential market is. Who would be using this model? What kind of capabilities do they want out of it?

So that can inform how we pre-train it, how we set everything up, the data and everything.

Ben Lorica. By the way, for our listeners and viewers, Andrew was one of the pioneers of, I guess, semi-supervised or unsupervised learning, in the sense of taking unlabeled text and using that as a flywheel for language somehow.

So is there an analog for visual reasoning? Can’t I just go to YouTube and TikTok and use all of that unlabeled video, and then that’s enough?

Andrew Dai. Yeah. That’s kind of the ideal objective that would be perfect for vision, but the objectives people have tried so far haven’t really scaled as well as they have for language.

For language, if you throw more compute at it, you get a significantly better language model—better at coding, math, etc. But that hasn’t been true for vision so far.

That’s why you see Fable. Even though Fable is a very big model trained on a ton of data, it still fails at some visual tasks compared to Gemini.

So we haven’t found that ideal objective for vision yet.

Ben Lorica. So simple scaling isn’t working as well.

In the robotics world, data is also an important thing when they’re building these frontier models. And there you have the following sources, right? You can go on YouTube and watch video, and maybe there are movements there that you can glean. You can pay someone to do a movement that the robot can mimic. You might be able to use synthetic data. You might be able to generate synthetic data. You might be able to build digital twins or simulators to create this data.

So are they roughly the same buckets for you?

Andrew Dai. For us, it’s a bit more varied than just that.

With robotics, there’s a lot of emphasis on egocentric data, data with manipulation, where you’re using your actions to affect the world.

But we want to build something broader than that, where you can use it to edit a design or make your drawing better automatically.

There are a lot of engineering applications in CAD where you can use this to build a faster, lighter car or spaceship that needs less fuel because you’ve optimized the design of the wing.

For that, your actions won’t be in the physical-world space. They might still be in the virtual space, but your understanding still needs to be very good in terms of understanding the physical world and physical AI.

So we see the application space as being broader than just robotics.

Ben Lorica. So let’s say you succeed. In the course of post-training, you not only start getting actual users who might be paying you, but you get enough learning that you decide, “Okay, now I’m ready. I’m going to pre-train a foundation model from scratch. I’m going to raise a round of money.”

I’m assuming that during this journey, you’re doing something somewhat in public, in that you’re publishing some results and blog posts, technical articles, maybe even helping establish benchmarks.

Then the frontier labs notice what you’re doing, and they decide, “Hey, we’re going to train our own foundation model because these guys have shown that there’s demand, and it’s viable because people have taken what they’ve done with post-training and actually built and deployed useful things with it.”

So am I wrong for being paranoid?

Andrew Dai. Yeah, that’s a great question. But I’d say that’s a question for a lot of startups.

Ben Lorica. Yeah.

Andrew Dai. The same thing: if a startup catches on, why doesn’t Anthropic just swipe-code a competitor?

Ben Lorica. No, no. In this case, you actually have to train a frontier model. Once Elorian has shown that, hey, enterprises will actually use this thing if it’s performant, right?

Andrew Dai. Yeah. The thing is that we are building a specialized model, so we’re taking a different approach from all the other models now, most of which are very heavily focused on coding.

The data, the architecture, the RL—it’s all focused on coding.

Because we’re taking this different approach, that means a competitor would also have to build a new specialized model. And that’s very resource-intensive in terms of chips, researchers, everything. It also takes time.

As a startup, the advantage we have is to move fast. That’s also why we want to get a model out there very quickly and raise the next round quickly, too, so we can move fast and stay ahead of the wave that’s going to be built up around these new multimodal use cases.

Ben Lorica. So, Andrew, is there a way for you to explain to a general audience why the architecture for visual reasoning is so different from coding or text?

Andrew Dai. It’s so different—you can see it even among people, right?

In white-collar work, the people who are doing math and coding are not the same people who are doing complex engineering designs, designing the next Starship, designing the next EV battery, or designing an oil refinery.

Those are essentially completely different groups of people. There is some overlap. I guess some people move between the industries. But you’re using very different skills and capabilities when you’re doing things like this.

And that’s kind of reflected in the model as well. Being better at coding doesn’t really improve your raw multimodal capabilities.

You also see that with benchmarks like ARC-AGI. ARC-AGI is on a 64-by-64 pixel grid. Coding is very well suited to 64-by-64 pixel problems, but I can’t think of any really interesting real-world problem that you can squeeze into 64 by 64 pixels.

So coding alone isn’t going to be able to solve these problems. And you can see it from the divide among humans and in white-collar work, too.

Ben Lorica. I guess my question is: Is there some secret architecture that will unlock visual reasoning that might actually depart from the architectures they use for text and coding, which seem like they’re still scaling?

Everything is about scaling. And I guess in the visual reasoning space, you can imagine scaling is also going to be a main ingredient—the more data, the more compute, the better the model.

But as you point out, humans don’t need a lot of examples, right? One or two examples, boom, we’re done. We can visually reason. Even a six-year-old who can’t yet articulate or speak complex words can already count the number of things in a picture.

So why do I need scale? Maybe there’s a different architecture altogether, no?

Andrew Dai. Yeah. That’s one of the beliefs we have. One of our theses for the company is that reasoning isn’t built on language.

Currently, reasoning is just learning language because it’s easy and we have a lot of data there. But we believe reasoning originates from the visual world, from visual inputs, from animals trying to survive in the wild.

You still need reasoning, but animals are probably not reasoning with words or tokens, right? They’re likely reasoning in the visual space.

It’s similar for human babies. Little babies or preschoolers can already do some visual tasks even though—

Ben Lorica. One-shot learning.

Andrew Dai. Yeah.

Ben Lorica. Even for a three-year-old, right?

Andrew Dai. Yeah, it’s very few-shot. They’re very data-efficient.

And so we believe that you need this kind of visual thinking, this kind of visual chain of thought, to really reason like a human reasons and to have that efficiency. And so that’s what we are building into our generation of models.

Ben Lorica. So do you have to teach the model basic physics? You know, like if you throw an apple up, it has to fall down.

Andrew Dai. A lot of these things the current frontier models already know. And because we’re starting with a frontier model, we don’t need to add more of that.

Ben Lorica. You mean visually, they already know this?

Andrew Dai. Yeah. They already know the basic physics visually.

It’s the more complex problems. Say you’re throwing a marble at a bunch of dominoes. Which dominoes are going to fall? Or how many dominoes will fall? Or, if you’re throwing a bowling ball, how many bowling pins are going to fall?

That kind of capability we need to train into the model because that’s significantly more advanced.

Ben Lorica. So how important is synthetic data?

Andrew Dai. Synthetic data is very important. It’s a way to scale datasets if you do it carefully. But if you don’t do it carefully, you can end up with things like mode collapse and a degenerate model.

Ben Lorica. So let’s say I’m in an enterprise, and a startup or a vendor like you comes to me and says, “Hey, Ben, our model beats Gemini at visual reasoning.”

What’s my next question? What are the questions I should be asking?

Andrew Dai. The next question should be, “When can we test it?” I’d say.

Ben Lorica. But are there even more specific tells or benchmarks that would allow you to immediately detect whether or not this is worth pursuing?

Andrew Dai. Yeah. I think finding the visual benchmarks that are closest to your use case can give you some signal as to whether the model will be good for your use case.

But we’re also planning to release an early-access API. So I think the best way is always to test the model directly on your own data, on your own use cases, and see.

You should never trust just the benchmark numbers, because there’s always going to be some distance between that and your use case.

Ben Lorica. So in LLMs, the reasoning-enhanced LLMs, you have the reasoning traces, right?

What’s the equivalent of reasoning traces for visual reasoning? And maybe it’s hard to interpret for a human, right?

Andrew Dai. Yeah, that would be visual reasoning traces. In our case, it would be things like generations of images or edits to images.

Ben Lorica. So it’s somewhat of a UX problem. How do I—you’d have to expose to me how it is arriving at counting 13 apples in this picture?

Andrew Dai. Yeah, exactly.

Ben Lorica. So does that exist?

Andrew Dai. No. We are building that. It doesn’t exist for the current frontier models, but we’re building it into our models.

Ben Lorica. So, Andrew, there’s another category which arguably you’re part of, which is world models.

And by the way, “world models” is kind of a marketing term because different people have different definitions of what a world model is. But you kind of fit into world models, no?

Andrew Dai. Yeah, it depends on how you define it.

I would say even LLMs—frontier models like ChatGPT and Gemini—fit into the standard definition of world models, because you can give them any picture or video about the world, and they will give you a reasonable answer.

But when I think of world models, I think of them as being action-oriented. You can make some action on the world, and usually they’re very centered around robotics or gaming.

So I usually say we are “world model adjacent,” because our models could be used for actions, but they could also be used in a lot of applications where you’re not doing an action at the end. You’re asking the model a question, or you’re asking it to edit some design—

Ben Lorica. Or analytics, right?

Andrew Dai. Yeah, or analytics rather than an action on the physical world.

Ben Lorica. Right.

So do these models consume more tokens than the reasoning models? Some of the coding models are just churning through tons and tons of tokens, right?

Andrew Dai. Yeah. Coding models and math models consume a ton of tokens. We see that because we do vibe coding ourselves.

Ben Lorica. Yeah.

Andrew Dai. Multimodal data definitely consumes more tokens by its nature. As the saying goes, a picture is worth a thousand words—or a thousand tokens.

So there are more tokens that are needed, but it’s not an order of magnitude more tokens. It’s not like the difference between Fable and Sonnet kinds of costs. It’s an incremental cost on top.

Ben Lorica. So for the frontier labs, how would you rank their offerings as far as visual reasoning? Who’s the best? Rank them in rank order.

Andrew Dai. I would say Gemini is the best. They train on a lot of multimodal data, and the multimodal capabilities, from our internal testing as well, are right up there.

I think close behind is ChatGPT with GPT-4o and now with GPT-5. They are also quite strong in multimodal understanding capabilities. Not quite as strong as Gemini, from what we’ve seen, but close.

And then further behind, I think we see models like Muse, Grok, and Claude as well.

Ben Lorica. What about Qwen and Kimi?

Andrew Dai. Yeah, they’re very good as well.

Ben Lorica. Claude doesn’t seem to be multimodal at all, right?

Andrew Dai. Yeah. We’ve seen that Claude is relatively weaker.

One thing we did was take a picture of the Claude web page, which has a hand-drawn hand on there, and ask it how many fingers the hand has. Claude always gives the wrong answer. So I think that’s pretty funny.

Ben Lorica. Yeah. It seems like they’ve been completely focused on text, right?

Even in the text world, people speculate that these models are really mixtures of experts, right? There’s a router involved.

Is that something that will happen in visual reasoning as well? You have a frontier model, but you might have a router in front of it, and then there might be smaller specialized models.

Andrew Dai. I’m not sure whether that level of routing is needed.

We are still using an MoE model, so we’re training mixture-of-experts large models. But the models are capable of language as well as vision.

That’s because we believe that human reasoning is not just visually based. Language is a very important part of human reasoning. So we’re building models that can understand both.

But we can expect that some agents might use our model if it’s a multimodal problem, and then, say, Claude or ChatGPT if there’s a coding problem. So we imagine that can just arise naturally.

Ben Lorica. So if you were to look 12 months from now—and this might be hard to answer right now—what would be the equivalent of coding that basically makes this visual reasoning space blow up? What’s the use case?

Andrew Dai. Yeah, I think that would be CAD.

The equivalent of coding in the real physical world, I think you could say, is CAD and design, because there you are building things that will be manufactured, that will exist in the real physical world.

So you can’t have any mistakes, because fixing them is going to be very expensive and very slow. And the physical world is built up of these components.

Ben Lorica. And I have the term for it: vibe cadding.

Andrew Dai. Vibe cadding. That’s a great term.

Ben Lorica. So obviously, with code, there’s vibe coding, which means basically describing the type of program you want to write.

What would be the equivalent of the interface between the visual reasoning model and CAD?

Andrew Dai. The current capabilities of models can get you from zero to one. You can give it some text, and it’ll give you a basic CAD part, like a very simple joint or a very simple component.

But it doesn’t really do what most mechanical engineers do. My father is a professor of mechanical engineering, so I’ve seen him do this kind of work.

Most of his time is spent working with hundreds of different parts in this very big assembly. Editing it takes a lot of time. It takes a painstaking amount of effort to make changes—hundreds of hours or thousands of hours of labor.

So having an AI able to do edits natively in CAD would save him a ton of time.

Ben Lorica. And so, for listeners who aren’t familiar with this, underneath the CAD design software, is there a programming language?

Andrew Dai. Yeah, kind of. The actions that you do in the software get translated into a kind of programming language.

Maybe one analogy is: say you have a Word document—or—

Ben Lorica. Or an Excel file.

Andrew Dai. Yeah, or an Excel file. And you draw a diagram or flowchart on that, and then you want to print it on your laser printer or inkjet printer.

What actually happens is that your diagram gets translated into a set of machine code that tells the laser printer or inkjet printer how to move its arms to print out your diagram in great quality.

That kind of thing is basically what’s happening in CAD as well. So you design it in the software, that gets translated into code, and that code then goes to manufacture the physical object.

Ben Lorica. But don’t they have some programming language, or at least a scripting language?

Let’s say I’m working in an architectural firm and we have 500 diagrams, and I want to make subtle tweaks to all 500. Isn’t there a script that I can write?

Andrew Dai. No. I think that would be a dream for a lot of architects, but I don’t think that exists.

Ben Lorica. The reason I’m going down this road is because if there were, then there would be the equivalent of vibe coding that script to do all of that work.

But since there isn’t, this is why you need this visual frontier model, right?

Andrew Dai. Yeah, exactly. You need to understand that drawing and the 500 different variations of that drawing to really know how to apply the change.

The file format does have code in it, but just having the code isn’t enough to know which part of the code you should change or how you should change it.

Ben Lorica. But the way users interact with this is through a piece of CAD software, like AutoCAD or Autodesk, and it’s very specific. It’s UX-specific, right?

So it’s like if I’m using Photoshop and I move to GIMP—that’s a different UX.

So is the idea here that you’re going to build a visual frontier model that can help me regardless of what CAD software I’m using?

Andrew Dai. Yeah, we want to be able to integrate with all kinds of CAD software.

Even if you switch between Photoshop and GIMP, I think you’ll recognize that a lot of the tools are the same, like the pencil tool, the eraser tool, the flood-fill tool.

It’s similar for CAD as well. In CAD, you have B-splines, you have extrusion tools, you have tools that can add a hole. So a lot of those tools are also similar between CAD formats.

Ben Lorica. So roughly, what’s the addressable market? What’s the addressable market for CAD software?

Andrew Dai. Well, the existing CAD software market is in the billions of dollars. I can’t remember offhand exactly what it is.

But I think a better way to look at it is how much we can accelerate design by adding our AI. And that potential can actually be much larger.

Ben Lorica. It will open up the space to more people, right? Because I imagine right now the people who are using CAD software are considered specialists.

Andrew Dai. Yeah, they’re specialists. You need to go through years of training to get it working.

Ben Lorica. So what you’re going to try to do is basically make it so that, using a text prompt, I’ll be able to use the CAD software.

Andrew Dai. Yeah. And you will want to use it because it makes your life a lot easier.

Ben Lorica. Yeah. Wow. That’s a great place to end.

And then, is there the equivalent of recursive self-improvement for visual intelligence?

Andrew Dai. Yeah, there is. And we are developing approaches for that internally.

Ben Lorica. And you should know. You probably worked with Quoc Le on the early ideas around recursive self-improvement, right?

Andrew Dai. Yeah, exactly.

Ben Lorica. Yeah. And with that, thank you, Andrew.

Exit mobile version