Let Domain Experts Build the Agents

Ben Lorica

Akshay Sharma and Nick Ung on Self-Serve Agents, Prompt Linters, and Trajectory Evals.

Subscribe: Apple • Spotify • Overcast • Pocket Casts • YouTube •  AntennaPod • Podcast Addict • Amazon •  RSS.

Akshay Sharma, ML Engineer at Lyft, and Nick Ung, Staff Applied Scientist at Lyft, explain how their team is building a platform that lets domain experts create customer-facing AI agents while engineers provide the underlying tools, guardrails, and evaluation infrastructure. We discuss why production traces are becoming critical training data, how Lyft evaluates entire agent trajectories rather than just model outputs, and what teams should know about prompts, multi-agent systems, memory, safety, and model flexibility.

Subscribe to the Gradient Flow Newsletter

Interview highlights – key sections from the video version:

Jump to transcript

Related content:

Support our work by subscribing to our newsletter📩

Transcript

Below is a polished and edited transcript.

Ben Lorica. All right. So today we have Akshay Sharma, machine learning engineer, and Nick Ung, staff applied scientist at Lyft. Guys, welcome to the podcast.

Akshay Sharma. Thank you, Ben. Thank you for having us.

Ben Lorica. Yeah. So the reason I have you here is that you have this new agent platform. As I understand it, in the past, when you had to build agents, it was quite an involved process involving a bunch of technical teams. The goal of this platform was to make that easier, streamline it, and, more importantly, allow some, if not most, of the agents to be built by domain experts who are not technical. Is that correct? Am I correct in my high-level description?

Akshay Sharma. Yes, definitely.

Ben Lorica. All right. So then, I guess, let’s start there. Building a platform for non-technical people—and we’ll get to the technical stuff later in the conversation—has both positives and negatives, right? The positive thing is they have the domain expertise. The negative thing is they may not understand data or data science. So I guess let’s start there. What are some of the wins as far as this platform is concerned for the personas that you targeted?

Akshay Sharma. I think one of the biggest wins is reducing the development time for our agents. That’s very visible and much more prominent than the other wins. Initially, as you said, we used to take a lot of time to build an agent around a product use case.

Our team especially focuses on solving customer problems, and customers for Lyft are drivers, riders, and people who are applying to become drivers. We had this vision in mind that we wanted to solve their problems, but we wanted to solve them through AI. And because customer support is one of the most prominent use cases for agentic stuff, we started building agents to solve these problems a year ago.

Then we realized that we did have a lot of issues to cover. We did have a lot of problems to solve, but the time we took building the agentic orchestration around every sub-agent or every specialized agent was a lot. Initially, it was a lot more because we were setting up some stuff. We were building the orchestration platform. We were doing the evals. We were just learning how to build these things and how to go about it.

So initially, it was a lot more. Then it gradually started to come down. But even then, there was a lot of expertise involved from ML engineers and software engineers who were bridging the gap with the domain experts. The domain experts decided how the agent should behave because they had that understanding from dealing with human agents and solving customer problems before we even started building AI agents.

We were trying to translate human-agent behavior, policies, response guides, and knowledge bases to the AI agents. Then there was a lot of tooling around it. There was this orchestration part, there was this tooling, and there was this prompt that instructed the AI agent on how to behave. There was a lot involved in tying all these components together, and then there was the question of how to make sure everything was working correctly.

So even if we shipped one agent to solve one or two specific issues, it involved a fairly elaborate process, orchestration, and development lifecycle.

Ben Lorica. So Nick, since you’re a data scientist, now you’re having people who may not know that much about data or stats or ML building agents. What are some of the pitfalls and some of the things you had to make sure were available in this platform to make it easy for them? Because these are people who may not know evals or things like that.

Nick Ung. I think you’re right. A lot of the time—and this is true for product managers as well—people who are not from the data domain have a certain mental model when they go into building an agent. For a specific customer issue, they know, “These are the five golden paths,” and then there may be one or two edge cases they want to cover.

As part of our own internal testing framework, we have a simulated app that allows users to test these flows. So their approach to eval is, “Hey, I know these are the golden paths. I’m going to write a prompt, and I’m just going to test it in the app to see if it works.”

Ben Lorica. So the goal is that this is really some sort of self-serve platform, in the sense that they’re doing that themselves.

Nick Ung. They are doing that, but we’ve put a lot of guardrails in place so we don’t just let them do that without any approval. We’re going over all the prompts before we let those prompts hit staging or even production.

There’s usually an engineer-in-the-loop process. We tell them, “Hey, these are all the MCP tools that this agent has access to.”

Ben Lorica. Do they know? Do they even know what MCP is?

Nick Ung. Obviously, we don’t tell them what MCP is. We just tell them, “Hey, the agent can look this up.” If the agent needs to understand whether this ride was canceled, we just tell them, “The agent has access to this. You can reference that tool name in your prompt, and then the agent will look it up.”

We obviously don’t explain to them what MCP is. There’s no point explaining what that is. But we’re building this shared language of, “These are the list of tools, these are the actions the agent can take or things it can look up, and here’s how you plug that into the prompt.”

And in a specific user journey, if we hit a point where the AI agent is not supposed to handle that specific issue—maybe it was safety-related, maybe it was fraud-related, or a driver deactivation issue—the AI agent is not supposed to handle that. So how do you instruct the agent to escalate to a human agent?

We’ve built these small building blocks in a way that the domain expert can understand, so they can build the initial version of the prompt. Then engineers review it and make sure they haven’t made any mistakes.

One of the quality projects we’re working on right now is a prompt linter. We have deterministic rules to check things like, “Did I make a mistake referencing the wrong tool name? Did they make any other kind of quality mistake in the prompt?” There are some things we can try to catch with the prompt linter, but we still have an engineer-in-the-loop process to double-check that the prompt is actually written in the way we intend.

Ben Lorica. So, a couple of questions. The first is: Give us a sense of the level of complexity of the agents that domain experts are building. I’m assuming they’re pretty simple agents whose workflow is something they can understand, right? It’s not some long-horizon agent with complicated branching and things like that. Is there an ideal sort of agent for these people? And are these mostly internal or customer-facing agents?

Nick Ung. These are actually customer-facing agents. Thinking about the types of customer support issues that Lyft handles, there’s generally a range from low complexity to high complexity.

We’re pretty opinionated that, for more long-horizon tasks—I mean, none of this is going to be as long-horizon as coding agents, to be fair. An agent is not going to go to work for two hours and then come back to you with the results.

Ben Lorica. Yeah.

Nick Ung. But there are more complex customer issues. For example, if a driver submitted a damage claim, they’re saying that the rider damaged their car, and they need to submit photos, and the AI agent needs to analyze the photos.

Ben Lorica. So the domain experts, who could be product people, are building customer-facing agents. They’re not necessarily building internal agents. Because, let’s face it, even product people might have some complicated tasks internally. So they’re not building agents to automate their internal tasks. They’re building agents that interact with customers.

Nick Ung. Yeah, they’re building agents to interact with customers. So far, we’ve asked them to start with lower-complexity issues.

Ben Lorica. And sorry to interrupt you. So the logic here is that you have them build it because they understand it better than the engineers, right?

Nick Ung. Yeah, exactly. These are people who either handle customer issues themselves or are literally looking at our tickets every day. They’re manually reviewing the tickets. They know the customer issues very well. They know how customers want their issues to be resolved.

And this is where we take ourselves out as the bottleneck. We, as ML engineers, don’t want to be looking at tickets all day.

Ben Lorica. But one question is: How do these people know, Nick and Akshay, when the agent they’ve developed using this platform is ready to deploy to production? How do they know if it’s good enough and that they’ve done enough evals? I’m assuming you guys are able to test for edge cases and things like that. How do these non-experts know that?

Akshay Sharma. I think, as Nick said, autonomy comes with some guardrails. We’re giving them freedom to write prompts and also define the behavior the AI agent should follow, but we do have guardrails and guidelines both for prompts and for the agent itself.

When we’re writing prompts, as Nick said, we have something called a prompt linter, and it has both components. It has deterministic checks to validate the syntax, the structure, the tool references, and things like that. Then we also have an LLM checking the prompt for inconsistent behavior in different phases. Every phase should have an exit condition and things like that.

So all the things that are non-deterministic and behavioral, LLMs are going to check. But once the prompt is ready, an ML engineer, data scientist, or software engineer is part of the review process. We have that review process in place.

Then let’s say the prompt is ready to be deployed, and all the tools and orchestration are already in place to make it function in production. We have evals running. We have offline evals and online evals. Offline evals are something we do before launching an agent into production.

Ben Lorica. Actually, I guess my question is: You have the evals, but the non-experts are not looking at the results of the evals. It’s the team that supports them, the data scientists and engineers. Is that right?

Akshay Sharma. Yeah. We do have data scientists involved in publishing or deploying an agent to production. Some of the domain experts can look at the metrics and understand things that are tied to the business, things tied to escalations, and so on.

Ben Lorica. I guess the reason I ask is, obviously, let’s say I’m a product person, but earlier in my career I was a data scientist. It’s not like I’m unaware of some of the basics. Do you make a distinction between people who have a little more background than others as far as their ability to go faster?

Nick Ung. Yeah, I can take this question. Ben, I think there are a few different ways of doing evals here. There’s definitely the data-driven approach. You get a sample dataset, maybe the initial verbatim, and then you run it through. You have deterministic checkers. You have an LLM judge.

Ben Lorica. Historical traces, right?

Nick Ung. Historical traces, right. Then you rescore them and understand whether a change is going to impact certain metrics that we care about.

But typically, for domain experts—and what we strictly refer to as domain experts here—they’re non-technical people. They don’t have a data science background. They might not even have a technical background.

Usually, what we ask them to do is deploy the prompt to staging. We ask them to use our app to test a couple of flows, the golden paths, and check a couple of edge cases. Then we have data science in the loop to provide a second check, which is the data-driven eval approach.

So I would say we have multiple gates that an agent deployment needs to go through before it can actually reach production. Different personas do different things well. Domain experts do these manual checks and manual testing really well, and analysts and data scientists come in and do more data-driven evals.

We run our offline evals to check if everything looks good. If everything looks promising, then we can try to deploy to production. Depending on the size of the change, we might run an online A/B test. We might not. It’s definitely a little bit more on a case-by-case basis.

Ben Lorica. So the types of agents we’re talking about here interact with customers, but they’re basically replacing or automating certain existing customer-facing offerings, right? In other words, it’s something where, as Nick describes, you have a golden path. It’s something the non-technical people have seen through a lot of previous customer interactions, so then they can go ahead and do that. But as far as inventing something new, customer-facing, that’s not happening yet.

Akshay Sharma. I think it’s both. The first part is moving interactions from human agents or human escalations toward AI agents to reduce the cost, so that we can automate things and respond and resolve things more quickly, with much less cost and much less resolution time.

But there are also some things, Ben, that are very difficult for human agents to do. Let’s say we have the damage example Nick was talking about. Drivers are submitting photos as proof that someone damaged their car.

For human agents to go into their history and check how many past claims they’ve had, what their driving history is, whether the uploaded images are fraudulent, whether the metadata gives us good enough proof that they’re legitimate, when the photo was uploaded, what the driving history was, whether the photo was uploaded even before the ride happened, and things like that—it becomes very complicated when we ask human agents to do that.

Before this, it used to take a lot of time, and there were a lot of gaps that we couldn’t fill. Now with AI, I think those things become simpler. So the innovation is handling issues much better and bringing these actions and components into the picture, which can reduce fraud and make a lot of things quicker that we weren’t even able to do with human agents.

Ben Lorica. So it’s interesting. The non-technical people who are authoring these agents mainly represent the agent through the prompt. So the main thing they have to come up with is the prompt. Is that correct?

Akshay Sharma. That’s the main thing, yes.

Ben Lorica. Interesting. So there’s no visual tool where you sketch out a workflow. It’s all just text, right? You’re describing something, and in the text you’re describing in as much detail as possible what the agent is going to do and the sequence of things that might happen.

What about—and I don’t know if you have to worry about this—things like LTL, linear temporal logic? By that I mean things like, always do X before Y, or never do Z once A has happened.

I guess someone can write a prompt and you can automatically discern the temporal logic that needs to be enforced, right? Based on the prompt and the tasks of the agent, you extract the linear temporal logic that must be enforced in order for this to be correct.

Akshay Sharma. Yes. I’ll also let Nick speak, but I think this is a good issue. We do have checks where we can deterministically check if, let’s say, a tool is a prerequisite for another tool call.

Let’s say we want to have a ride ID and ride information. We need the ride ID to get the ride information, and things like a driver ID to get driver information. These are some things we resolve deterministically.

The temporal logic you’re talking about can be enforced by deterministic checks.

Ben Lorica. Like the example you said, right? Maybe I have to look up a certain database first, get that information, and then proceed.

Akshay Sharma. Right.

Ben Lorica. Yeah.

Akshay Sharma. In the prompts we can write, “You have to get the ride ID first, and then look up ride information,” and then—

Ben Lorica. Now, the non-technical person won’t be able to do that, right?

Akshay Sharma. Right. So what happens in situations like this is that we have tools. Let’s say we need to get ride information, and that tool itself has a docstring that says the ride ID is a prerequisite for this tool, as an example.

Let’s say there’s an action that shows the user a ride selector. The user selects the ride, and then we can get the ride ID from the agentic state. That tool will again call the LLM node and instruct it, “We don’t have a ride ID for this. Can you please ask the user for the ride ID?”

So it’ll go and show the ride selector to the user, collect the ride ID, and then call the tool again.

Ben Lorica. But the point is, this is all automatically discerned from the prompt.

Akshay Sharma. Yes, sometimes. When things are very dynamic and there are things that aren’t captured by the prompt behavior, we have to do these kinds of things. But mostly, for responses or workflows that are very straightforward, I think we can put most of the things in the prompt. Then for hallucinations, we can have guardrails and deterministic checks.

Ben Lorica. So, one of the fashionable things right now is formal verification. Is there a role for formal verification in some of these agents?

Nick Ung. Can you elaborate a little bit more? I’m not too familiar with that body of work.

Ben Lorica. It’s basically, if you can break down what an agent is doing in a way that’s verifiable, you can formally verify it. Some people use Lean, for example, a theorem prover. Maybe that’s too advanced. I’ll skip it.

All right. One of the things that you folks do, of course, is you have a router. A router means, obviously, you’re using multiple models. I don’t know how comfortable you folks are or whether you’re allowed to talk about the suite of models you use. I’m assuming there are commercial proprietary models, but do you guys use open-weight models?

Akshay Sharma. We’re in the process of exploring that. Not yet. This is happening.

I think probably all the companies using LLMs have realized that the cost is going up a lot. We’re trying to identify places where it’s kind of overkill to use frontier LLMs and where we can replace them with open-weight models, now that they’ve become a lot more capable within the last six to eight months.

We explore situations where we don’t have a lot of complicated decision-making. We can use open-weight models, and we’re also looking to fine-tune some of the open-weight models for very specific use cases that contain a lot of policy learning. That means the case is narrow and niche, and we don’t want a generic frontier LLM for that. We can probably fine-tune a smaller open-weight model to handle those kinds of scenarios.

Ben Lorica. So right now, the routing you’re doing is just deciding between what tools?

Akshay Sharma. The routing right now basically acts like a filter that understands what the issue is for the customer.

Ben Lorica. Which agent to point it to?

Akshay Sharma. Which agent to point it to. And let’s say we don’t have an AI agent to handle that kind of problem. It can be anything. Someone can be asking a question about a tutorial or how to follow some steps in the app. They can be asking about a special use case, like a ride refund. They can also be asking about something that cannot be handled by AI right now.

We identify all these things, and based on what the problem is and what the issue is, we divert that to either a specialized AI agent, an educational response from our knowledge base using RAG, or escalate them to human agents if we can’t handle it.

Ben Lorica. So Nick, I’m assuming you’ve paid attention to all the hullabaloo around Jev.

Nick Ung. A little bit, yeah.

Ben Lorica. I guess I’m calling those things the decider, right? It’s trying to decide using probability or some score. It’ll give you a probability or a score, and then you can decide what to do next. Do you see a role for that in what you folks do, based on what Akshay just described here?

Nick Ung. I do. I think there is definitely a place for that as well. Jev is essentially a universal router, whether you’re trying to point a certain issue to certain agents or—

Ben Lorica. But the point is, it’s not a chatbot. It’s not going to generate text.

Nick Ung. Right.

Ben Lorica. So it’s simply helping you decide what to do next.

Nick Ung. Yeah. That’s the reasoning piece of it.

Ben Lorica. Which, by the way, is basically just going back to old-school—

Nick Ung. Old-school classifiers. Which is interesting, because we talk about this agent router that we have to decide which agent to point to. But at that decision point itself, we also have to determine, “Did the user give us enough information to make that routing decision?”

Our current flow is actually both a classifier as well as a chatbot. If the user is just saying they’re frustrated or they say something nonsensical, we don’t know which agent to route to. We have to prompt them and say, “Can you tell me more about that?”

So the current setup is a little bit of both. You have to know, “I don’t have enough information to route. I need to go through a generation model to generate a response to the end user.”

Then at a certain point, you know, “I have enough information to route,” and you can route to the downstream agent.

I think where Jev comes in is that it can make the decision of, “I don’t have enough information. I need to call a generation model,” or, “I have enough information and I can route to a multi-agent model.”

Right now, we’re just using a frontier LLM to do both. But I think there are more cost efficiencies or lower latency we can achieve by replacing some of those steps with Jev.

Akshay Sharma. I think another place where we can use Jev is for LLM judges. Instead of LLM judges for evaluations, we can use Jev because it has very structured output, like a Boolean or a score, which is very useful when we’re trying to evaluate a metric from the responses.

And because it’s very fast and a lot cheaper than frontier LLMs, it becomes a very good model to use for evaluations. Doing evaluations with LLMs is very expensive, so everyone wants to do them, but the cost, how much to sample, and at what frequency we need to do that becomes a bottleneck. I think Jev can solve those issues.

Ben Lorica. By the way, there are already a ton of open-source versions of Jev. If you go down that path, you might as well check out the open-source versions. That way, you control your data. Why send it to another API provider?

On another topic, let’s say I use one of your agents. As Nick alluded to, the prompt from the customer is something an agent can address, but maybe the agent has—I’m just speculating here—scenarios where the agent actually has sub-agents. The agent decides, “Hey, I can partially do this task, but to complete this task, let me call this other agent.”

Akshay Sharma. Yeah, we do have these kinds of cases. We have two different types of sub-agents—

Ben Lorica. And by the way, when you get to the sub-agent, you’ve got an agent calling a sub-agent. Is that still in the realm of an app that a non-technical person created, or is this already something more complicated that an ML engineer or data scientist should handle?

Akshay Sharma. We do have those flows in more complicated agents. The agents that are built by domain experts right now are simple ReAct loops. We have an LLM, we have tools, and we have a loop between them based on what the situation is.

We’re thinking of expanding those horizons by making dynamic agents with different contexts if the issue becomes complicated. But right now, we have simpler agents for the configurable agent-builder platform.

We also have big, complex agents that have different phases and can spawn sub-agents based on the situation, with limited scope and context, and then solve those issues. That’s reserved for the more complicated ones.

Ben Lorica. So Nick, what about safety? I don’t know if this ever happens at Lyft, but maybe a customer sends a malicious prompt to try to extract information. I’m just making it up. What about safety and safety-related issues?

Nick Ung. I think you’re probably referring more to malicious actors who are trying to do prompt injections or some other type of—

Ben Lorica. Or someone—I mean, you can imagine a customer opening a complaint, and then I don’t know how that turns into an attack. Or maybe they just start threatening the chatbot. I don’t know.

Akshay Sharma. Yeah. They can also ask it to expose internal policy.

Ben Lorica. Yeah, it can be anything.

Nick Ung. We generally refer to these as guardrails. We have malicious-intent checks, which are essentially an LLM as a judge. We calibrate the prompt on that.

We’re basically following traditional ML best practices: having a set of data, train, validation, and test sets, and calibrating this malicious detector or malicious classifier that basically runs on every user input message.

Ben Lorica. So this is just part of the platform.

Nick Ung. Yeah, this is built in. It’s not even something we ask the domain expert to do. All these different guardrail checks—the malicious detector, hallucination detector—are ML-provided capabilities that come with the platform. The domain expert doesn’t need to worry about it.

Ben Lorica. So Akshay alluded to RAG, grounding the agent. As best I can tell, there are three ways, right? There’s the context in the prompt, there’s memory, and then there’s actual enterprise knowledge, which could be a database or a knowledge graph.

I guess the types of agents you have must have some memory, right? At least within the conversation.

Akshay Sharma. Yes, exactly. All our customer support agents are multi-turn conversations, so we definitely need memory. We need to maintain the agent state so that, based on what the situation is, it can make decisions. And we also have—this is the checkpoint—

Ben Lorica. If you don’t mind me asking, what’s your memory solution? Do you have your own solution?

Akshay Sharma. We have our own solution. For checkpointing, which is stopping and resuming conversations, we use our internal DynamoDB. We have adapters on top of it, but we use DynamoDB because it needs to be fast and efficient.

For maintaining long-term memories and facts and these kinds of things, we have different databases like S3 and SQL tables.

Ben Lorica. For the enterprise knowledge, which will include RAG, do you use knowledge graphs, or just—

Akshay Sharma. We use vector databases for that.

Ben Lorica. Which one?

Akshay Sharma. We’re using something from AWS, but even that is in the process of improvement. We’re looking at chunking strategies and whether we can create semantic embeddings on top of it so that we’re able to retrieve—

Ben Lorica. For some reason, I thought you guys used Ray and LanceDB.

Akshay Sharma. No, we don’t.

Ben Lorica. You use Ray, no?

Akshay Sharma. I think our platform team probably uses Ray for the fine-tuning pipeline, but we don’t use Ray for RAG.

Ben Lorica. All right. Let’s do a deep dive on eval. Nick started to allude to this. Walk me through the different stages of eval.

Nick Ung. Sure. Eval is interesting. This is something I’ve noticed a lot with people building agents today, especially engineers who are taking the lead on building agents. A lot of the time, people don’t really follow ML best practices.

Ben Lorica. Well, they’re engineers, man.

Nick Ung. Yeah, they’re engineers. So I think the principle that I will always preach in AI evals is that it’s not that different from traditional ML or data science best practices.

You wouldn’t ship a neural network blind without any offline validation or offline testing. But I think the challenge, especially with these customer-facing use cases and products, is that offline eval isn’t that straightforward.

It’s not like a regression task or a classification task where you gather sets of inputs and then have labels that you can run offline eval on over and over again as you tune the model and agents.

With these customer-facing use cases, we have to understand how we recreate that production-like environment when you’re running each offline eval task. You’re actually talking to a user and finishing the entire trajectory and traces, which you can then score.

Ben Lorica. So aren’t you starting to collect an eval dataset of some sort for each of these agents? And I guess the second question is, do you use synthetic data simulations?

Nick Ung. Yeah. The way we think about this is that there’s a task specification. Think about each eval as a task specification or a configuration file.

You have the initial user verbatim and how that conversation is being started, and then you have some characteristics around what the issues are, what tools are available, and what data points are true for this particular user. You have to recreate that whole scenario of this customer support environment.

Initially, we curate these scenarios from some production data, and some of them are through synthetic data that we generated with other LLMs. It’s a mix of both.

Ben Lorica. And it’s not a static thing, because obviously these agents are interacting with real-world customers, so you’ll keep improving that dataset, right?

Nick Ung. Yeah. We’ll curate and add more data points that we think are representative of what good looks like or of challenging tasks.

Then we built this offline environment where, given each of these task specifications, we can replay that full environment and the interactions between our LLM agents and a synthetic user that speaks somewhat like a Lyft user.

If you just use a frontier LLM to role-play your Lyft user, it’s going to spit out something really polite, and your eval will be very, very easy. In a real-life customer support setting, people are usually not that easy.

Ben Lorica. But also, these agents, Nick, might be doing things, right? They might be looking things up in a database. In other words, there’s a whole harness. It might be using RAG or something. So you almost have to have all these production traces, right?

Nick Ung. Yeah. There’s the task specification about what the task is and what some of the mock outputs for the tool calls are. Then there’s that harness of how you hook up all these different components so you can reproduce or replay an entire run.

There’s a lot of scaffolding required to set this eval environment up properly, which is also why we think most people skip this step and go to production and just try to understand it there.

Ben Lorica. And because you have these historical traces, there are certain interactions where you have the desired outcome and resolution, but it turns out that the agent was very inefficient. Maybe the agent asked too many questions, checked the wrong databases, had to go back to the customer, and so on.

How do you discern that from the production traces to improve the agent so that the next time the situation comes up, boom, it’s done in one shot?

Nick Ung. This is where judging is important. How you score traces and your evaluator become a lot more important.

Conceptually, we all know that if your user LLM is asking too many questions and looking up the wrong database, it’s not going to lead to a good resolution. But how do you actually—

Ben Lorica. Sometimes it will. When you interact with a customer service person, sometimes it takes them a while. Same thing with an agent. The first thing the agent looks up could be wrong, and then the second thing is right.

I’m not sure if you have mechanisms to discern that just because the outcome was achieved, maybe this wasn’t the most efficient path.

Akshay Sharma. Yeah. I think only judging the output shouldn’t be the ideal way to evaluate.

We have trajectories that should have been followed by an agent if it’s trying to solve a customer issue. If there are wrong tool calls in between, or wrong database reads in between, we have checks specifically for those tool calls. Was this tool called correctly? Was it called with incorrect parameters and then went back to the LLM node and was called again?

We have these sorts of metrics when we’re trying to judge the agents.

When you talk about identifying patterns and identifying signals from production traces, I think it’s a difficult task. We do have to have some manual intervention in between. Data scientists can help in those aspects, where we can identify patterns.

It’s like how we do EDA in ML, doing some sort of error analysis. What are the patterns? What are the common issues we observe when we look at the production traces?

I think evals should start with that first step. You’ll only know your actual issues once you look at the production traces. So there is some data science involvement in understanding and identifying the patterns.

From there, we try to convert them into metrics or evaluation prompts that we want to actually evaluate in production for these agents. Then we see how these are performing and what we need to improve based on the metric scores we see in production.

Ben Lorica. It’s like the production traces and production trajectories become a form of training data.

Akshay Sharma. Exactly. Traces are the most important thing, I think, for agent stuff. They’re the source of truth for any kind of data.

If you want to do fine-tuning, continual improvement, judging—anything and everything comes from traces. I think those are the most important things.

Ben Lorica. What about the harness? That’s also internal?

Akshay Sharma. That is internal, yes.

Ben Lorica. You’re not using Codex or Claude Code?

Akshay Sharma. No. We have used LangGraph, which is an open-source framework by LangChain. But we built our own harness.

We use nodes, we use graph and subgraph patterns, and we have our own state machine where we have our own state schema. Everything around it is internal. The harness is built by us.

Ben Lorica. And the eval right now—do you consider eval a piece of shared infrastructure, or is it still mostly kind of ad hoc?

Akshay Sharma. I think it’s definitely shared infrastructure, and it’s becoming more and more important because, as agents become more complex, it’s becoming difficult to identify issues and even solve them.

Anyone who’s working on multiple agents or a multi-agent architecture—even within a company, if multiple teams are working on different agents—can definitely benefit from an evaluation platform.

It can be a platform layer that’s tied to data curation with your own traces from production. Then it can be tied to annotations. It can be tied to offline evals. It can be tied to LLM judges, and even fine-tuning later on.

Ben Lorica. So Nick, has there been a noticeable improvement in terms of this eval infrastructure in the following sense?

In the early days of this agent platform, you would deploy agents and then withdraw them in two days because, for some reason, they weren’t hardened enough. But now, because the eval infrastructure is much better, the agents tend to be much better right off the bat.

Nick Ung. Yeah. I think that’s sort of the evolution of our eval journey as well. As these ideas of trajectory, eval harnesses, and task specification really solidified, we also built this shared infrastructure where different teams can build upon these different infrastructure layers.

Even some of our test engineers are able to contribute to our test eval use cases. Building a shared understanding of where and how we run these evals certainly helps streamline the end-to-end agent development lifecycle.

Before, when we first started, we would do some testing in staging, feel good about it, roll it out to production, and then a bunch of people would be all hands on deck looking at traces that were coming in to see if it was doing any good.

The beautiful part about eval is that we get this firsthand experience of what a good eval is, and then we slowly automate these different parts of it to make our lives easier, building these automated observability and improvement layers to help the next agent get better.

Ben Lorica. All right, so we’re winding down. Let’s do a quick round of tips for other teams building either agents or tools for an agent platform just like yours.

Let’s do two tips each, but one at a time. You don’t have to stick with your area. Akshay doesn’t have to stick only with infrastructure, and Nick doesn’t have to stick only with ML.

Akshay, you start. Tip number one.

Akshay Sharma. I think tip number one is that prompt quality is probably more important than you think.

As agents have evolved through multiple generations, initially we had single-shot agents. Then we had these complex multi-turn agents with our own orchestration. Now people are moving this orchestration piece to the LLMs.

Everything is now kind of controlled by these frontier LLMs: what decision to make, what sub-agent to spawn, which tool to call, even creating multiple sub-agents in parallel to work on different things.

Now that a lot of these things are abstracted from us, even the low-level orchestration, it becomes more and more important to focus on how you write the prompt.

The difference between a good engineer and a bad engineer, if we all have access to the same tools, is how you use your own brain in reviewing things and how you write verbatims or write the prompt to make use of that AI. I think that’s the difference.

Ben Lorica. Does something like DSPy help?

Akshay Sharma. I haven’t used it personally, but yeah, probably.

Ben Lorica. Yeah. Nick, tip number two.

Nick Ung. Tip number two: I think, as I said, eval is your competitive advantage.

A lot of teams may initially be looking for product-market fit, and eval wasn’t so much of a focus at first, which is totally understandable. A bunch of Series A and Series B companies are trying to make it.

But as you really think about, “Okay, what’s next? How do you continue to improve your product?” without eval, it just creates a blind spot.

I think eval is non-negotiable. It’s a little bit of an art to get it right in the first place, but deeply investing in your eval infrastructure is how you go a long way.

Ben Lorica. And what are some resources if people want to learn how to do this? Do you happen to know any good resources?

Nick Ung. Especially for product teams, I highly recommend Hamel Husain and Shreya Shankar.

Ben Lorica. Yep.

Nick Ung. They’ve been doing this for a while. They have a course. I’m not marketing for them, but I think they share a lot of good insights and blog posts about how to approach eval, especially as you’re building a zero-to-one product in the beginning.

Ben Lorica. Both have been on the podcast.

Nick Ung. Both have been on the podcast. Yeah. I’m a big fan of them. I think the blog posts are great.

And LangSmith has also done—folks at LangChain have a really great philosophy on eval as well. I highly recommend checking them out.

Ben Lorica. Akshay, tip number three.

Akshay Sharma. Tip number three: I would say let’s not blindly follow what other companies are doing.

This AI space is so dynamic, and it’s changing every day. New models are coming in, new products are coming in, and people are building application layers on top of LLMs. Every day you hear things like, “RAG is dead,” “evals are dead,” and things like that.

But I think you shouldn’t just blindly go with the flow. Understand your own problem, because different teams and different companies have different use cases with different levels of criticality.

I think it’s important to stick to the basics and then have a vision. What is your infrastructure or agentic platform going to look like one year from now or two years from now?

If we cannot envision that, and we cannot build a scalable platform to support that, I think that’s the gap we need to fill.

Ben Lorica. Nick, tip number four.

Nick Ung. I think one thing we learned early on in our journey is to be as flexible as you can with your tech stack and platform, and don’t introduce lock-in too early.

One of the reasons we’re using LangGraph and LangChain is that they’re overall model-agnostic, bring-your-own-model types of frameworks.

There were a lot of other agent SDKs early on that were very popular. OpenAI has an SDK, Claude has their own as well.

I think the reality is that agent engineering is simpler than it looks. You don’t need to go with a solution that abstracts too much of the detail. Owning that low-level layer is actually also part of your competitive advantage.

If you want to do model customization in the future, keeping your tech stack open and agnostic is the way to go.

Ben Lorica. All right. I’m going to close with my own two tips and have you guys either agree with me or tell me I’m crazy.

The first one is: Assume that your agent will make mistakes. That means you have to design for some sort of recovery from mistakes. That could be checkpoints. That could be some other form. What do you think about that?

Akshay Sharma. Yeah, definitely agree with you, Ben. It doesn’t matter how good the LLMs become. There’s definitely always scope for error.

It can be because the prompt isn’t well written. It can be because the tool docstring isn’t well written. It can be because of a million reasons.

So definitely assume that there are going to be mistakes. It cannot be perfect. Build your guardrails around it and focus on evaluation so that you can build a robust system, especially if the use case is critical.

Ben Lorica. Tip number two is related, but there are actually two things in here.

One is: Give the agent only as much autonomy as it needs. In other words, keep them a little bit more boxed in. I’m not comfortable with OpenClaw running on my laptop.

And also, don’t confuse your agents with too many tools. In other words, if you have multiple search engines, multiple ways of doing the same thing, the agent will just get confused.

What do you think about that, Nick?

Nick Ung. Akshay’s probably got a lot to say about this, but this is why we have progressive disclosure. We only give the agent as much information as it needs at a given point in time.

I think I showed you some study out there where model performance degrades when you have a certain number of tools. Context rot is definitely real.

On your first point about only giving the agent as much autonomy as it needs, I’m more of a proponent of giving more autonomy than boxing agents in. I think you’ll be surprised.

Ben Lorica. So your agents may end up exploring Hugging Face.

Nick Ung. Well, obviously, we don’t want it to do that, and we don’t want it to delete records from our database. That’s where we’ll limit it.

But what I’m trying to say is, especially in the early days of LLM product development, product managers would come in with these strict workflows and Figma rules. Then it’s not an AI agent. It’s just an agentic workflow.

There’s very specific logic branching in how the PM wants the product to work. I think that limits the ability of agents to handle the non-golden paths. It creates a lot of performance issues or quality issues in production.

So I don’t want people to think that when you design an AI agent, it needs to strictly follow, “If A, then B, and then C.”

Ben Lorica. One more thing. When it comes to multi-agent systems, keep it small, because I think right now most teams are not yet at the point where—I think multi-agent systems are a bit of an advanced thing.

A lot of things can go wrong. For one thing, there’s communication between the agents. Then there’s temporal logic, the sequence of events.

I’m not sure how much multi-agent work you folks use, but I’m a little wary of it. Learn to walk before you run, I guess, is my tip when it comes to multi-agent systems. What do you think, Akshay?

Akshay Sharma. I think that’s a good tip, Ben. But our team does a lot of multi-agent architecture, and I think in the current industry there are ways to efficiently do the communication.

I just don’t want people to overdo it so that there’s bloat in your checkpointing table or context bloat that causes agents to hallucinate or get confused.

I think you can build multi-agent architectures for use cases like customer support, but beware of all the consequences and make sure you have all the checks in place.

You should be doing context summarization or truncating to prevent bloating. You should have progressive disclosure for tools. You should have some skills. That’s why we have skills. Skills will tell you which tools to use instead of introducing 100 tools to the agent at the beginning.

Make sure you understand which part of the state you want to share between agents and which part of the state you shouldn’t share.

There are a lot of design principles that come into the picture. But if you make sure those are in line with your expectations and your robust design, I think it becomes easier to build a multi-agent architecture.

Ben Lorica. And in closing, I like what Nick said about eval. The one thing I will add is that I think a lot of people confuse eval with evaluating the model, but it’s really the entire harness.

It’s everything, because your agent might be retrieving things from a database. It could be doing all sorts of things. It’s not just the model. It’s the entire harness.

And the teams that get really good at eval, I think, are the teams that will ultimately win.

Nick Ung. Couldn’t agree more.

Ben Lorica. Yeah. And with that, thank you, Akshay and Nick.

Nick Ung. Thank you, Ben, for having us.

Akshay Sharma. Thank you so much, Ben.