Why Your AI Committee Might Be Your Biggest AI Problem

Ben Lorica

Ben Lorica and Evangelos Simoudis on AI Committees, Innovation Pods, and Generative Engine Optimization

Subscribe: Apple • Spotify • Overcast • Pocket Casts • AntennaPod • Podcast Addict • Amazon •  RSS.

In this episode, Ben Lorica and Evangelos Simoudis discuss how enterprises are structuring their internal AI efforts, highlighting the friction between top-down AI governance committees and bottom-up “shadow IT” innovation. They also explore the emerging concept of Generative Engine Optimization (GEO), explaining how brands must adapt their information architecture to remain visible as consumers increasingly replace traditional search engines with AI chatbots.

Interview highlights – key sections from the video version:

Jump to transcript

Subscribe to the Gradient Flow Newsletter

Related content:

Support our work by subscribing to our newsletter📩

Transcript

Below is a polished and edited transcript.

Ben Lorica: All right, so we’re back with our regular check-in with Evangelos Simoudis, and I’ll link to his blog in the episode notes. Topic number one: Evangelos wants to talk about how companies are organizing around AI. So, Evangelos, very quickly, what are you seeing?

Evangelos Simoudis: It’s not so much how companies are organizing around AI, but really how it started is that we’re seeing companies setting up these AI committees. Sometimes it’s under the guise of governance, sometimes under experimentation—you know, what to try with AI. And I think there are some issues there that require rethinking.

Ben Lorica: In the past, the phrase that was used was “Center of Excellence.” So is this the equivalent of a Center of Excellence, or not as formal?

Evangelos Simoudis: Actually, I would say that every time we have a new technology platform, or a new technology, there are two things that corporations tend to do. In my case, I first encountered it when I started working, when I was leading IBM’s AI division with business intelligence.

The Center of Excellence tends to deal with the technology implementation and dissemination in the corporation. But then there is a governance body that decides what the practices are going to be, what the issues are, and what we should try. And my first comment is that this second body, that governance body, many times has a deceleration effect on how the technology—in this case, AI—is being adopted by the enterprise, what is being tested, and how budgets are allocated.

Again, different enterprises assign a different mission to this group, to this governance group, to this AI committee. Broadly, I will call it an AI committee. So I think it’s important to chat a little bit about what we’re seeing and maybe what’s a better way of thinking about it. But we really need to distinguish between an AI committee and an AI Center of Excellence.

Ben Lorica: The reason you wanted to talk about this is that you believe this is an important decision for companies and that there are certain pros and cons. So let’s first go with why companies are doing this. Because to me, that signifies that they believe this is the right way to go. So what is their reasoning?

Evangelos Simoudis: First of all, companies recognize that AI, this time around, will be transformative for their operations. So, check on that. Number two, they are starting to be concerned about the bottom-up efforts that are proliferating within many enterprises. We have, again, the reincarnation of so-called shadow IT, right? We have a lot of efforts, and we’ve talked to many enterprises that admit they are losing control of who is doing what from the bottom up. So they’re trying to put some order into this. And that order could be: how can we avoid IP leakage? How can we avoid cybersecurity problems?

Ben Lorica: So the usual things.

Evangelos Simoudis: The usual things, exactly. Now, unfortunately, what is happening—

Ben Lorica: Before you proceed, is there also a bit of politics? In other words, “I want the budget for AI, so I…”

Evangelos Simoudis: “I want the mission for AI, I want the budget for AI, I want my people on this committee,” right? Because, again, corporations have—and we’ve seen this, this is not new, right? And this is not specific to AI. I’m being very deliberate about stating that.

A lot of times, what you end up with in corporations is people ending up on these committees without having the proper background, the proper ability to distinguish between what is critical, what is important, and what direction the corporation should be going in, right, to provide that overall guidance. There is control of the budget, which can be difficult.

Ben Lorica: Before you proceed further, as you’re talking there, it just occurred to me: can you give our listeners a sense of how prevalent this approach is, in your mind?

Evangelos Simoudis: I would say that most large corporations—let’s call them Fortune 1000, Fortune 500 to 1000, or whatever—but most large corporations have established these committees. They may go by different names, but there is starting to be a body.

Ben Lorica: But even then, there are certain gradations, right? Some of these committees are more powerful than others.

Evangelos Simoudis: Yeah, and I would say that, obviously, like with every committee, you have who is setting it up, who the members are—

Ben Lorica: No, but what I mean is Ford may have a committee that’s super influential, but then GM may have a committee that basically exists only on paper, right?

Evangelos Simoudis: You’re right about that. You’re right about that. And I was also going to say that one of the things creating a reason for these committees is regulatory uncertainty. Right? And that depends on the field. Obviously, if you’re in biotech or if you’re in automotive or mobility, regulations play an important role. If you are in, I don’t know, retailing, maybe regulation is less important.

Ben Lorica: So in your mind, the ones where the committees are real—how powerful are they? In other words, I have an idea for AI. Will these committees really impact me?

Evangelos Simoudis: Yeah, because essentially they dictate—or they could dictate, again, depending—they could dictate what pilots you start, how you evaluate your pilots. I mean, we’re talking to a bank, for example—

Ben Lorica: What models or tools you’re allowed to use.

Evangelos Simoudis: Yeah, that’s right. What budget you have, what you can and cannot do. But there is the positive side of what you can and cannot do, and the negative is that you may be ignoring really important things in the process because the committee is not set up properly, or the members of that committee have—I keep calling it a knowledge gap—in terms of what is possible, why AI should be part of the corporation’s strategy, and the like.

Ben Lorica: Before we go into the reasons why these committees may not be a good idea, have you talked to people inside some of these corporations where these committees are actually working well?

Evangelos Simoudis: Um…

Ben Lorica: Oh, so your hesitation there is the answer.

Evangelos Simoudis: Well, I will say the following. In corporations that have strong leadership on AI—in other words, where AI is not something new, but they have been working on it diligently and deliberately over the years, and they have set the right practices and all that—instituting an AI committee acts as an accelerant, right? The problems begin when the corporation discovers AI now and says, “Oh my God, we really need to be here and…”

Ben Lorica: “So let’s set up a committee.”

Evangelos Simoudis: Yeah, and sometimes it’s influenced by vendors and sometimes it’s influenced by the board. And now you have all of the issues that can come with that type of haphazard, let’s call it, organization, just so that you can say, “We have a committee, check,” right? “We have a Center of Excellence, check.” Okay—what do they do? What are their resources? Why should this person be part of this committee? What are they bringing to the committee? Those are some of the issues that we are seeing.

Now, the companies that are doing it well, I would say, bring together three competencies at the business-unit level, right? There is a deliberate effort for the business unit to say, “Here’s my mission, here’s what I’m trying to achieve with AI, and here’s how I’m going to start doing it on a step-by-step basis.” And typically, these groups bring together three competencies. They bring together the people who understand the business unit operations well, typically the general manager or somebody in the direct reporting line of that general manager. They bring IT, which has the technology know-how. And they bring together HR.

And HR actually, when I first started thinking about it, was not the most obvious addition. But it’s obvious because it helps you understand both the strengths of your current team—who is going to assess how the process is going to be modified or which processes are going to be selected for automation—but also who to go hire. Because today, for a variety of reasons that we don’t have to go into in this session, there is a lot of fear. Because of uncertainty, employees relabel themselves in very aggressive ways. And I can understand why this is happening, but it can be very detrimental to the enterprise. So we work with corporations where they say, “Oh yeah, we have a lot of AI experts,” and when you do a deep inventory, you realize that they don’t. Now, it’s not that they have nobody, but there is a big delta between perception and reality.

And that’s where I think engaging HR in a very focused way helps. So this triumvirate of business unit, IT, and HR working in unison—without walls, without the business unit sending a memo to HR and saying, “Oh, you know, we need three data scientists and five prompt engineers”—but having them in one group and having them work without walls, without silos. Those are, to me, the committees that work very well. And I don’t call them committees. I call them innovation pods or ambidextrous pods. But this is when you bring together a group of complementary expertise and you make sure that everybody is at the same level of knowledge regarding AI. I’m not talking about the nitty-gritty details of how the model works, but more, “Here’s what AI can do for our corporation, for our use cases, and we move forward.”

Ben Lorica: So two rapid-fire questions to close. Keep your answers short. The first is: it’s interesting that you had this triumvirate, but you didn’t include legal and compliance. So it seems like you would need them early on in certain domains, right?

Evangelos Simoudis: In certain domains, I agree with you.

Ben Lorica: Yeah. And secondly, the ones that are not establishing these pods or committees, what are they doing?

Evangelos Simoudis: They’re having, again as I said, these groups that are asked to make important decisions, but in the end, they end up slowing things down.

Ben Lorica: So are the committees centralized? Or is there a decentralized version of the committee?

Evangelos Simoudis: There might be. I do not know. The case that I’m talking about is where the corporation says, “We have a centralized AI committee, and if you want to bring your application, your expertise, whatever, through our corporation, you need to go through that committee.”

Ben Lorica: I see, I see. And so then the ones who don’t have that committee just basically do what?

Evangelos Simoudis: Well, as I was saying before, in many instances you have this bottom-up approach, which has its own issues and detriments, right? I’m not advocating proceeding just through this bottom-up enthusiasm.

Ben Lorica: Right. So the difference between the two is the committee might be perceived as slowing you down, and then with bottom-up you may move faster, but each has its own strengths and weaknesses, right? By the way, on the bottom-up side, let’s say you can actually do the bottom-up well, you still need a mechanism where a team over here can learn from a team over here, right? Because if a team over here already did something similar, you need to be able to share that somehow.

Evangelos Simoudis: Exactly. And look, the last time I saw this was when SaaS applications were being introduced in the enterprise and you had individual employees pulling out their credit cards and getting subscriptions. And that was encouraged, right, by the vendors for obvious reasons. And that led to many issues for the corporation, which again caused corporations to change their practices. I actually think that this time around, with AI, the stakes can be even higher. Because CRM systems and HR systems became systems of record, so they were collecting data. AI systems become systems of action and systems of inference. And again, from my learning position, they can have a much bigger impact.

Ben Lorica: All right. So the next topic is something I wrote about recently, which is around information architecture in the age of AI. By that I mean, increasingly people are using chatbots in place of search engines. But interestingly enough, this article—which I will link to in the episode notes—came out of conversations with friends in the marketing space. And in retrospect, it’s obvious, right? Now that people are using chatbots, what’s the equivalent of search engine optimization? And it turns out that people are starting to think about this, and I think increasingly—

Evangelos Simoudis: Even I myself, anecdotally, needed a very specialized electrician recently. I used Gemini and ChatGPT to find a person in San Francisco because they both recommended the same person. The person came over, and I told him how I discovered him. He was really surprised, but then immediately said, “Huh, maybe because I’m active on Reddit.”

Ben Lorica: Right. So anyway, this area has many labels. One of them is generative engine optimization, answer engine optimization, and so on. But the point is that you need to start thinking about it. And what I try to lay out in this post are some best practices that people are starting to use. I’m not going to go through them right now, but I mentioned one, which is: be active in forums.

I think at the most basic level, if you’re a company or a brand, the first thing you should do is go to all the chatbots and ask about your topic and see if you appear, right? So it’s almost like running a search and seeing if you appear on the first page. The same thing here. So anyway, Evangelos, it seems like this is a topic that more companies and brands should be thinking about, right?

Evangelos Simoudis: Well, they’re thinking about it. Actually, I think they’re thinking about it from two angles. The first and most important is that, as commerce and agentic interactions are starting to be incorporated into this AI web, corporations need to determine—corporations from the Fortune 10 to mom-and-pop shops that have a presence on the web—they need to start determining how to represent themselves to the agents, right? Because otherwise, they run the risk of being completely disintermediated.

Ben Lorica: Yeah, yeah. I have super-concrete steps that you can take. But the question is, to me it doesn’t seem like people are actually doing much about this yet.

Evangelos Simoudis: I don’t think they are. Look, Ben, I’ve told you before that one of the things that is fascinating to me this time around is that AI can be as transformative as the Industrial Revolution. And these types of transformations take a long time to materialize and have their intended impact. But we’re treating it in many respects like something that has immediate impact. And I think people are still digesting it, they’re still getting to know what is possible. So I think that’s why we’re seeing that.

Ben Lorica: And the reason you should think about this now is that this is a long game that is cumulative. These AI models get trained on data now, and it’s a really strange thing because if you’re not active on places like Reddit or wherever else you need to be active, someone asks a question about a topic that your brand or product plays in, and you’re not going to show up there. Or a reporter wants to write an article about a topic and your brand doesn’t show up there, the reporter writes the article, and then the next generation of AI gets trained on that article. So it’s a cumulative thing. The next generation of AI is actually using data generated by the previous generation of AI for training.

Evangelos Simoudis: So let me ask you this—

Ben Lorica: If you’re not doing anything about whatever this is called—generative engine optimization—you’re going to see this erode your brand over time. And some people even have a term for this: they call it dark revenue loss, because you’re losing revenue but it’s not something that will ever show up in your analytics dashboards.

Evangelos Simoudis: Right. So do you think that, in the same way that we had search engine optimization and we specialized what words to use and what terms to use, and the SEO engines became popular, do you think we will get to that point with how we represent special terms so that model training picks them up differently?

Ben Lorica: I think so. You need to do a few basic things, right? One is you need to make sure that the AI crawlers can crawl you. Otherwise, they won’t even see your content. And then, as you said, make sure that your content is optimized so that AI can glean information quickly. So it could be in a structured format, it could be FAQs—frequently asked questions. It could be in short, information-dense paragraphs where things are clear.

These are things that a lot of people are trying to figure out. Obviously, just like SEO was highly empirical and a bit of a black art, I think this will also be somewhat of a black art. But the point is that if you’re not doing anything about it now, like I said, reporters are using these tools to write articles, and then those articles are going to be fed into the next generation of AI, right? So—

Evangelos Simoudis: Yeah, actually I will say—or podcasters like us, before we go on a podcast, will research with a chatbot.

Ben Lorica: We’ll use a chatbot to do research and then, you know—

Evangelos Simoudis: So, we’re working with a few retailers, and for retailers in particular—and I think your example shows why—this is becoming a big issue. And I think you need to distinguish between companies like Amazon and Walmart that have their own agents, and they already know how to expose their catalog, how one agent can deal with another agent. I mean, we’re not yet at the point where we can rely on an agent to close the transaction.

Ben Lorica: And there’s also the possibility that the really big companies—maybe not officially through ads like what ChatGPT has—but they may have deals with some of the big chatbot creators to make their content available, or they may talk directly to the chatbot creators and provide their data for training in a much more friendly way, right? We don’t know all of those things.

Evangelos Simoudis: Yeah. Today, I would say, if I were to summarize the state of practice, particularly for retailers, it’s that they’re looking to have a presence so they can generate demand—kind of like top-of-the-funnel demand. Very few are starting to think about, “How does my brand operate in a fully agentic world where a consumer issues a request to their agent to go find something, but also close the transaction?” So the consumer essentially has something delivered to their door, and the whole end-to-end transaction was completed by agents. We’re not there yet. I mean, we will get there—that’s the whole idea of agentic commerce. Today, retailers in particular want to make sure that chatbots and whatever tools consumers use are able to bring top-of-the-funnel opportunities to them.

Ben Lorica: By the way, in this highly specialized world of retail, you can imagine consumers prompting these agents the way they would fine-tune a search engine query, right? For example—I’m just spitballing here—if they trust a review site like Wirecutter, they might say, “All right, I’m looking for a consumer drone, and I really trust the reviews from Wirecutter, from Wired, from this and that,” and that’s how the agent would go about it.

And so that means you need to make sure that you are all over the place. That’s why I think increasingly people are becoming active on places like Reddit, for example, right? So if you are in whatever topic you’re in, you don’t have to be there constantly shilling your brand or your product either, right? But it could be a feature that you’re particularly strong in—

Evangelos Simoudis: I think it’s going to be about showing your differentiation because—

Ben Lorica: Yeah, thought leadership at scale, basically, right? Yeah. And with that, thank you, Evangelos.

Evangelos Simoudis: Thank you.