The Data Exchange Podcast: Nicolas Hohn on challenges and best practices for using RL and machine learning in the enterprise.
Subscribe: Apple • Android • Spotify • Stitcher • Google • RSS.
This week’s guest is Nicolas (Nic) Hohn, Chief Data Scientist, McKinsey/QuantumBlack Australia. Nic led a team of data scientists charged with helping America’s Cup winning team, Emirates Team New Zealand, test new designs for hydrofoils – important sailing boat components that could be modified based on rules set forth by race organizers. More precisely the QuantumBlack team used Ray RLlib to design an AI agent that could learn to sail the boat for a given design at an optimal speed, and this AI agent proved crucial during the design process.
As many listeners may be aware, I’ve recently written a series of posts describing applications of RL within companies (see: “Enterprise Applications of Reinforcement Learning: Recommenders and Simulation Modeling” and “Applications of Reinforcement Learning: recent examples from large U.S. companies”). Given Nic’s position working with a wide variety of companies, I asked him about how companies can incorporate RL into products and services:
- Nic Hohn: ❛ You need to have the right circumstances for RL to be the right answer, as opposed to trying to make RL the answer to any type of problem. In my experience, some of the key ingredients are highly complex optimizations usually involving a sequence of actions of a time. I think RL is also really good when you only learn the consequences of your actions even further in time. These are situations where the reward might not actually be instant, for instance. And so keeping this very broad lens, I think the examples … could be dynamic supply chains, or it could be next best action models are complex optimization of a telecommunication network, where your needs will vary depending on how the users are interfacing with the network.
I think in all those situations, I think I would strongly advise against using RL as the starting point. But if you find that traditional techniques are limiting, it could be that you need a lot of heuristics in your optimizations to make it converge. Or it could be that you can only run it up to a certain scale, and then your optimization no longer works at all. There’s many reasons why you might want to go further. But you do need to do that hard work first. So that you have a clear idea of what you’re solving for, you have a good idea of your optimization function. Only then do you bring in the Deep RL.
I think in terms of the barriers to entry, a lot of the challenges we see currently are really around getting access to a good simulation environment. RL right now only works if you have a simulation environment – you can call it the digital twin, you can call it something else, it doesn’t matter. You need some environment with which your agent can interface. Sometimes the challenge on the environment is that people think they need the highest fidelity simulator, otherwise they think it’s not worth it. I would argue against that, I think it really depends on the problem you’re solving. Sometimes all you want to know is whether you should be choosing Option A or Option B, because you have a whole bunch of other constraints. So in that case, maybe a simpler simulator is enough.
Download a complete transcript of this episode by filling out the form below:
Related content and resources:
- A video version of this conversation is available on our YouTube channel.
- “Applications of Reinforcement Learning: recent examples from large U.S. companies”
- Model Monitoring Enables Robust Machine Learning Applications
- Zhe Zhang: “How Technology Companies Are Using Ray”
- Travis Addair: “The Future of Machine Learning Lies in Better Abstractions”
- Piero Molino: “Making deep learning accessible”
- Max Pumperla: “Connecting Reinforcement Learning to Simulation Software”
- Dean Wampler: “Scalable Machine Learning, Scalable Python, For Everyone”
2021 Data Engineering Survey
The 2021 Data Engineering Survey is now open and we need your help. The survey takes less than 5 minutes to fill out and we’ll share the report of the survey findings with you. You’ll also be entered in a drawing for free copies of the Data Teams book and other prizes.
Subscribe to our Newsletter:
We also publish a popular newsletter where we share highlights from recent episodes, trends in AI / machine learning / data, and a collection of recommendations.
[Image: America’s Cup World Series San Francisco by Phil Uhl on Wikimedia.]
