Delivering Continuous Intelligence at Scale

Simon Crosby on distributed actors and data infrastructure that enable companies to analyze, learn and predict on the fly.

SubscribeApple • Android • Spotify • Stitcher • Google • AntennaPod • RSS.

Simon Crosby is CTO of, a startup building tools (based on the Swim open source project) for next-generation data and AI applications. Swim is one of several projects (along with Ray and Akka) contributing to interest in the Actor Model for building large-scale machine learning and data applications and infrastructure. Simon describes the applications made possible by technologies like Swim, and notes that these technologies require considerably less hardware and DevOps resources. If you’re interested in building AI applications you need to listen to this episode.

Simon Crosby:

There are three kinds of data: first is data in a SQL database, then there is big data, and finally we have the case where every single process, every single product, every bit of infrastructure is instrumented. … The next generation of IT systems have to stay in sync with the real world. And when I say in sync, I mean, you need to be able to drive a robot, or when you look at your Uber, you really want to know the car that you’re tracking is in front of you, and not minutes away.

… For applications that need to be in sync with the world, you have to analyze, learn and predict on the fly. You can’t afford to store data first, and then analyze.

… Let me give an example from a large telecommunications company. I mentioned before that they accrue around four petabytes of data a day. … They were using 400 servers worth of big data per day, and with Swim and Actors they’re down to 40!  Not only that, it used to take them 10 hours to produce insights, that’s down to 10 milliseconds.

… The other cool thing here is that we can automatically build digital twins, which are needed in many AI applications. Every single additional twin is just an actor somewhere in memory. And in Swim every actor is also on the web. It’s API so it’s vertically integrated.

Highlights in the video version:

Related content:

FREE report:

[Image courtesy of Simon Crosby.]