An open source and end-to-end library for causal inference

Amit Sharma and Emre Kiciman on making causal inference and learning more accessible.

SubscribeApple • Android • Spotify • Stitcher • Google • AntennaPod • RSS.

This week’s guests are Amit Sharma (Principal Researcher) and Emre Kiciman (Senior Principal Researcher) of Microsoft Research. We talk about practical applications of causal inference, a set of tools and techniques that enable data teams to draw causal conclusions based on data.  Amit and Emre are part of the team behind DoWhy, a new open source library for estimating causal effects based on historical data alone, particularly useful when we cannot run an experiment because of time, expense, or ethical concerns. In this episode we discuss the growing interest in causal inference, and we do a deep dive into the state of tools (also see this paper on DoWhy).

Amit Sharma:

So whenever you have a decision, for example, whether the vaccine works or not, or what kind of economic policy a country should adopt, these have been historical decisions, where causality has traditionally been applied. But today, what we are seeing is with AI systems being deployed in many different settings, the same kinds of questions are also coming up for those systems. Let’s say when you are recommending certain jobs to people, or when you’re recommending financial decisions, we want to make sure that those decisions also have the right impact for users. Then there are also questions about fairness and robustness of systems. Causal inference is the sibling of prediction in terms of scientific inquiry. … Causality comes in when you want to ask, “I see the system proceeding in a way I now want to change the systems, I want to make the system better, what should I do, so that my intended action actually also has the right consequence on the system.”

Emre Kiciman:

What we saw is other libraries focused really on  statistical estimation. In some sense, that makes sense. That’s the hard part to code. The other pieces are easy to code if you know what to do. But our end-to-end solution really is aimed at data scientists who want to use causal inference.

Highlights in the video version:

Related content:

FREE report:

[Image: Newton’s Cradle from Wikimedia.]