A brief on Causal Impact Analysis

What is Causal Impact Analysis?
Causal Impact Analysis is a methodology based on advanced machine learning and statistical concepts which allows an analyst to measure the incremental effect of a “change” on a time series. Now this time series can represent the returns of a new marketing campaign, a feature change in a product, or a product pivot itself.

How does it work?
Assume you are the Marketing Manager of a music streaming company and you launched a great campaign last quarter to grow your daily active user base for your Android App. Great! It has been 4 months and your boss asks you to show how has this campaign benefitted the company.

This is where an analyst comes in. Now if S/he’s up to date with the current trends of Marketing Measurement. S/he shall definitely run a Causal Impact Analysis and let you know how much your campaign affected the active user base.

Get Ayush Tripathi’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Steps Involved

• Will collect the data for daily active users of the music streaming app. It will be a time series data. And the data shall be from last year to the present. (To keep it simple right now)
• An Intervention date shall be defined. This is when the campaign went live. Say Q2 1 April 2022
• The collected data shall be divided into two parts PRE and POST using the above-defined Intervention Date.

A sample time series data with intervention

• Now, the PRE time series data shall be used to train a model and estimate a “COUNTERFACTUAL”. That is, the daily active users in the POST period if there had been no marketing campaign.
• Once we have the estimated Counterfactual, all that is left to do is compare the Observed POST with it and analyse the differences.

Comparison of Counterfactual and POST time series outputs

Conclusion
Now, that doesn’t sound too complex right? It doesn’t, but actually doing it has a lot of nuances involved. For example, the PRE data should be a proper control group else the results won’t be a direct representation of the “Effect of Intervention”. Or the fact that you need to fit a time series model which is a good representative of your data

So to make your work easy, we do have a ready-to-use package PyCausalImpact in Python or CausalImpact in R. This was developed at Google and what it allows you to do is to focus more on the analysis and less on fitting the right statistical model or coding, etc.

(For a deeper understanding you can either explore the documentation of the package or reach out to me here)