Causal Impact Analysis in R
Causal Impact Analysis is a statistical analysis used to measure the effects of an intervention (policy change).
Through the analysis, we are able to answer the following questions:
1. Did the intervention have a significant impact on the outcome?
2. How much of the observed change can be attributed to the intervention?
Normally, when we want to assess these effects, researchers design randomized control trials to investigate this. However, these experiments are sometimes expensive or unethical and therefore not suitable.
Applications
Impact analyses are applicable in diverse industries such as;
1. Health: Impact of vaccinations on infectious disease cases, the impact of lockdowns, the impact of new drugs
2. Economics: Impact of policy changes to economic indicators such as interest rates, the balance of trade, consumer price indices
3. Marketing: Impact of campaigns and promotions on customer behaviours
Assumptions
Like any other model, causal impact analysis are applicable under the assumptions:
1. Pre-intervention trend: The outcome variable should follow a stable trend before the intervention, allowing for a reliable counterfactual.
2. Parallel trends: The treated and control groups should have similar trends before the intervention, ensuring that any observed differences in the post-intervention period are attributable to the treatment.
3. Stationarity: The relationship between the control variables and the outcome should remain stable over time.
Implementation in R
Steps
To perform Causal Impact analysis in R, you will need:
1. Install the package
install.packages("CausalImpact")2. The dataset should contain the following variables
a. Date
b. Variable of interest containing values whose effect we are interested to analyze
c. co-variate. This is any variable(s) that remain constant and are not directly impacted by the intervention
3. Create time series objects for each of the variables in parts b and c
4. Define the pre-intervention and post-intervention periods
5. Create a zoo object for all the variables to merge the data
6. Perform the analysis
Importing the Data
The data for this project is from the Central Bank of Kenya (CBK). It contains monthly interest rates set by the CBK, and commercial bank interest rates for loans and deposits from June 2006 to October 2019.
The Kenya Gazette dated 31st August 2016 published the Section 33B of the Banking Act that had been introduced by the Banking (Amendment) Act, 2016. It provided for, among other things, a ceiling of 4% over the rate set by the Central Bank of Kenya on interest charged by financial institutions regulated under the Banking Act.
This legislation saw Kenyans enjoy affordable interest rates and was abolished in October 2019 following the President’s recommendation to Parliament to repeal the interest rate cap law has sailed through Parliament, paving the way for the repeal of the law.
The President refused to assent to the Finance Bill, 2019 (the Finance Bill) and referred the Finance Bill back to Parliament with a memorandum outlining the reasons for his refusal. His concern was the presence of the interest rate cap and he recommended that it be abolished by repealing section 33B of the Banking Act.
Get Ian Leboo’s stories in your inbox
Join Medium for free to get updates from this writer.
In this analysis, we assess the impact of this legislation. The variable of interest will be the difference between Commercial banks' interest and CBK's interest.
Importing the Data
Visualizing the Difference in Rates
The line plot visualizes trends in the difference between commercial banks and CBK interest rates.
Preparing the Data for Impact Analysis
To prepare the data, create time-series elements for the variable and the covariate. Define the pre-intervention and post-intervention periods and create a date variable with the entire time period of interest. Finally, merge the two time-series objects to the time period using the zoo package starting with the variable of interest.
Causal Impact Analysis
Import the CausalImpact Package and perform the analysis as shown in the code chunk below
The intervention had a negative effect of -3 and a relative effect of 11% as shown in the figure above.
When further output is generated as guided above we obtain a full narrative as shown in the code and output below.
Analysis report {CausalImpact}
During the post-intervention period, the response variable had an average value
of approx. 3.64. By contrast, in the absence of an intervention, we would have
expected an average response of 6.60. The 95% interval of this counterfactual
prediction is [4.46, 8.96]. Subtracting this prediction from the observed
response yields an estimate of the causal effect the intervention had on the
response variable. This effect is -2.96 with a 95% interval of [-5.32, -0.82].
For a discussion of the significance of this effect, see below.
Summing up the individual data points during the post-intervention period
(which can only sometimes be meaningfully interpreted), the response variable
had an overall value of 138.34. By contrast, had the intervention not taken
place, we would have expected a sum of 250.84. The 95% interval of this
prediction is [169.59, 340.34].
The above results are given in terms of absolute numbers. In relative terms,
the response variable showed a decrease of -43%. The 95% interval of this
percentage is [-59%, -18%].
This means that the negative effect observed during the intervention period is
statistically significant. If the experimenter had expected a positive effect,
it is recommended to double-check whether anomalies in the control variables
may have caused an overly optimistic expectation of what should have happened
in the response variable in the absence of the intervention.
The probability of obtaining this effect by chance is very small
(Bayesian one-sided tail-area probability p = 0.006). This means the causal
effect can be considered statistically significant.The output above is detailed and self-explanatory.
You can further plot the impact output as shown below
The top graph is the most important. The dotted vertical line is the intervention. The black line shows the difference in rates while the dotted blues is the predicted outcome if the intervention was not put in place. These predictions are based on probabilities computed prior to the intervention.
Conclusion
Causal Impact analysis is useful in assessing the impact of interventions. Note that causal impact is bounded by the assumptions and unobserved confounding factors may influence the outcome.
References
1. Justin Eloriaga. Justin Eloriaga (2020, July 30). Causal Impact Analysis in Time Series using R [Youtube Video]. https://youtu.be/H64ucBjgY6w
2. Grab N Go Info. Grab N Go Info (2022, Sept 15). Time Series Causal Impact Analysis in R | Machine Learning[Youtube Video]. https://youtu.be/aUbUlquTrCg
3. Kenya Law Weekly Newsletter. http://kenyalaw.org/kl/index.php?id=9166#:~:text=Relevant%20Provisions%20of%20the%20Law&text=%E2%80%9C(1)%20A%20bank%20or,the%20Central%20Bank%20of%20Kenya.%204.%20Kenya%20Gazette%20Supplement%202016.
4. Kenya Gazette Supplement 2016. http://kenyalaw.org/kl/fileadmin/pdfdownloads/Acts/2016/No._25_of_2016.pdf
