AI Can Write R Code. But Can It Do Statistics?
Writing statistical code is becoming easier, but it is often more challenging to evaluate whether it is answering the right question. A few years ago, learning R meant learning how to write R. You had to understand R functions, packages, data frames, formulas, graphics, the associated syntax, and the occasional frustrations of statistical programming. Today, you can ask for help writing R code to accomplish a task, and get a solution in seconds.
Your assistant might even be able to explain what each line does. This creates an uncomfortable question for anyone learning statistics. If an AI assistant can write the R code for us, do we need to know statistics? I think the answer is clearly yes. But it is worth noting that while AI is likely to become extremely proficient at writing statistical code, it may not be as adept at figuring out what statistical analysis to perform, or why. This is an important distinction.
R Code is Not Statistical Reasoning
Let’s say you want to evaluate whether a patient’s age predicts their diabetes diagnosis.
An AI assistant can provide R code to perform this analysis:
model <- glm(
diabetes ~ age,
data = data,
family = binomial()
)
summary(model)This is a valid statistical model with an interpretable result. But statistical analysis rarely starts with figuring out how to implement glm(…). It often starts with questions:
- What type of variable is diabetes?
- Was it measured as a binary outcome?
3. How many observations are in the dataset?
4. Are there missing observations?
5. Is age a valid predictor?
6. Are there other variables that should be included?
7. Is the relationship between age and log odds approximately linear?
8. Are observations independent?
9. Is the sample representative of the population?
10. Are we predicting or explaining diabetes?
These are all questions that cannot be answered through knowledge of R syntax. And this is where the distinction between coding and statistical reasoning becomes important.
AI Can Produce Impressive Statistical Code
This is not meant to dismiss the value of AI in statistical analysis. On the contrary, recent studies evaluating the statistical coding ability of large language models on R tasks found that they were often capable of producing syntactically correct and interpretable code.
Similarly, an analysis of ChatGPT’s statistical coding capabilities found that it performed well on many tasks while also noting opportunities for improvement and the need for human oversight. These are all valuable contributions that make statistical analysis more accessible to non-experts.
For example, if you know you want to plot a Kaplan-Meier survival curve, you can ask for help recalling the syntax:
library(survival)
library(survminer)
fit <- survfit(
Surv(time, status) ~ treatment,
data = patients
)
ggsurvplot(fit, data = patients)You can spend less time poring over documentation; ask the assistant to explain functions, convert code from Python to R
- debug your code.
2. Generate a plot or suggest alternative approaches
This is an incredible productivity boost for everyone from students to practitioners, but there is a catch.
Interpreting Results Can Be Tricky
Let’s say you get a p-value from a statistical test that looks like this
p-value = 0.012
and the assistant explains
“Because p < 0.05, the treatment is statistically significant and therefore effective.”
The R code may be correct; the p-value may be correctly calculated, but the conclusion could be wrong. A p-value does not indicate the probability of treatment effectiveness, nor does it provide information about the magnitude of benefit.
It does not make one analysis more “credible” than another, and it certainly does not establish a causal link between two variables. This is a problem of statistical interpretation, not one of coding.
A Perfect Model Can Answer the Wrong Question
One of the most common pitfalls of statistical modeling is answering the wrong question. Let’s say we have a dataset that includes age, sex, BMI, blood pressure, cholesterol levels, smoking status, and diabetes, and we ask an AI assistant to build the best possible model to predict diabetes. It might produce a complex machine learning pipeline with excellent performance metrics, but you should ask yourself: what does “best” mean?
- Does it have the best accuracy?
2. The best sensitivity?
3. The best calibration?
4. The best generalizability?
5. The best interpretability?
These are not questions that an AI can answer. It can optimize the technical task you give it but cannot recognize if that task is meaningful
The Assumption Problem
Statistics is rife with modelling assumptions. Linear regression assumes normality, homoscedasticity, independence, and more. Logistic regression makes similar assumptions about the data. Survival analysis has assumptions built into its models:
- ANOVA has assumptions
- Mixed-effects models have assumptions
Even machine learning models have failure modes and areas of known weakness
An AI assistant can write this code for you
model <- lm(y ~ x1 + x2 + x3, data = data)
But it cannot tell you what to do next
Did you check the residuals?
Do influential observations need to be removed?
Is the relationship nonlinear?
Are there interactions?
Are the observations independent?
Is the variance stable across groups?
Get Saroj Parajuli’s stories in your inbox
Join Medium for free to get updates from this writer.
This is a crucial point: code that runs successfully is not the same as analysis that works
The Illusion of Confidence
One final issue with AI-driven statistical analysis is its tendency to generate confident, but incorrect, statements
This can be especially misleading for students and practitioners who are unfamiliar with statistical concepts. Let’s say you ask an AI assistant what statistical test to use. It might give you a convenient table:
- Situation Recommended Test
2. Two independent groups t-test
3. More than two groups: ANOVA
4. Categorical variables: Chi-square
Two continuous variables: Pearson correlation.
This is mostly correct, but it’s also massively oversimplified
What if the data is skewed?
What if there are few observations?
What if the observations are not independent?
What if the groups are matched?
What if the outcome is ordinal?
What if there are important confounders?
What if the study is observational?
What if the research question is causal?
This highlights a key reality about statistics: it is not a decision tree with three simple branches. Context is often crucial.
AI Can Make Statistics Easier to Learn
This brings me to a point that I think is more interesting. The answer is not that we should stop using AI, but that we should start using it differently. Instead of asking for help writing statistical code, you might ask for help understanding statistical concepts.
For example, instead of asking “write the R code for an independent-samples t-test” you might ask “explain what an independent-samples t-test is, what assumptions it makes, what types of data it can be applied to, and how to check these assumptions in R.”
Then you might ask, “Give me a dataset where an independent-samples t-test would not be appropriate and explain why”.
Finally, you might ask, “What are some alternatives to an independent-samples t-test?”
In this way, an AI assistant can help you learn statistics. This was a very long-winded way of saying that if you are going to use an AI assistant, learn to use it wisely.
What I Would Learn if I was Learning R Today
If you want to learn R for data analysis purposes, I would urge you to focus less on memorizing R functions and invest more effort in understanding the concepts listed below:
1. Research questions
2. Data quality
3. Design
4. Statistical concepts
5. Assumptions
6. Interpretation
7. Validation
8. Communication
The syntax is becoming less important by the day.
Can AI Do Statistics?
The honest answer is sometimes surprisingly yes, and that is an important point. It would be disingenuous to ignore the ways in which AI contributes substantially to statistical analysis. Modern language models can write sophisticated R code, explain statistical concepts, perform routine analyses, and help conduct quantitative research.
But we should not be naïve about their limitations either. Research in the field of statistics and biostatistics has repeatedly shown that statistical analysis requires human judgement and should not be fully automated. A major concern is that language models often provide incorrect statements about statistics, inappropriate analyses, or unreasonable conclusions.
Thus, I would not argue that the future belongs to humans vs AI but rather that it belongs to statistical judgement + AI assistance. An analyst who knows statistics and uses AI will be much better positioned to succeed than one who refuses to use it. But the person who knows nothing about statistics and blindly trusts in AI will always be at risk
A New Definition of an R Programmer
Perhaps the most interesting implication of all this is that the definition of an R programmer is changing. In the past, it might have been something like:
Do you know how to write R code?
Today, it might be more useful to ask
Do you know what code should be written?
Do you know whether it is appropriate?
Can you detect when the output is wrong?
Can you explain what the result means?
Can you recognize when the data cannot answer the question?
These are important considerations, especially as coding assistance becomes ubiquitous.
AI Makes Statistical Knowledge More Valuable
This may seem like an ironic statement. If statistical analysis becomes easier, why would statistical knowledge become more valuable? The reason is simple: when code becomes cheap, judgement becomes expensive
Let’s say you have two analysts:
Analyst A knows very little statistics, but can ask an AI assistant to write sophisticated R code. Analyst B knows a lot about statistics and can also ask an AI assistant for help with code. Who is more valuable? It depends on the situation. If you just need someone to write statistical code, Analyst A may be sufficient. But if you have a nuanced analysis that requires strong statistical judgement, Analyst B is likely to have an advantage. This dynamic will become more common as coding assistance becomes more widespread.
As a result, statistical knowledge will become more valuable, especially in relation to coding skills
The One Skill I Would Invest In
If I had to pick one thing to invest in for the future of statistical analysis, it would be understanding whether an analysis makes sense. R is incredibly useful, as is Python, as is AI assistance, but they are all tools. The challenging part of statistics has never been coding. It has always been understanding what to do with the data and why. That is unlikely to change, and AI is likely to help us with that process.
Final Thoughts
AI has changed the value of learning R. It has not eliminated it. You may not need to memorize all the R functions, but you should still understand what they do, when to use them, what assumptions they make, and whether the output is trustworthy. AI can write R code; the more important question is whether you know when it is wrong. This may be the most important statistical skill in the age of AI.
