The Problem with Measuring Engagement Without Understanding It
Journalism, Digital Media and Analytics | Friday
A social media post receives 10,000 likes. Another receives 2,000.
At first glance, the first post appears to have performed better. It attracted more visible interaction, reached a larger audience or generated greater interest. But what if the second post prompted more meaningful discussion, encouraged people to explore an important issue, or led more people to read the full article?
Would the first post still be more successful?
The answer depends on what success was supposed to mean in the first place.
The appeal of measurable engagement
Digital platforms have made audience interaction easier to observe. Likes, comments, shares, views, impressions and follower counts provide numerical indicators of activity. For publishers, brands and communication professionals, these measures can help evaluate how audiences interact with content.
They are useful, but they have limitations. A like may indicate approval, agreement, recognition or simply a quick reaction. A share may suggest that someone found content interesting, useful or controversial.
The number itself does not reveal the full motivation behind the interaction. This is where the difference between measuring engagement and understanding engagement becomes important.
What research tells us
A systematic literature review by Mariapina Trunfio and Simona Rossi, published in the Italian Journal of Marketing in 2021, examined how social media engagement has been conceptualised and measured.
The authors reviewed 41 academic articles and identified different approaches to measuring engagement, including quantitative metrics, normalised indexes and qualitative measures.
Their discussion highlights an important limitation: traditional measures such as likes, comments and shares do not necessarily capture the full cognitive and emotional dimensions of engagement.
In other words, visible interaction is only part of the picture. This matters because organisations can become overly focused on metrics that are easy to count rather than outcomes that are meaningful.
The difference between activity and understanding
Imagine a news organisation publishes two articles.
The first is a short, highly shareable headline about a trending topic. It receives considerable engagement. The second is an in-depth article examining a complex public issue. It receives fewer likes but generates thoughtful comments, attracts repeat readers and leads to further discussion.
Both articles may have achieved different objectives. If the organisation evaluates them using likes alone, it risks overlooking the second article’s contribution.
This is not an argument that likes or shares are meaningless. Rather, they should be interpreted according to the purpose of the content.
A campaign intended to increase awareness may prioritise reach. An educational article may be more concerned with comprehension. A customer service announcement may prioritise whether people understand the information and take the required action.
The measurement framework should reflect the objective.
Asking better analytical questions
A more useful approach begins by identifying what the organisation wants to understand.
Communication objectivePossible measuresIncrease awarenessReach, impressions and audience growthEncourage discussionRelevant comments, discussion quality and shares Support information consumption Article completion, reading time and return visits Generate enquiries Qualified enquiries and completed formsImprove understandingSurvey responses, comprehension checks and feedback
These measures are not interchangeable. For example, a high impression count does not necessarily demonstrate that an audience understood a message. Similarly, a high number of comments does not automatically mean the discussion was constructive.
Quantitative measures should therefore be interpreted alongside qualitative evidence where appropriate. Reading comments, examining audience feedback and conducting surveys can help explain patterns that numerical indicators cannot fully capture.
Why this matters for journalism
The issue becomes particularly important when considering news consumption.
Get Gomolemo Chipane’s stories in your inbox
Join Medium for free to get updates from this writer.
Journalism does not exist simply to attract attention. It also has a responsibility to provide accurate, contextualised information that helps audiences understand issues affecting society.
A headline may generate clicks without communicating the complexity of the story.
A short video may attract thousands of views without establishing whether viewers understood the information.
This creates a challenge for digital journalism: how can publishers remain visible in competitive online environments without allowing engagement metrics to become substitutes for editorial value?
My Honours research, The Influence of TikTok on News Consumption and Audience Engagement Among Pretoria North Youth and Its Implications for Modern Journalism, explored news consumption and audience engagement among young people in Pretoria North.
The study focused on participants’ reported news consumption and perceptions of credibility on TikTok. It was not designed to measure actual comprehension or prove that high engagement produces greater understanding.
That distinction matters.
An audience may frequently encounter news content and interact with it without necessarily developing a deeper understanding of the subject. Further research would be needed to investigate comprehension directly.
The analytical lesson
This discussion also connects to a broader principle in data analysis: a metric is only useful when it answers the question it was selected to measure.
Before interpreting a number, we should ask:
- What exactly is being measured?
- How was the metric defined?
- What is the relevant comparison?
- What does the number exclude?
- Does the result support the conclusion being drawn?
These questions help prevent organisations from confusing correlation with causation or treating a convenient indicator as a complete explanation.
For communication professionals, this means developing the ability to interpret performance data rather than simply report it.
For journalists, it means questioning the assumptions behind claims about online popularity and audience behaviour.
For analysts, it means understanding that choosing the right metric is part of the analytical process itself.
Beyond the numbers
Digital engagement is valuable because it provides organisations with opportunities to observe audience behaviour.
But observation is not the same as understanding.
A number can tell us how many people interacted with content. It cannot always tell us why they did so, what they understood or what they took away from the experience.
Those questions require interpretation, context and, sometimes, additional research.
Perhaps the most important question is not how many people engaged with a piece of content, but what that engagement actually tells us about the people who encountered it.
Sources and further reading
- Trunfio, M. and Rossi, S. (2021). Conceptualising and Measuring Social Media Engagement: A Systematic Literature Review. Italian Journal of Marketing, 2021, pp. 267–292.
- See also: The Psychology of Clear Communication in Digital Products.
