Mental Workload vs Cognitive Load vs everything else in HCI
One of my major… annoyances (?) in HCI is the conflation of various mental workload, mental effort, cognitive load, cognitive workload terms that fly around, only some of which are even established terms, and even fewer that are empirically validated. And let’s not even get started on what is ‘stress’ [1][2], nor what people mean when they say focus, attention, and vigilance. A super scientific search on google scholar for some of these terms shows some ‘popularity’ of the terms (see below). But not all of these are distinct — reading papers (like [1]) about mental load, shows they refer consistently to mental workload. Terms like mental effort, in popularity, seem to be used more colloquially and thus appear more often in literature. More research would be needed to fully establish which terms are based on well established theories, and which ones are being used purposefully, or descriptively. Some attempts at this are starting to be published [3 — an awesome read for this topic].
What we typically say vs what we typically measure
Let’s begin with what is often said in papers, which quite often is ‘cognitive load’ — we want to reduce the cognitive load that people are experiencing while performing tasks or interacting with computers, if not while e.g. driving in AutoUI. When people write about this, they typically then cite Cognitive Load Theory [4] (from educational psychology). In the same papers, we then frequently see authors deciding how to measure cognitive load, and they stumble on what has seemingly become the most established method in HCI: NASA TLX [5]. NASA Task Load indeX is based upon an entirely different theoretical basis (Mental Workload.. or actually Workload) from a different field (Engineering, and Cognitive Psychology). Of note at this point, NASA TLX includes 6 dimensions including physical workload, so in many cases it is not evident that it entirely applies, and researchers thus examine which dimensions change or sample dimensions for uses in their analysis. Anyway, lets talk about both. And… lets note, in many ways they are talking about the same things in different words, which doesnt make things easier for this conversation. Interestingly, if you want the wikipedia starting guides for these topics, Googling mental workload takes you to a cognitive load page of wikipedia, where as mental workload is actually discussed on a page about ‘Workload’ on wikipedia.
Cognitive Load Theory
Cognitive Load Theory [4] is from educational psychology, as noted above, and comes from the perspective of articulating how learning resources are designed, or experiences by learners. The argument, developing in the 1980s, was that instructional material can be designed in ways that reduce the cognitive load experienced by learners, thus making it easier for them to learn. Perhaps usefully for HCI, CLT argues that there are three types of cognitive load: 1) Extrinsic, which refers to the load imposed by the instructional material (or could be imposed by the complexity of the user interface in HCI); 2) Intrinsic, which is imposed by the topic being learned or task being completed; and 3) Germane, which is the spare capacity that is needed and used ‘learn’, or create schema in memory. In HCI, we can consider that we are reducing user interface complexity (extrinsic) or designing the tasks better (intrinsic), perhaps so as to have more capacity for things (germane). I find this division of load very interesting, and there are many papers about these developments, and how they relate to design. Their work also considers how the maximum capacity varies on resources available, affected by readiness, caffeine, etc — again a topic that comes up in mental workload.
Mental Workload
Get Cognitive Personal Informatics’s stories in your inbox
Join Medium for free to get updates from this writer.
Mental Workload, or research into workload more broadly [6 — different Wilson, but same Sharples in my papers with Sharples], comes from the engineering/psychology field. These are built on the foundational research in the 1970s-80s from people like Christopher Wickens and Alan Baddeley: of limited resources, multiple resource theory [7], multi-component models of working memory [8] etc, which are all empirically validated understandings of how the brain deals with incoming sensory data and how much it can manage. These theories have been used more in human factors, studying e.g. safety critical work in order to understand when and why people make mistakes. Most notably, the precipice of performance is where people reach their limited capacity (mentioned also in Cognitive Load Thoery), but then suddenly the performance drops (or error rate increases) when they become overloaded. Models also talk about underload relating to what happens when attention becomes divided through lack of demand in the primary task.
NASA Task Load indeX is based upon this engineering/psychology grounding, measuring dimensions of workload. One common consideration for using NASA TLX is that it takes some time for people to fill in, and its usually done reflectively/retrospectively about a task at the next available moment in the study design that doesn't interrupt the conditions. Choosing this time is a regular point of tension for study validity and validity of NASA TLX measurements. Other measures that researchers use include Subjective Workload Assessment Technique (SWAT) [9] and Instantaneous Self-Assessment (ISA) of workload [10], which are simpler to use during a study to get momentary workload. ISA, where a user provides a single rating out of 5 for example, can be done orally during a task, although research often considers if this is creating additional workload (if the study is about measuring workload). This has to be done carefully, ideally considering multiple resource theory and duel processing for which mode is best in relation to the task.
Anyway, these and other measures used in HCI are typically grounded in Mental Workload, and to my knowledge, not based upon the Cognitive Load Theory that people cite for creating their hypotheses.
What does this mean for HCI?
Importantly, I think it is time for us to establish what the HCI community is using as its theoretical basis, and its measurement, and to be aware of these differences and similarities. One awesome paper that was recently published by Kosch et al [3] performs a large systematic review of how all these things are used in HCI, which is worth reading for more specifics. I’m glad Kosch’s paper begins to address this question about what are we using, and asks whether we are being consistent about our studies of the cognitive/metal aspects of interaction. Critically, this will be increasingly important as psychophysiological measurements of workload mature, and as consumer neurotechnology arrives en masse to every day users in the wearable market. As we establish what differentiates Cognitive Personal Informatics [13], it seems clear that these kinds of theories (along with disambiguations between mental workload and other things like stress [1][2], fatigue [11], attention [12], focus etc) will play a crucial role in understanding people and how they will use cognitively aware technology in the future.
Im glad I got that out of my brain — tell me what you think! And what I’ve forgotten.
Doing work in this area?
We are running a workshop at CHI2024 in Hawai’i on these kinds of topics, which you should submit to! The deadline is late feb 2024, and you can submit a) researcher bios (to show you are working in this area), b) position papers (to argue your point about something), and c) research summaries (to describe something you’ve done lately in a non-archival paper). See you there!
[1] Gaillard, A. W. (1993). Comparing the concepts of mental load and stress. Ergonomics, 36(9), 991–1005.
[2] Alsuraykh, N. H., Wilson, M. L., Tennent, P., & Sharples, S. (2019, May). How stress and mental workload are connected. In Proceedings of the 13th EAI International Conference on Pervasive Computing Technologies for Healthcare (pp. 371–376).
[3] Kosch, T., Karolus, J., Zagermann, J., Reiterer, H., Schmidt, A., & Woźniak, P. W. (2023). A survey on measuring cognitive workload in human-computer interaction. ACM Computing Surveys.
[4] Plass, J. L., Moreno, R., & Brünken, R. (Eds.). (2010). Cognitive load theory.
[5] Hart, S. G. (2006, October). NASA-task load index (NASA-TLX); 20 years later. In Proceedings of the human factors and ergonomics society annual meeting (Vol. 50, №9, pp. 904–908). Sage CA: Los Angeles, CA: Sage publications.
[6] Wilson, J. R., & Sharples, S. (Eds.). (2015). Evaluation of human work. CRC press.
[7] Wickens, C. D. (2008). Multiple resources and mental workload. Human factors, 50(3), 449–455.
[8] Baddeley, A. (1992). Working memory. Science, 255(5044), 556–559.
[9] Rubio, S., Díaz, E., Martín, J., & Puente, J. M. (2004). Evaluation of subjective mental workload: A comparison of SWAT, NASA‐TLX, and workload profile methods. Applied psychology, 53(1), 61–86.
[10] Tattersall, A. J., & Foord, P. S. (1996). An experimental evaluation of instantaneous self-assessment as a measure of workload. Ergonomics, 39(5), 740–748.
[11] Argyle, E. M., Marinescu, A., Wilson, M. L., Lawson, G., & Sharples, S. (2021). Physiological indicators of task demand, fatigue, and cognition in future digital manufacturing environments. International Journal of Human-Computer Studies, 145, 102522.
[12] Bruya, B. (2010). Effortless attention. A Bradford Book, the MIT Press, Cambridge, Massachusetts Institute of Technology, 75–92.
[13] Wilson, M. L., Midha, S., Maior, H. A., Cox, A. L., Chuang, L. L., & Urquhart, L. D. (2022, April). SIG: Moving from Brain-Computer Interfaces to Personal Cognitive Informatics. In CHI Conference on Human Factors in Computing Systems Extended Abstracts (pp. 1–4).
