Traditional and Computer-Based Assessment of Executive Functions

Authors

DOI:

https://doi.org/10.19090/pp.v18i1.2601

Keywords:

Computer-based assessments, Executive functions, Psychometric properties, Cognitive flexibility, Inhibition

Abstract

This study aimed to assess the psychometric properties of two newly developed computer-based tasks (i.e., Mental Shifting/Flexibility Task and Auditory-Visual Go/No-Go Task) for measuring two key domains of executive functions (EF) - inhibition and mental shifting (flexibility) - in healthy adults. Together with these tasks, traditional paper-and-pencil tests were used for assessing construct validity (Wisconsin Card Sorting Test - WCST, Trail Making Test - TMT, Verbal Fluency Tests, and Advanced Progressive Matrices- APM). The sample consisted of 468 adult twins (70.7% female, mean age 24.06 years) or 234 twin pairs. Results revealed low to moderate correlations between the reaction times and the number of errors in the computer-based tasks and traditional tests. Specifically, the Mental Shifting/Flexibility Task showed significant correlations with the TMT and the WCST. The Auditory-Visual Go/No-Go Task was significantly related to TMT and APM, suggesting shared cognitive processes linked to inhibition, cognitive flexibility, and processing speed. The computer-based tasks demonstrated moderate to good ICC reliability, especially in reaction time measures, while error rates showed poorer reliability. It was concluded that computer-based tasks are useful for measuring executive functions. However, further validation, development of standardized norms, and optimization of these tools are needed. Future research should explore how these tools can be integrated into existing cognitive assessment batteries for more accurate measurement of executive functions across diverse populations and clinical contexts.

Metrics

No metrics found.

References

Allport, A., & Wylie, G. (1999). Task-switching: Positive and negative priming of task-set. In G. W. Humphreys, J. Duncan, & A. Treisman (Eds.), Attention, space, and action: Studies in cognitive neuroscience (pp. 273–296). Oxford University Press.

Alloway, T. P., & Carpenter, R. K. (2020). The relationship among children’s learning disabilities, working memory, and problem behaviours in a classroom setting: Three case studies. The Educational and Developmental Psychologist, 37(1), 4–10. https://doi.org/10.1017/edp.2020.1

Baron, I. S. (2004). Neuropsychological evaluation of the child. Oxford University Press.

Capovilla, A. G. S., Montiel, J. M., Macedo, E. C., & Charin, S. (2005). Computerized Stroop Test. University São Francisco.

Cianchetti, C., Corona, S., Foscoliano, M., Scalas, F., & Sannio–Fancello, G. (2005). Modified Wisconsin Card Sorting Test: Proposal of a supplementary sorting method. Archives of Clinical Neuropsychology, 20, 555–558. https://doi.org/10.1016/j.acn.2004.12.002

Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6(4), 284–290. https://doi.org/10.1037/1040-3590.6.4.284

Collerton, J., Collerton, D., Yasumichi, A., Barrass, K., Eccles, M., Jagger, C., … Kirkwood, T. (2007). A comparison of computerized and pencil-and-paper tasks in assessing cognitive function in community-dwelling older people in the Newcastle 85+ study. Journal of the American Geriatrics Society, 55, 1630–1635. https://doi.org/10.1111/j.1532-5415.2007.01379.x

Dancy, C. L., & Ritter, F. E. (2017). IGT-Open: An open-source, computerized version of the Iowa Gambling Task. Behavior Research Methods, 49(3), 972–978. https://doi.org/10.3758/s13428-016-0759-4

Diamond A. (2013). Executive functions. Annual Review of Psychology, 64, 135–168. https://doi.org/10.1146/annurev-psych-113011-143750

Feenstra, H. E. M., Vermeulen, I. E., Murre, J. M. J., & Schagen, S. B. (2017). Online cognition: Factors facilitating reliable online neuropsychological test results. Clinical Neuropsychologist, 31(1), 59–84. https://doi.org/10.1080/13854046.2016.1190405

Friedman, N. P., Miyake, A., Young, S. E., DeFries, J. C., Corley, R. P., & Hewitt, J. K. (2008). Individual differences in executive functions are almost entirely genetic in origin. Journal of Experimental Psychology: General, 137(2), 201–225. https://doi.org/10.1037/0096-3445.137.2.201

Friedman, N. P., & Miyake, A. (2004). The relations among inhibition and interference control functions: A latent-variable analysis. Journal of Experimental Psychology: General, 133(1), 101–135. https://doi.org/10.1037/0096-3445.133.1.101

Gajewski, P. D., Hanisch, E., Falkenstein, M., Thönes, S., & Wascher, E. (2018). What does the n-back task measure as we get older? Relations between working-memory measures and other cognitive functions across the lifespan. Frontiers in Psychology, 9, 2208. https://doi.org/10.3389/fpsyg.2018.02208

Garavan, H., Ross, T. J., & Stein, E. A. (1999). Right hemispheric dominance of inhibitory control: an event-related functional MRI study. Proceedings of the National Academy of Sciences of the United States of America, 96(14), 8301–8306. https://doi.org/10.1073/pnas.96.14.8301

Golden, C. J. (1978). Stroop color and word test: a manual for clinical and experimental uses. Stoelting Company.

Goodglass, H., & Kaplan, E. (1983). The assessment of aphasia and related disorders. Lea Febiger.

Heaton, R. K. (1981). Wisconsin card sorting test manual. Psychological assessment resources.

Heaton, R. K., Chelune, G. J., Tallei, J. L., Key, G. G. & Curtiss, G. (1993). Wisconsin Card Sorting Test manual: Revised and expanded. Psychological Assessment Resources.

Henrich, J., Heine, S. J., & Norenzayan, A. (2010). The weirdest people in the world? Behavioral and Brain Sciences, 33(2-3), 61-83. https://doi.org/10.1017/S0140525X0999152X

Hinkin, C. H., Castellon, S. A., Hardy, D. J., Granholm, E., & Siegle, G. (1999). Computerized and traditional Stroop Task dysfunction in HIV-1 infection. Neuropsychology, 13(2), 306–316. https://doi.org/10.1037//0894-4105.13.2.306

Horn, J. L., & Noll, J. (1997). Human cognitive capabilities: Gf-Gc theory. In D. P. Flanagan, J. L. Genshaft, & P. L. Harrison (Eds.), Contemporary intellectual assessment: Theories, tests, and issues (pp. 53–91). The Guilford Press.

IBM Corp. (2017). IBM SPSS Statistics for Windows, Version 25.0. IBM Corp.

Jacola, L. M., Willard, V. W., Ashford, J. M., Ogg, R. J., Scoggins, M. A., Jones, M. M., Wu, S., & Conklin, H. M. (2014). Clinical utility of the N-back task in functional neuroimaging studies of working memory. Journal of Clinical and Experimental Neuropsychology, 36(8), 875–886. https://doi.org/10.1080/13803395.2014.953039

Jurado, M. B., & Rosselli, M. (2007). The elusive nature of executive functions: A review of our current understanding. Neuropsychology Review, 17, 213–233. https://doi.org/10.1007/s11065-007-9040-z

Kane, R. L., & Kay, G. G. (1992). Computerized assessment in neuropsychology: a review of tests and test batteries. Neuropsychology Review, 3(1), 1–117. https://doi.org/10.1007/BF01108787

Kessels, R. P. (2019). Improving precision in neuropsychological assessment: Bridging the gap between classic paper-and-pencil tests and paradigms from cognitive neuroscience. The Clinical Neuropsychologist, 33(2), 357-368. https://doi.org/10.1080/13854046.2018.1518489

Khng, K. H., & Lee, K. (2014). The relationship between Stroop and stop-signal measures of inhibition in adolescents: influences from variations in context and measure estimation. PloS one, 9(7), e101356. https://doi.org/10.1371/journal.pone.0101356

Latendorf, A., Runde, L. M., Salminen, T., & Steinert, A. (2021). Digitization of neuropsychological diagnostics: a pilot study to compare three paper-based and digitized cognitive assessments. Aging Clinical and Experimental Research, 33, 1585-1597. https://doi.org/10.1007/s40520-020-01668-z

Lezak, M. D., Howieson, D. B., Loring, D. W., Hannay, H. J., & Fischer, J. S. (2004). Neuropsychological assessment (4th ed.). Oxford University Press.

Luciano, M., Wright, M., Smith, G. A., Geffen, G. M., Geffen, L. B., & Martin, N. G. (2001). Genetic covariance among measures of information processing speed, working memory, and IQ. Behavior Genetics, 31(6), 581–592. https://doi.org/10.1023/a:1013397428612

McDonald, A. S. (2002). The impact of individual differences on the equivalence of computer-based and paper-and-pencil educational assessments. Computers & Education, 39, 299–312. https://doi.org/10.1016/S0360-1315(02)00032-5

McElreath, R. (2020). Statistical rethinking: A Bayesian course with examples in R and Stan. Chapman & Hall. https://doi.org/10.1201/9780429029608

Mead, A. D., & Drasgow, F. (1993). Equivalence of computerized and paper-and-pencil cognitive ability tests: A meta-analysis. Psychological Bulletin, 114(3), 449–458. https://doi.org/10.1037/0033-2909.114.3.449

Miller, J. B., & Barr, W. B. (2017). The technology crisis in neuropsychology. Archives of Clinical Neuropsychology, 32(5), 541–554. https://doi.org/10.1093/arclin/acx050

Mitrović, D., Smederevac, S., Delgado-Cruzata, L., Sadiković, S., Pajić, D., Prinz, M., Budimlija, Z., Oljača, M., Kušić-Tišma, J., Vučinić, N., & Milutinović, A. (2024). Personality and COMT gene: Molecular-genetic and epigenetic associations with NEO-PI-R personality domains and facets in monozygotic twins. Frontiers in Genetics, 15, Article 1455872. https://doi.org/10.3389/fgene.2024.1455872

Miyake, A., Friedman, N. P., Emerson, M. J., Witzki, A. H., Howerter, A., & Wager, T. D. (2000). The unity and diversity of executive functions and their contributions to complex “frontal lobe” tasks: A latent variable analysis. Cognitive Psychology, 41, 49–100. https://doi.org/10.1006/cogp.1999.0734

Morris, R. G., Miotto, E. C., Feigenbaum, J. D., Bullock, P., & Polkey, C. E. (1997). The effect of goal-subgoal conflict on planning ability after frontal- and temporal-lobe lesions in humans. Neuropsychologia, 35(8), 1147-1157. https://doi.org/10.1016/S0028-3932(97)00009-2

Neubauer, A. C., Riemann, R., Mayer, R., & Angleitner, A. (1997). Intelligence and reaction times in the Hick, Sternberg and Posner paradigms. Personality and Individual Differences, 22(6), 885–894. https://doi.org/10.1016/S0191-8869(97)00003-2

Park, S. Y., & Schott, N. (2022). The trail-making-test: Comparison between paper-and-pencil and computerized versions in young and healthy older adults. Applied Neuropsychology. Adult, 29(5), 1208–1220. https://doi.org/10.1080/23279095.2020.1864374

Pinheiro, J., Bates, D., DebRoy, S., Sarkar, D., & R Core Team. (2021). Nlme: Linear and Nonlinear Mixed Effects Models (Version 3.1-153). R package. https://CRAN.R-project.org/package=nlme

R Core Team. (2020). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org/

Raven, J., Raven, J. C., & Court, J. H. (1998). Raven manual: Section 1, general overview, 1998 edition. Oxford Psychologists Press Ltd.

Reitan, R. M. (1955). The relation of the trail making test to organic brain damage. Journal of Consulting Psychology, 19, 393–394. https://doi.org/10.1037/h0044509

Riordan, P., Lombardo, T., & Schulenberg, S. E. (2013). Evaluation of a computer-based administration of the Rey complex figure test. Applied Neuropsychology, 20(3), 169–178. https://doi.org/10.1080/09084282.2012.670171

RStudio Team. (2023). RStudio: Integrated Development Environment for R (Version 2023.12.0 Build 369). RStudio, PBC. https://posit.co/

Sanders, L. M. J., Hortobágyi, T., Balasingham, M., Van der Zee, E. A., & van Heuvelen, M. J. G. (2018). Psychometric properties of a Flanker Task in a sample of patients with dementia: A pilot study. Dementia and Geriatric Cognitive Disorders Extra, 8(3), 382–392. https://doi.org/10.1159/000493750

Schatz, P., & Browndyke, J. (2002). Applications of computer-based neuropsychological assessment. The Journal of Head Trauma Rehabilitation, 17(5), 395–410. https://doi.org/10.1097/00001199-200210000-00003

Schwartz, S., Baldo, J., Graves, R. E., & Brugger, P. (2003). Pervasive influence of semantics in letter and category fluency: A multidimensional approach. Brain and Language, 87(3), 400–411. https://doi.org/10.1016/S0093-934X(03)00141-X

Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86(2), 420–428. https://doi.org/10.1037/0033-2909.86.2.420

Smederevac, S., Mitrović, D., Sadiković, S., Milovanović, I., Branovački, B., Dinić, B. M., & others. (2019). Serbian twin registry. Twin Research and Human Genetics, 22(6), 660–666. https://doi.org/10.1017/thg.2019.114

Spreen, O., & Strauss, E. (1991). A compendium of neuropsychological tests. Administration, norms and commentary. Oxford University Press.

Steinmetz, J. P., Brunner, M., Loarer, E., & Houssemand, C. (2010). Incomplete psychometric equivalence of scores obtained on the manual and the computer version of the Wisconsin Card Sorting test? Psychological Assessment, 22(1), 199–202. https://psycnet.apa.org/doi/10.1037/a0017661

Swan, G. E., & Carmelli, D. (2002). Evidence for genetic mediation of executive control: a study of aging male twins. The Journals of Gerontology, Series B: Psychological Sciences and Social Sciences, 57, 133–143. https://doi.org/10.1093/geronb/57.2.P133

Tien, A. Y., Spevack, T. V., Jones, D. W., Pearlson, G. D., Schlaepfer, T. E., & Strauss, M. E. (1996). Computerized Wisconsin Card Sorting Test: Comparison with manual administration. Kaohsiung Journal of Medical Sciences, 12, 479–485.

Troyer, A. K., Moscovitch, M., & Winocur, G. (1997). Clustering and switching as two components of verbal fluency: Evidence from younger and older healthy adults. Neuropsychology, 11(1), 138–146. https://doi.org/10.1037/0894-4105.11.1.138

Tyburski, E., Kerestey, M., Kerestey, P., Radoń, S., & Mueller, S. T. (2021). Assessment of motor planning and inhibition performance in non-clinical sample—reliability and factor structure of the Tower of London and Go/No Go computerized tasks. Brain Sciences, 11(11), 1420. https://doi.org/10.3390/brainsci11111420

Vernon, P. A., & Jensen, A. R. (1984). Individual and group differences in intelligence and speed of information processing. Personality and Individual Differences, 5(4), 411–423. https://doi.org/10.1016/0191-8869(84)90006-0

Welsh, M. C., Cicerello, A., Cuneo, K., & Brennan, M. (1995). Error and temporal patterns in Tower of Hanoi performance: Cognitive mechanisms and individual differences. Journal of General Psychology, 122(1), 69-81. https://doi.org/10.1080/00221309.1995.9914919

Vermeent, S., Dotsch, R., Schmand, B., Klaming, L., Miller, J. B., & van Elswijk, G. (2020). Evidence of validity for a newly developed digital cognitive test battery. Frontiers in Psychology, 11, 770. https://doi.org/10.3389/fpsyg.2020.00770

Wager, T. D., Sylvester, C. Y., Lacey, S. C., Nee, D. E., Franklin, M., & Jonides, J. (2005). Common and unique components of response inhibition revealed by fMRI. NeuroImage, 27(2), 323–340. https://doi.org/10.1016/j.neuroimage.2005.01.054

Wagner, G. P., & Trentini, C. M. (2009). Assessing executive functions in older adults: a comparison between the manual and the computer-based versions of the Wisconsin Card Sorting Test. Psychology & Neuroscience, 2, 195-198. https://doi.org/10.3922/j.psns.2009.2.011

Welsh, M. C., Pennington, B. F., & Groisser, D. B. (1991). A normative developmental study of executive function: A window on prefrontal function in children. Developmental Neuropsychology, 7(2), 131-149. https://doi.org/10.1080/87565649109540577

Downloads

Published

17.04.2025

How to Cite

Nikolašević, Željka, Dinić, B. M., Oljača, M., Milovanović, I. ., Kodžopeljić, J., & Bugarski Ignjatović, V. (2025). Traditional and Computer-Based Assessment of Executive Functions. Primenjena Psihologija, 18(1). https://doi.org/10.19090/pp.v18i1.2601

Issue

Section

Regular issues