Connect
MJA

A web-based normative data tool for assessing cognitive performance in healthy older Australians

Simon J McBride, Cassandra E I Szoeke, Norm M Good, David Ames, Ralph N Martins, Colin L Masters, Paul T Maruff, Christopher C Rowe, Greg Savage and Kathryn A Ellis, and the AIBL Research Group
Med J Aust 2011; 194 (4): S12. || doi: 10.5694/j.1326-5377.2011.tb02936.x
Published online: 21 February 2011

In clinical settings, cognition is assessed using standardised tasks, and performance of these tasks typically declines with age.1 In older people, disorders of the central nervous system (CNS) often present with impairment in cognitive function. In degenerative diseases, such as Alzheimer’s disease (AD), cognitive impairment is a requirement for diagnosis. However, subtle cognitive impairment can provide the earliest preclinical indicator of the disease process. Therefore, one important challenge for clinicians is to separate expected age-related changes in performance from performance impairments that suggest some CNS disease process. This task usually falls to clinical neuropsychologists, who use well validated tests with sound psychometric properties and normative standards.

The availability of demographically appropriate, normative performance ranges is crucial to this endeavour. However, for many neuropsychological tests, such normative ranges are based on small samples with demographic characteristics quite different from those in the Australian context. Comparing performance on neuropsychological tests with normative data ranges based on small or non-representative samples can diminish the accuracy of decisions made about an individual’s cognitive functioning.

The recruitment of large samples for establishing normative data ranges can be expensive and time consuming. Therefore, it is important that, when such resources have been committed, the results are made available to the broader community of scientists and clinicians managing CNS diseases in older Australians. Advances in information technology now make it possible to disseminate this information widely to the community and even provide tools promoting ease of access to such databases.

The Australian Imaging, Biomarkers and Lifestyle (AIBL) Flagship Study of Ageing2,3 collected neuropsychological data on more than 700 healthy Australians. This AIBL dataset presents a unique opportunity to provide reliable and representative normative data ranges for clinicians and researchers working with older Australians.

A collaborative project between the AIBL clinical research team and the Australian e-Health Research Centre led to the development of a web-based tool to access databases of normal subjects based on the AIBL sample. This tool allows immediate comparison of patients’ results with the normative dataset. Here, we introduce the AIBL normative data tool and consider its usefulness in both research and clinical settings.

Development of the normative data tool
Selection of subjects

Of the AIBL study’s baseline cohort of 1112 participants, 211 had AD, 133 had mild cognitive impairment, and the remaining 768 were healthy control subjects.2

To select a healthy normal subset, the following procedures were performed on the data from the 768 control subjects. All participants were classified using a variation of a random forests classifier.4 Individuals in the healthy control group “most” misclassified as having either mild cognitive impairment or AD were removed from the dataset, and the model was run again. This was repeated until no healthy control subjects were misclassified. In addition, those individuals with scores over 0.5 on the clinical dementia rating scale (a five-point scale used to characterise six domains of cognitive and functional performance) were also excluded from the analysis.

This led to the exclusion of 41 individuals, leaving a sample of 727 healthy subjects. Box 1 provides a summary of the characteristics of the normative dataset. For constructing the normative tables, age was divided into five intervals, and level of education into fewer than 12 years and 12 or more years of education.

Neuropsychological measures

The web tool will use the normative dataset to characterise a person’s performance in the following cognitive domains: focused attention and working memory (digit span); speed of information processing and sustained attention (symbol transcription); language production (picture naming, word generation); visuoconstruction (figure copying, clock drawing); executive functioning (word generation, response inhibition); verbal memory (prose recall; word-list recall/recognition); and non-verbal memory (drawing recall/recognition).

Data analysis

To select which demographic variables were significant predictors of the neuropsychological measures, the following procedures were employed.

  • Linear regression models were fitted to normally distributed or easily transformed non-normal data.

  • To account for any demographic confounding, all cognitive variables were first regressed against age (continuous), sex (male, female), education level (< 12 years, ≥ 12 years) and all first-order interactions.

  • Stepwise regression was then employed, eliminating the least significant variables (at the two-tailed, P = 0.05, cut-off value) until a final model was selected.

  • A number of regression analysis assumptions were then checked.

  • Homoscedasticity was tested by grouping patients into quartiles of the predicted scores and applying the Fligner test5 to the residuals.

  • Normality of the residuals was assessed by inspection of normal probability plots.

  • Variance inflation factors were calculated to examine possible multicollinearity between variables, which should not exceed 10.

  • Cook’s distance measures6 were plotted to identify possible outliers.

Some variables were prone to ceiling effects in performance (eg, a normal score on the Short Form Boston Naming Test is close to the maximum value of 30). In such cases, a Tobit regression model7 was fitted to the data. For variables in which no model seemed to be appropriate, quantiles were produced.

Online calculator

To assist with the translation of this research into clinical practice, the Cognitive Performance Calculator (CPC) was developed, which simplifies the clinical use of the normative data. The CPC generates a summary report comparing the neuropsychological measures supplied by the user with the statistical models fitted to the normative dataset. To generate the report, all of the mandatory demographic variables and at least one neuropsychological measure must be entered into the CPC’s data entry page (Box 2).

Box 3 shows an example report generated for a 78-year-old woman with fewer than 12 years’ education, who scored 8 on the Rey Complex Figure Test (30-minute delayed recall).8 The result of this test equates to a z score of 0.80; that is, slightly below the value predicted by the model.

The report summarises the relationship between the normative dataset and the input variables, and can be downloaded in text or PDF formats.

Discussion

We have described the development of a novel, Australian-specific web tool to compare a patient’s cognition with a representative Australian dataset. This is the first such normative guide, and it is augmented by a user-friendly web interface.

The data produced are in formats commonly used by specialist neuropsychologists and other psychologists, who are trained in interpreting patterns of performance on psychometric tests of cognition. The ability to quickly and accurately compare patients’ scores with the expected scores for their age, education and demographic characteristics will allow improved classification, standardised results and greater efficiency. The web tool also provides a pictorial representation, allowing a quick examination of a number of test scores compared with the normative population. This can give an overall view of a patient’s performance.

Increased efficiency is a key element of this tool. Clinicians can enter a few values into the CPC and will be provided with a report giving both a pictorial snapshot of performance (compared with the average), as well as values that can be imported into commonly used computer programs. This is particularly useful if the data form part of a clinical trial or other research study.

After release to qualified users in the near future, this tool, and the normative data on which it is based, will continue to be refined as new data become available. This will allow for greater specificity in distinguishing cognitive decline associated with mild cognitive impairment from normal age-related cognitive change in the “worried well”.

Conclusion

Analysis of data from the AIBL Flagship Study of Ageing has provided normative values for older Australians for commonly used cognitive tasks. These data can be used in preference to currently available norms based on smaller samples from international cohorts. A web-based application automates the calculation of normed data. Time is a key resource in clinical and research settings, and this tool will improve both the accuracy and efficiency of interpreting neurocognitive results.

1 Descriptive characteristics of the normative sample (= 727)

Age group

No. of subjects

Mean (SD) age (years)

< 12 years’ education

≥ 12 years’ education

Male: female ratio


0–64

190

62 (1.3)

80

110

65 : 125

65–69

206

67 (1.4)

101

105

84 : 122

70–74

151

72 (1.4)

71

80

71 : 80

75–79

102

77 (1.4)

52

50

49 : 53

80 +

78

83 (3.5)

32

46

34 : 44

2 Data entry page of the Cognitive Performance Calculator, with optional fields hidden


The data entry page has been designed to make data entry as fast and easy as possible. The cursor is automatically placed in the “Age” field and the “Tab” (forward) and “Shift-Tab” (backward) keys are used for field navigation. On the last field, “Tab” moves the cursor to the “Calculate” button. The “Clear” button clears all values in all fields. The “Calculate” button is not enabled until the minimum amount of data for a valid calculation has been entered. All demographic variables and at least one score for a neuropsychological measure are required.

3 Cognitive Performance Calculator — example report


Report for a 78-year-old woman with fewer than 12 years’ education, who scored 8 on the Rey Complex Figure Test (30-minute delayed recall).

  • Simon J McBride1
  • Cassandra E I Szoeke2
  • Norm M Good1
  • David Ames4
  • Ralph N Martins5,6
  • Colin L Masters7
  • Paul T Maruff8
  • Christopher C Rowe9
  • Greg Savage10
  • Kathryn A Ellis3
  • and the AIBL Research Group

  • 1 Australian e-Health Research Centre, CSIRO ICT Centre, Brisbane, QLD.
  • 2 CSIRO Preventative Health Flagship, Melbourne, VIC.
  • 3 Academic Unit for Psychiatry of Old Age, Department of Psychiatry, University of Melbourne, and St Vincent’s Aged Psychiatry Service, St George’s Hospital Campus, Melbourne, VIC.
  • 4 National Ageing Research Institute, Melbourne, VIC.
  • 5 Centre of Excellence for Alzheimer’s Disease Research and Care, School of Exercise, Biomedical and Health Sciences, Edith Cowan University, Perth, WA.
  • 6 Sir James McCusker Alzheimer’s Disease Research Unit, Hollywood Private Hospital, Perth, WA.
  • 7 Mental Health Research Institute, and Centre for Neuroscience, University of Melbourne, Melbourne, VIC.
  • 8 CogState Pty Ltd, Melbourne, VIC.
  • 9 Department of Nuclear Medicine and Centre for Positron Emission Tomography, Austin Hospital, Melbourne, VIC.
  • 10 Macquarie Centre for Cognitive Science, Macquarie University, Sydney, NSW.

Correspondence: kellis@unimelb.edu.au

Acknowledgements: 

For a list of members of the AIBL Research Group, see . Cassandra Szoeke’s research has been partly supported by Alzheimer’s Australia, the Scobie and Claire Mackinnon Foundation, the Collier Charitable Fund, and the National Health and Medical Research Council.

Competing interests:

Cassandra Szoeke has received honoraria from Pfizer and Lundbeck (most recently in 2009) and Sanofi-Aventis and Mayne Pharma (most recently in 2006). Paul Maruff holds CogState Pty Ltd stock options.

  • 1. Deary IJ, Corley J, Gow AJ, et al. Age-associated cognitive decline. Br Med Bull 2009; 92: 135-152.
  • 2. Ellis KA, Bush AI, Darby D, et al; AIBL research group. The Australian Imaging, Biomarkers and Lifestyle (AIBL) study of aging: methodology and baseline characteristics of 1112 individuals recruited for a longitudinal study of Alzheimer’s disease. Int Psychogeriatr 2009; 21: 672-687.
  • 3. The Australian Imaging, Biomarker and Lifestyle (AIBL) Flagship Study of Ageing [website]. http://www.aibl.csiro.au/ (accessed Aug 2010).
  • 4. Breiman L. Random forests. Machine Learning 2001; 45: 5-32.
  • 5. Conover WJ, Johnson ME, Johnson MM. A comparative study of tests for homogeneity of variances, with applications to the outer continental shelf bidding data. Technometrics 1981; 23: 351-361.
  • 6. Cook DR. Detection of influential observation in linear regression. Technometrics 1977; 19: 15-18.
  • 7. Tobin J. Estimation of relationships for limited dependent variables. Econometrica 1958; 26: 24-36.
  • 8. Meyers JE, Meyers KR. Rey complex figure test and recognition trial professional manual. Odessa, Fl: Psychological Assessment Resource, Inc, 1995.

Author

remove_circle_outline Delete Author
add_circle_outline Add Author

Comment
Do you have any competing interests to declare? *

I/we agree to assign copyright to the Medical Journal of Australia and agree to the Conditions of publication *
I/we agree to the Terms of use of the Medical Journal of Australia *
Email me when people comment on this article

Online responses are no longer available. Please refer to our instructions for authors page for more information.