In clinical settings, cognition is assessed using standardised tasks, and performance of these tasks typically declines with age.1 In older people, disorders of the central nervous system (CNS) often present with impairment in cognitive function. In degenerative diseases, such as Alzheimer’s disease (AD), cognitive impairment is a requirement for diagnosis. However, subtle cognitive impairment can provide the earliest preclinical indicator of the disease process. Therefore, one important challenge for clinicians is to separate expected age-related changes in performance from performance impairments that suggest some CNS disease process. This task usually falls to clinical neuropsychologists, who use well validated tests with sound psychometric properties and normative standards.
The availability of demographically appropriate, normative performance ranges is crucial to this endeavour. However, for many neuropsychological tests, such normative ranges are based on small samples with demographic characteristics quite different from those in the Australian context. Comparing performance on neuropsychological tests with normative data ranges based on small or non-representative samples can diminish the accuracy of decisions made about an individual’s cognitive functioning.
The recruitment of large samples for establishing normative data ranges can be expensive and time consuming. Therefore, it is important that, when such resources have been committed, the results are made available to the broader community of scientists and clinicians managing CNS diseases in older Australians. Advances in information technology now make it possible to disseminate this information widely to the community and even provide tools promoting ease of access to such databases.
The Australian Imaging, Biomarkers and Lifestyle (AIBL) Flagship Study of Ageing2,3 collected neuropsychological data on more than 700 healthy Australians. This AIBL dataset presents a unique opportunity to provide reliable and representative normative data ranges for clinicians and researchers working with older Australians.
A collaborative project between the AIBL clinical research team and the Australian e-Health Research Centre led to the development of a web-based tool to access databases of normal subjects based on the AIBL sample. This tool allows immediate comparison of patients’ results with the normative dataset. Here, we introduce the AIBL normative data tool and consider its usefulness in both research and clinical settings.
Of the AIBL study’s baseline cohort of 1112 participants, 211 had AD, 133 had mild cognitive impairment, and the remaining 768 were healthy control subjects.2
To select a healthy normal subset, the following procedures were performed on the data from the 768 control subjects. All participants were classified using a variation of a random forests classifier.4 Individuals in the healthy control group “most” misclassified as having either mild cognitive impairment or AD were removed from the dataset, and the model was run again. This was repeated until no healthy control subjects were misclassified. In addition, those individuals with scores over 0.5 on the clinical dementia rating scale (a five-point scale used to characterise six domains of cognitive and functional performance) were also excluded from the analysis.
This led to the exclusion of 41 individuals, leaving a sample of 727 healthy subjects. Box 1 provides a summary of the characteristics of the normative dataset. For constructing the normative tables, age was divided into five intervals, and level of education into fewer than 12 years and 12 or more years of education.
The web tool will use the normative dataset to characterise a person’s performance in the following cognitive domains: focused attention and working memory (digit span); speed of information processing and sustained attention (symbol transcription); language production (picture naming, word generation); visuoconstruction (figure copying, clock drawing); executive functioning (word generation, response inhibition); verbal memory (prose recall; word-list recall/recognition); and non-verbal memory (drawing recall/recognition).
To select which demographic variables were significant predictors of the neuropsychological measures, the following procedures were employed.
Linear regression models were fitted to normally distributed or easily transformed non-normal data.
To account for any demographic confounding, all cognitive variables were first regressed against age (continuous), sex (male, female), education level (< 12 years, ≥ 12 years) and all first-order interactions.
Stepwise regression was then employed, eliminating the least significant variables (at the two-tailed, P = 0.05, cut-off value) until a final model was selected.
A number of regression analysis assumptions were then checked.
Homoscedasticity was tested by grouping patients into quartiles of the predicted scores and applying the Fligner test5 to the residuals.
Normality of the residuals was assessed by inspection of normal probability plots.
Variance inflation factors were calculated to examine possible multicollinearity between variables, which should not exceed 10.
Cook’s distance measures6 were plotted to identify possible outliers.
Some variables were prone to ceiling effects in performance (eg, a normal score on the Short Form Boston Naming Test is close to the maximum value of 30). In such cases, a Tobit regression model7 was fitted to the data. For variables in which no model seemed to be appropriate, quantiles were produced.
To assist with the translation of this research into clinical practice, the Cognitive Performance Calculator (CPC) was developed, which simplifies the clinical use of the normative data. The CPC generates a summary report comparing the neuropsychological measures supplied by the user with the statistical models fitted to the normative dataset. To generate the report, all of the mandatory demographic variables and at least one neuropsychological measure must be entered into the CPC’s data entry page (Box 2).
Box 3 shows an example report generated for a 78-year-old woman with fewer than 12 years’ education, who scored 8 on the Rey Complex Figure Test (30-minute delayed recall).8 The result of this test equates to a z score of − 0.80; that is, slightly below the value predicted by the model.
The report summarises the relationship between the normative dataset and the input variables, and can be downloaded in text or PDF formats.
We have described the development of a novel, Australian-specific web tool to compare a patient’s cognition with a representative Australian dataset. This is the first such normative guide, and it is augmented by a user-friendly web interface.
The data produced are in formats commonly used by specialist neuropsychologists and other psychologists, who are trained in interpreting patterns of performance on psychometric tests of cognition. The ability to quickly and accurately compare patients’ scores with the expected scores for their age, education and demographic characteristics will allow improved classification, standardised results and greater efficiency. The web tool also provides a pictorial representation, allowing a quick examination of a number of test scores compared with the normative population. This can give an overall view of a patient’s performance.
Increased efficiency is a key element of this tool. Clinicians can enter a few values into the CPC and will be provided with a report giving both a pictorial snapshot of performance (compared with the average), as well as values that can be imported into commonly used computer programs. This is particularly useful if the data form part of a clinical trial or other research study.
After release to qualified users in the near future, this tool, and the normative data on which it is based, will continue to be refined as new data become available. This will allow for greater specificity in distinguishing cognitive decline associated with mild cognitive impairment from normal age-related cognitive change in the “worried well”.
Analysis of data from the AIBL Flagship Study of Ageing has provided normative values for older Australians for commonly used cognitive tasks. These data can be used in preference to currently available norms based on smaller samples from international cohorts. A web-based application automates the calculation of normed data. Time is a key resource in clinical and research settings, and this tool will improve both the accuracy and efficiency of interpreting neurocognitive results.
2 Data entry page of the Cognitive Performance Calculator, with optional fields hidden
The data entry page has been designed to make data entry as fast and easy as possible. The cursor is automatically placed in the “Age” field and the “Tab” (forward) and “Shift-Tab” (backward) keys are used for field navigation. On the last field, “Tab” moves the cursor to the “Calculate” button. The “Clear” button clears all values in all fields. The “Calculate” button is not enabled until the minimum amount of data for a valid calculation has been entered. All demographic variables and at least one score for a neuropsychological measure are required.
- Simon J McBride1
- Cassandra E I Szoeke2
- Norm M Good1
- David Ames4
- Ralph N Martins5,6
- Colin L Masters7
- Paul T Maruff8
- Christopher C Rowe9
- Greg Savage10
- Kathryn A Ellis3
- and the AIBL Research Group
- 1 Australian e-Health Research Centre, CSIRO ICT Centre, Brisbane, QLD.
- 2 CSIRO Preventative Health Flagship, Melbourne, VIC.
- 3 Academic Unit for Psychiatry of Old Age, Department of Psychiatry, University of Melbourne, and St Vincent’s Aged Psychiatry Service, St George’s Hospital Campus, Melbourne, VIC.
- 4 National Ageing Research Institute, Melbourne, VIC.
- 5 Centre of Excellence for Alzheimer’s Disease Research and Care, School of Exercise, Biomedical and Health Sciences, Edith Cowan University, Perth, WA.
- 6 Sir James McCusker Alzheimer’s Disease Research Unit, Hollywood Private Hospital, Perth, WA.
- 7 Mental Health Research Institute, and Centre for Neuroscience, University of Melbourne, Melbourne, VIC.
- 8 CogState Pty Ltd, Melbourne, VIC.
- 9 Department of Nuclear Medicine and Centre for Positron Emission Tomography, Austin Hospital, Melbourne, VIC.
- 10 Macquarie Centre for Cognitive Science, Macquarie University, Sydney, NSW.
For a list of members of the AIBL Research Group, see
Cassandra Szoeke has received honoraria from Pfizer and Lundbeck (most recently in 2009) and Sanofi-Aventis and Mayne Pharma (most recently in 2006). Paul Maruff holds CogState Pty Ltd stock options.
- 1. Deary IJ, Corley J, Gow AJ, et al. Age-associated cognitive decline. Br Med Bull 2009; 92: 135-152.
- 2. Ellis KA, Bush AI, Darby D, et al; AIBL research group. The Australian Imaging, Biomarkers and Lifestyle (AIBL) study of aging: methodology and baseline characteristics of 1112 individuals recruited for a longitudinal study of Alzheimer’s disease. Int Psychogeriatr 2009; 21: 672-687.
- 3. The Australian Imaging, Biomarker and Lifestyle (AIBL) Flagship Study of Ageing [website]. http://www.aibl.csiro.au/ (accessed Aug 2010).
- 4. Breiman L. Random forests. Machine Learning 2001; 45: 5-32.
- 5. Conover WJ, Johnson ME, Johnson MM. A comparative study of tests for homogeneity of variances, with applications to the outer continental shelf bidding data. Technometrics 1981; 23: 351-361.
- 6. Cook DR. Detection of influential observation in linear regression. Technometrics 1977; 19: 15-18.
- 7. Tobin J. Estimation of relationships for limited dependent variables. Econometrica 1958; 26: 24-36.
- 8. Meyers JE, Meyers KR. Rey complex figure test and recognition trial professional manual. Odessa, Fl: Psychological Assessment Resource, Inc, 1995.
Abstract
A decline in cognition greater than expected with ageing and accompanied by subjective cognitive concerns or functional changes may be indicative of a dementing disorder.
The capacity to correctly identify cognitive decline relies on comparisons with normative data from a suitably matched healthy reference group with relatively homogeneous demographic features.
Formal assessment of cognition is usually performed by specialist neuropsychologists trained in administration and interpretation of psychometric tests. With a scarcity of normative data from large cohorts of older adults, Australian neuropsychologists commonly use representative data from small international studies.
Data from 727 healthy older Australians participating in the Australian Imaging, Biomarkers and Lifestyle (AIBL) Flagship Study of Ageing have been used to create a normative dataset.
A web-based calculator was developed to simplify the time-consuming process of comparing cognitive performance scores with these representative data.