Self-observation scales explained

Two scales live under the same acronym, and confusing them explains a great deal of the misunderstanding. The first, published by Kessler and colleagues in 2005 in Psychological Medicine, has eighteen items, six of which were selected by logistic regression to form a short version. The second, published by Ustun and colleagues in 2017 in JAMA Psychiatry, is a six-item version recalibrated on different criteria.

Their figures do not match, and that is instructive. On 154 respondents from a United States general-population survey, the 2005 short version picks up 68.7% of the people concerned and correctly leaves 99.5% of the others below the threshold. The 2017 version reaches 91.4% sensitivity and 96.0% specificity in the general population. Same field, same number of items, two calibrations: the figure belongs to the study, not to the scale.

What these scales measure is more modest than what is often claimed for them: the self-reported frequency, over recent months, of situations of inattention and inner restlessness. They measure neither how far back the difficulties go, nor their impact, nor what else might explain them. Those three elements sit at the heart of an assessment carried out by a professional.

Two instruments dominate the adult literature. The first is a ten-item short questionnaire published by Allison, Auyeung and Baron-Cohen in 2012 from a thousand cases and three thousand controls; it derives from a longer questionnaire produced by the same research centre. The second is the revised scale by Ritvo and colleagues, published in 2011, which runs to eighty items and takes considerably longer.

The Ritvo scale reports striking figures in its original publication: 97% sensitivity and 100% specificity across 779 participants, 201 of them concerned and 578 controls. Those results come from the team that built the scale, on a sample where the two groups were sharply contrasted. It is the most favourable configuration there is, and it looks nothing like someone answering alone at home.

The most useful demonstration comes from an independent study. Ashwood and colleagues applied the questionnaire in 2016 to 476 adults referred to a specialist service, of whom 346, that is 73%, had the condition recognised at the end of the assessment. At the usual threshold, the short version picks up 77% of the people concerned but correctly classifies only 29% of the others. The authors conclude that the questionnaire did not predict the outcome in their sample.

The reference instrument was presented by Christina Maslach and Susan Jackson in 1981 in the Journal of Organizational Behavior. It was built with human services professionals, and the authors describe three distinct dimensions: emotional exhaustion, depersonalisation, and personal accomplishment. Three dimensions, therefore three readings, not one single score.

That structure is what online adaptations most often lose. A site that displays a single exhaustion percentage manufactures a figure the original instrument does not produce. Someone very tired who still feels their work is worthwhile is not in the same position as someone whose three dimensions tip together, and a single score erases precisely that difference.

A second limit is stated less often: work exhaustion is not a disorder in the sense of the medical classifications. What the instrument measures is a work-related state, described along three axes. It does not separate what stems from an unworkable organisation of work from what stems from something else, and no questionnaire makes that distinction.

Because a published scale remains a protected work, and reuse terms differ from one rights holder to another. This is the least visible point for a reader, and the one that most clearly separates a site that has checked its rights from a site that has not.

The consequence is simple, and we own it. Brain Quiz questionnaires reproduce no item from these scales. They were written for this product from the literature, their weights and result bands were set by the editorial team, and none of them comes from calibration on a population. Every methodology page says so, and it is also why none of our scores is presented as a validated threshold.

These three words appear in every publication cited here, and they do not say the same thing. The first two describe the scale. The third describes what concerns you: what to make of your own score. It is the only one of the three that changes with the audience the scale is given to.

One worked example beats a definition. The short attention scale published in 2017 shows 96.0% specificity in the general population, which looks excellent. Its positive predictive value there is nonetheless 67.3%: among people scoring above the threshold, about two in three meet the criteria after assessment and one in three does not. Both figures come from the same study and do not contradict each other.

• A performance figure belongs to a study, a population and a threshold: taking it out of that context makes it false.
• High specificity in the general population guarantees nothing for an audience already asking itself the question.
• A translated or shortened scale does not inherit the original's performance: it has to be measured again.
• None of these indicators says what to do next. A conversation with a professional does.