Table 2

Indicator studies

Indicator set
Author and date
TruthTarget populationProposed method of measurementTesting or implementationReliabilityFeasibility
Evidence synthesisConsensus method
RAND Quality of Care Assessment Tools (RAND QA)
Moore (2000)29
Literature review, not specified to be systematicRAND Appropriateness MethodNot specifiedMedical record reviewMcGlynn et al30
Asch et al31
Tested in McGlynn et al30 and Asch et al31 —in a 4% sample reabstraction reliability was substantial at 3 levels: presence of a condition (к=0.83), indicator eligibility (к=0.76) and indicator scoring к=0.80)McGlynn et al30 use a national sample of US citizens in metropolitan areas, using a telephone interview to collect data with subsequent analysis of medical records where consent was given. Asch et al31 use the same data as a comparator for data collected from a random sample of veterans’ health affairs clients and their records
ACOVE -1
MacLean (2001)32 33
Systematic review supporting indicators produced by a content expert working with a project member knowledgeable about systematic reviews and quality indicator developmentModified RAND/UCLA Appropriateness Method‘Vulnerable elders’—persons ≥65 years who are at increased risk for death or functional declineNot specifiedWenger et al34
Chodosh et al35
Higashi et al36 37
Ganz et al38
MacLean et al39 Østerås et al40
Tested in Wenger et al34—10% sample reabstraction: overall error rate was 1.6%; also in Chodosh et al35—‘Inter-rater reliability of chart abstraction for eligibility and scoring of indicators was 95%.’; Higashi et al36 37—10% reabstraction sample showed 97% identical eligibility and 95% identical eligibility and quality score; MacLean et al39—10% reabstraction sample with к=0.85 (93% agreement); Østerås et al40 questionnaire-based test-retest к=0.20–0.80, % exact agreement from 62–90%Wenger et al34 implemented the indicators in community dwelling VEs in the USA—medical record abstraction by trained nurses supplemented for some indicators by telephone interview. Chodosh et al,35 Higashi et al36 37 and MacLean et al39 used the same population and methods. Ganz et al used a similar population and methodology at a different time point38
Østerås et al40 implemented some of these indicators in a patient self-report format
ACOVE-1 adapted for the ELSA
Steel et al (2004)41
Transposition of previous ACOVE work (referenced). 26 new indicators for the set were suggested by the panelModified RAND/UCLA Appropriateness MethodOlder patients in the UK ≥65 years)Interviews for the ELSASteel et al,12 14 Broadbent et al,13 Østerås et al40Tested in Steel et al12—(к=0.8, 95% CI=0.7 to 0.9); Østerås et al40 questionnaire-based test-retest к=0.20–0.80, % exact agreement from 62–90%Broadbent et al13 and Steel et al12 separately implemented indicators in UK general practice, using medical record review (computerised and paper notes).
Østerås et al40 implemented some of these indicators in a patient self-report format
ACOVE-1 adapted for NH implementation (ACOVE/NH)
Saliba et al (2005)42
Previous referenced (ACOVE) work, plus expert opinion (for modification) and additional indicator development, methodology not specified in detailModified Delphi; subsequent overview by ACOVE clinical committeeLong-stay NH residents ≥65 years Exclusions for advanced dementia or poor prognosisNot specifiedCadogan et al43
Zingmond et al44
Tested in Cadogan et al43—к=0.65–1.00 and percentage agreement 80–100 where к could not be calculated (numbers too low)Cadogan et al43 implemented indicators in 30 nursing homes in California using medical record review. Zingmond et al44 implemented using Medicare and Medicaid eligibility and claims data and a nursing home minimum dataset
ACOVE-1 adapted for the HPCQI
Smith et al (2007)45
Based on ACOVE indicators, plus some additional (non-OA) indicators. ACOVE work referenced; additional expert opinionModified Delphi techniquesPatients ≥60 years who are homeboundNot specifiedNo published examples of testing identifiedNo reliability testing identifiedNo feasibility testing identified
ACOVE-3
ACOVE investigators (2007)46–48
A systematic review supporting potential indicators produced by a content expert working with a project member knowledgeable about systematic reviews and indicator developmentModified version of the RAND/UCLA Appropriateness MethodCommunity-dwelling individuals aged ≥65 years who are at greater risk of death or functional decline over a 2-year periodMedical records and/or administrative data, patient or proxy interviewØsterås et al40Østerås et al40 questionnaire-based test-retest к=0.20–0.80, % exact agreement from 62–90%Østerås et al40 implemented some of these indicators in a modified patient self-report format
QIGP
Underwood (2002)49
Various sources used (Cochrane, DARE, Medline) but not clear how the evidence was assembled. Cites meta-analyses, systematic reviews, randomised controlled trialsNot statedNot specifiedNot specifiedKirk et al50
Steel et al12
Broadbent et al13
Østerås et al40
Tested for non-OA indicators in Kirk et al50—as OA was not included in this exercise, it is not known what degree of reliability exists for these indicators; Østerås et al40 questionnaire-based test-retest к=0.20–0.80, % exact agreement from 62–90%Kirk et al50 implemented in 16 UK general practices in two areas using data from electronic and paper records. Steel et al12 and Broadbent et al13 separately implemented the NSAID indicator in UK general practice, using medical record review (computerised and paper notes).
Østerås et al40 implemented some of these indicators in a modified patient self-report format
Arthritis Foundation
Arthritis Foundation 200451 52
Comprehensive literature search and expert opinionModified RAND/UCLA Appropriateness MethodPatients with OANot specifiedLi et al53No reliability testing identifiedLi et al53 used a postal survey in Canada (sampling frame from an administrative database in British Columbia) to assess non-pharmacological indicators
PCPI (2006)54PCPI website refers to a methodology committee but no specific information in the indicator set to identify how it was developedAll patients aged ≥21 years with a diagnosis of OAMedical record data extraction (detailed numerator and denominator information provided)No published examples of testing identifiedNo reliability testing identifiedNo feasibility testing identified
EUMUSC.net (2012)55Developed from the EUMUSC.net standards of care for OA and refined by researchers and patient representativesAll adult patients with OA of hand, hip or kneeVaries. Examples include patient record or survey.
Numerator and denominator clearly identified
No published examples of testing identifiedNo reliability testing identifiedNo feasibility testing identified
  • ACOVE, Assessing Care of Vulnerable Elders; DARE, Database of Abstracts of Reviews of Effects; ELSA, English Longitudinal Study of Ageing; EUMUSC.net, European Musculoskeletal Conditions Surveillance and Information Network; HPCQI, Home-based Primary Care Quality Initiative; NH, nursing home; NSAIDs, non-steroidal anti-inflammatories; OA, osteoarthritis, PCPI, Physician Consortium for Performance Improvement; QIGP, Quality Indicators for General Practice; UCLA, University of California, Los Angeles; VE, vulnerable elder.