Indicator studies
| Indicator set Author and date | Truth | Target population | Proposed method of measurement | Testing or implementation | Reliability | Feasibility | |
|---|---|---|---|---|---|---|---|
| Evidence synthesis | Consensus method | ||||||
| RAND Quality of Care Assessment Tools (RAND QA) Moore (2000)29 | Literature review, not specified to be systematic | RAND Appropriateness Method | Not specified | Medical record review | McGlynn et al30 Asch et al31 | Tested in McGlynn et al30 and Asch et al31 —in a 4% sample reabstraction reliability was substantial at 3 levels: presence of a condition (к=0.83), indicator eligibility (к=0.76) and indicator scoring к=0.80) | McGlynn et al30 use a national sample of US citizens in metropolitan areas, using a telephone interview to collect data with subsequent analysis of medical records where consent was given. Asch et al31 use the same data as a comparator for data collected from a random sample of veterans’ health affairs clients and their records |
| ACOVE -1 MacLean (2001)32 33 | Systematic review supporting indicators produced by a content expert working with a project member knowledgeable about systematic reviews and quality indicator development | Modified RAND/UCLA Appropriateness Method | ‘Vulnerable elders’—persons ≥65 years who are at increased risk for death or functional decline | Not specified | Wenger et al34 Chodosh et al35 Higashi et al36 37 Ganz et al38 MacLean et al39 Østerås et al40 | Tested in Wenger et al34—10% sample reabstraction: overall error rate was 1.6%; also in Chodosh et al35—‘Inter-rater reliability of chart abstraction for eligibility and scoring of indicators was 95%.’; Higashi et al36 37—10% reabstraction sample showed 97% identical eligibility and 95% identical eligibility and quality score; MacLean et al39—10% reabstraction sample with к=0.85 (93% agreement); Østerås et al40 questionnaire-based test-retest к=0.20–0.80, % exact agreement from 62–90% | Wenger et al34 implemented the indicators in community dwelling VEs in the USA—medical record abstraction by trained nurses supplemented for some indicators by telephone interview. Chodosh et al,35 Higashi et al36 37 and MacLean et al39 used the same population and methods. Ganz et al used a similar population and methodology at a different time point38 Østerås et al40 implemented some of these indicators in a patient self-report format |
| ACOVE-1 adapted for the ELSA Steel et al (2004)41 | Transposition of previous ACOVE work (referenced). 26 new indicators for the set were suggested by the panel | Modified RAND/UCLA Appropriateness Method | Older patients in the UK ≥65 years) | Interviews for the ELSA | Steel et al,12 14 Broadbent et al,13 Østerås et al40 | Tested in Steel et al12—(к=0.8, 95% CI=0.7 to 0.9); Østerås et al40 questionnaire-based test-retest к=0.20–0.80, % exact agreement from 62–90% | Broadbent et al13 and Steel et al12 separately implemented indicators in UK general practice, using medical record review (computerised and paper notes). Østerås et al40 implemented some of these indicators in a patient self-report format |
| ACOVE-1 adapted for NH implementation (ACOVE/NH) Saliba et al (2005)42 | Previous referenced (ACOVE) work, plus expert opinion (for modification) and additional indicator development, methodology not specified in detail | Modified Delphi; subsequent overview by ACOVE clinical committee | Long-stay NH residents ≥65 years Exclusions for advanced dementia or poor prognosis | Not specified | Cadogan et al43 Zingmond et al44 | Tested in Cadogan et al43—к=0.65–1.00 and percentage agreement 80–100 where к could not be calculated (numbers too low) | Cadogan et al43 implemented indicators in 30 nursing homes in California using medical record review. Zingmond et al44 implemented using Medicare and Medicaid eligibility and claims data and a nursing home minimum dataset |
| ACOVE-1 adapted for the HPCQI Smith et al (2007)45 | Based on ACOVE indicators, plus some additional (non-OA) indicators. ACOVE work referenced; additional expert opinion | Modified Delphi techniques | Patients ≥60 years who are homebound | Not specified | No published examples of testing identified | No reliability testing identified | No feasibility testing identified |
| ACOVE-3 ACOVE investigators (2007)46–48 | A systematic review supporting potential indicators produced by a content expert working with a project member knowledgeable about systematic reviews and indicator development | Modified version of the RAND/UCLA Appropriateness Method | Community-dwelling individuals aged ≥65 years who are at greater risk of death or functional decline over a 2-year period | Medical records and/or administrative data, patient or proxy interview | Østerås et al40 | Østerås et al40 questionnaire-based test-retest к=0.20–0.80, % exact agreement from 62–90% | Østerås et al40 implemented some of these indicators in a modified patient self-report format |
| QIGP Underwood (2002)49 | Various sources used (Cochrane, DARE, Medline) but not clear how the evidence was assembled. Cites meta-analyses, systematic reviews, randomised controlled trials | Not stated | Not specified | Not specified | Kirk et al50 Steel et al12 Broadbent et al13 Østerås et al40 | Tested for non-OA indicators in Kirk et al50—as OA was not included in this exercise, it is not known what degree of reliability exists for these indicators; Østerås et al40 questionnaire-based test-retest к=0.20–0.80, % exact agreement from 62–90% | Kirk et al50 implemented in 16 UK general practices in two areas using data from electronic and paper records. Steel et al12 and Broadbent et al13 separately implemented the NSAID indicator in UK general practice, using medical record review (computerised and paper notes). Østerås et al40 implemented some of these indicators in a modified patient self-report format |
| Arthritis Foundation Arthritis Foundation 200451 52 | Comprehensive literature search and expert opinion | Modified RAND/UCLA Appropriateness Method | Patients with OA | Not specified | Li et al53 | No reliability testing identified | Li et al53 used a postal survey in Canada (sampling frame from an administrative database in British Columbia) to assess non-pharmacological indicators |
| PCPI (2006)54 | PCPI website refers to a methodology committee but no specific information in the indicator set to identify how it was developed | All patients aged ≥21 years with a diagnosis of OA | Medical record data extraction (detailed numerator and denominator information provided) | No published examples of testing identified | No reliability testing identified | No feasibility testing identified | |
| EUMUSC.net (2012)55 | Developed from the EUMUSC.net standards of care for OA and refined by researchers and patient representatives | All adult patients with OA of hand, hip or knee | Varies. Examples include patient record or survey. Numerator and denominator clearly identified | No published examples of testing identified | No reliability testing identified | No feasibility testing identified | |
ACOVE, Assessing Care of Vulnerable Elders; DARE, Database of Abstracts of Reviews of Effects; ELSA, English Longitudinal Study of Ageing; EUMUSC.net, European Musculoskeletal Conditions Surveillance and Information Network; HPCQI, Home-based Primary Care Quality Initiative; NH, nursing home; NSAIDs, non-steroidal anti-inflammatories; OA, osteoarthritis, PCPI, Physician Consortium for Performance Improvement; QIGP, Quality Indicators for General Practice; UCLA, University of California, Los Angeles; VE, vulnerable elder.