Validation of diagnostic codes within medical services claims
Introduction
Administrative clinical databases are increasingly being used for the purposes of medical research. During the past decade alone, many studies have been published that use hospital discharge claims and physician medical services claims data to examine various health and policy issues. Examples of this literature include studies that investigate health outcomes [1], [2], [3], [4], [5], [6], drug utilization review [7], utilization of services [8], [9], [10], [11], [12], policy evaluation [13], [14], prevalence/incidence and surveillance [15], [16], [17], risk adjustment/health economics [18], [19], [20], and physician profiling and quality of care [21], [22], [23], [24], [25], [26], [27], [28], [29].
Two areas of research where administrative clinical databases are of importance are observational epidemiologic studies and drug utilization reviews. The primary advantages of using administrative data for these purposes are that these data are comprehensive, cost efficient, and free of the usual biases associated with survey methods such as recall bias, nonresponse, and subject attrition [30]. However, the utility of these databases for research differs substantially, particularly as it relates to the comprehensiveness of population information coverage, as well as the source and reason for documentation of diagnostic information.
Hospital claims are frequently used for research, but they are limited to information during periods of hospitalization, capturing diagnostic and treatment information for a very ill population over a defined, and usually brief window of time. The chief advantage of hospital claims data for research is that discharge diagnostic and medical procedure information is recorded by medical archivists, based on a detailed review of the medical chart, increasing the likelihood of accurate documentation.
In contrast, medical services claims data cover the full continuum from ambulatory to hospital-based care, providing information on almost all contacts with physicians in the health care system. As medical services data are created as a by-product of claims for physician reimbursement in a fee-for-service billing system, almost all services provided by fee-for-service physicians will be recorded [31], [32]. However, only the procedure code (e.g., major assessment visit; closed reduction of femur fracture), which is linked to the level of reimbursement, is carefully audited. Diagnostic information that is recorded on each medical service claim, indicating the reason for the medical service, is not typically validated, as these data are not usually linked to remuneration.
There is increasing interest in using medical services claims data for medical research as they capture many of the populations of interest for epidemiologic studies, particularly those that receive the majority of care in the ambulatory setting [24], [32], [33], [34], [35], [36], [37], [38], [39], [40], [41], [42]. Indeed, Klabunde showed that the medical services claims diagnoses augmented information recorded in hospital claims, increasing the detected prevalence of Charlson comorbidity index conditions from 10 to 25%, particularly for conditions such as diabetes that tend to be managed on an ambulatory basis [32]. Yet, the accuracy of the diagnosis data remains suspect as it is neither collected for the purpose of clinical research nor is it directly linked to remuneration. This unproven accuracy limits the potential utility of these databases for research.
Few studies have attempted to validate the diagnoses recorded in medical service claims, and only a small proportion of these have attempted to do so using the medical chart as a gold standard. A number of studies have compared patient self-reported health problems with diagnoses recorded within medical claims and showed modest concordance [30], [43]. However, patient self-report likely underestimates the accuracy of diagnostic codes in claims data because patients are not necessarily aware of all diagnoses recorded by their physicians [44]. Direct comparisons between chart documented diagnoses by physicians and diagnostic data recorded in medical services claims is generally associated with higher degrees of concordance [45], [46], but investigation has been limited to a select number of conditions [47], or a small number of patients and physicians [46]. Furthermore, there has been no direct chart-based validation of medical services claims diagnoses for commonly used methods of comorbidity assessment or case-mix adjustment [35]. To increase the potential utility of medical services claims for research, priority needs to be placed on the validation of frequently used measures of diagnosis-based risk adjustment, as these measures are essential for unbiased comparisons in observational studies.
The Charlson index is one of the most frequently used comorbidity measures [35]. The index is based on diagnoses of 18 disease conditions, and scores are weighted by the relative risk of mortality [48]. Although developed and validated as a hospital-based measure of mortality risk [48], it is increasingly used for ambulatory populations [33]. A recent study, which compared Charlson index values based on hospital relative to medical services claims diagnostic data suggested that medical services diagnostic data yielded higher estimates of mortality risk for some conditions, while risk estimates were equivalent or lower for others [32]. The possibility that the discrepancy in estimated mortality risks is related to coding inaccuracies in medical claims diagnoses has not been assessed.
Case-mix adjustment is essential for unbiased comparisons in quality of care and outcome studies [37], [39], [49]. The Johns Hopkins adjusted clinical groups systems is an approach that is used to predict health care utilization and costs based on groupings of medical service claims diagnoses into 32 homogeneous classes and subclasses [50]. Although the Johns Hopkins system has been shown to predict up to 30% of future physician use and costs, variation in predictive capacity has been shown between studies and populations [37], [38]. To date, there has been no direct chart-based validation of the accuracy of diagnostic information used to classify patients into ambulatory diagnostic groups, and this assessment is particularly timely given the interest in using diagnosis-based case-mix adjustment to formulate equitable reimbursement policies for physicians paid on a per capita basis [24], [39], [51].
Finally, the demand for better methods of drug utilization review has highlighted the need to validate diagnostic information in medical services claims data so that it could be used in the surveillance of current practices [52]. Many potentially preventable admissions for drug-related illness are related to drug-disease contraindications [53], yet in ambulatory settings there is no systematically collected source of validated disease information that could be used for drug and disease utilization review. Medical services claims diagnostic codes provide a potentially viable option by which drug-disease utilization can be conducted.
In this study, data collected for a cohort of elderly patients enrolled in a clinical trial were used to assess the validity of medical services claims diagnoses in relationship to diagnoses recorded in the medical chart. These data were used to determine the sensitivity and specificity of medical services claims diagnoses for surveillance of 14 drug disease contraindications used in drug utilization review, the Charlson comorbidity index [48], and the Johns Hopkins Adjusted Ambulatory Care Group (ACG) Case-Mix profile [54].
Section snippets
Context
The validity of medical service claims diagnostic data was assessed in Quebec, where a universal health insurance program covers the costs of medical and hospital care for all provincial residents. Similar to other Canadian provinces [55], a provincial health insurance agency administers the universal health plan, which includes the registration of provincial beneficiaries and payment of physicians who provide services to Quebec beneficiaries. Services provided outside of the province or
Results
In the year prior to the start of the MOXXI study, 631,488 fee-for-service claims were retrieved for the 14,980 patients, of which 163,129 (26%) were claims submitted by the MOXXI primary care physicians. Overall 70% of the claims contained valid operational ICD-9 codes; both for all physicians and the subset of claims submitted by MOXXI physicians (Table 2). The most frequent invalid code was V999, a default code that is commonly inserted in computerized billing software when no diagnostic
Discussion
Data within medical services claims files represent a potentially rich resource for health service and epidemiologic research. However, the assembly of populations for study, and the adjustment for differences in case-mix between comparison groups, depends to a great extent on the validity of diagnostic information in these administrative databases. The relative paucity of scientific investigation concerning the validity of diagnostic data within medical services claims data is, in part,
Acknowledgements
This study was funded by the Medical Research Council of Canada, Fonds de la recherche en santé du Québec. We are extremely thankful to the medical staff at the Royal Victoria Hospital for their helpful assistance in identifying procedure codes within the RAMQ database, Dr. Michael Edwardes for his helpful comments and advice, and of course, to the MOXXI research team, without whom this research would not have been possible.
References (72)
- et al.
Use of insurance claims databases to evaluate the outcomes of ophthalmic surgery
Surv Ophthalmol
(1997) - et al.
Selecting s patient characteristics index for the prediction of medical outcomes using administrative claims data
J Clin Epidemiol
(1995) - et al.
Trends in physician-diagnosed asthma prevalence in Manitoba between 1980 and 1990
Chest
(1993) - et al.
Comparison of survey and physician claims data for detecting hypertension
J Clin Epidemiol
(1997) - et al.
Development of a comorbidity index using physician claims data
J Clin Epidemiol
(2000) - et al.
Risk adjustment for older hospitalized persons: a comparison of two methods of data collection for the Charlson index
J Clin Epidemiol
(2001) - et al.
A new method of classifying prognostic comorbidity in longitudinal studies: development and validation
J Chronic Dis
(1987) - et al.
Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases
J Clin Epidemiol
(1992) - et al.
Adapting a clinical comorbidity index for use with ICD-9-CM administrative data. Differing perspectives
J Clin Epidemiol
(1993) - et al.
Evaluation of two competing methods for calculating Charlson's comorbidity index when analyzing short-term mortality using administrative data
J Clin Epidemiol
(1997)
Using administrative data to predict important health outcomes: entry to hospital, nursing home, and death
Med Care
Myocardial infarction in newly diagnosed hypertensive Medicaid patients free of coronary heart disease and treated with calcium channel blockers
Am J Med
Using physician claims to identify postoperative complications of carotid endarterectomy
Health Serv Res
Dissemination of clinical results. Mastectomy versus lumpectomy and radiation therapy
Med Care
Antimicrobials prescribed for otitis media in a pediatric Medicaid population
Am J Health Syst Pharm
Ambulatory care practice variation within a medicaid program
Health Serv Res
Use of electroconvulsive therapy in the Medicare population between 1987 and 1992
Psychiatr Serv
A detailed comparison of physician services for the elderly in the United States and Canada
JAMA
Mammography and pap smear use by older rural women
Public Health Rep
Health care policy evaluation using longitudinal insurance claims data: an application of the panel ToBit Estimator
Health Econ
The use of EPSDT and other health care services by children enrolled in Medicaid: the impact of OBRA'89
Milbank Q
Diabetes in Hawaii: estimating prevalence from insurance claims data
Am J Public Health
Creating injury episodes using medical claims data
J Trauma
Economic evaluation of Medi-Cal epilepsy patients on monotherapy or polytherapy in 1994–95
AHSR.FHSR Annu Meet Abstr Book
The economic burden of congestive heart failure in a managed care population
Am J Managed Care
Risk-adjusted Medicare capitation rates using ambulatory and inpatient diagnoses
Health Care Financ Rev
Focus on quality: profiling physicians' practice patterns
J Ambul Care Manage
Developing a quality improvement database using health insurance data: a guided tour with application to Medicare's National Claims History file
Am J Med Qual
Variation in office-based quality: a claims-based profile of care provided to Medicare patients with diabetes
JAMA
Profiling resource use by primary-care practices: managed medical implications
Health Care Financ Rev
Profiling primary care physicians for a new managed care network
Health Serv Manage Res
Systemwide provider performance in a Medicaid program. Profiling the care of patients with chronic illnesses
Med Care
Costs vs quality in different types of primary care settings
JAMA
The effect of physician specialty on resource use and outcomes
AHSR.FHSR Annu Meet Abstr Book
Apply case mix adjustment in physician performance profiling using large administrative database
AHSR. FHSR Annu Meet Abstr Book
Cited by (375)
Sex-specific medication trajectories in older adults newly diagnosed with diabetes
2023, Exploratory Research in Clinical and Social PharmacyEffectiveness and Safety of Direct Oral Anticoagulants Among Octogenarians with Venous Thromboembolism: An International Multidatabase Cohort Study
2023, American Journal of MedicineCitation Excerpt :In Québec, medical services are covered for all residents, but the Public Prescription Drug Insurance Plan covers all residents aged ≥65 years, those without private drug insurance plans, and recipients of financial assistance.15 The quality of RAMQ data is high.16,17 The MED-ÉCHO includes records of all hospitalizations in Québec with date of admission and discharge, inpatient diagnoses (coded using ICD-10-CA) and procedures.
Impact of discontinuing government-funded routine eye exams on new glaucoma diagnoses in Ontario
2022, Canadian Journal of Ophthalmology