You are here

  • Home
  • Archive
  • Volume 99, Issue 1
  • Evaluation of the NICE mini-GRACE risk scores for acute myocardial infarction using the Myocardial Ischaemia National Audit Project (MINAP) 2003–2009: National Institute for Cardiovascular Outcomes Research (NICOR)

Article Text

Article menu

Download PDFPDF

Original articles
Acute coronary syndromes
Evaluation of the NICE mini-GRACE risk scores for acute myocardial infarction using the Myocardial Ischaemia National Audit Project (MINAP) 2003–2009: National Institute for Cardiovascular Outcomes Research (NICOR)
  1. Alexander D Simms1,2,3,
  2. Stephanie Reynolds1,
  3. Karen Pieper4,
  4. Paul D Baxter1,
  5. Brian A Cattle1,
  6. Phillip D Batin5,
  7. John I Wilson5,
  8. John E Deanfield6,
  9. Robert M West7,
  10. Keith A A Fox8,
  11. Alistair S Hall1,9,
  12. Christopher P Gale1,3
  1. 1Centre for Epidemiology and Biostatistics, University of Leeds, Leeds, UK
  2. 2York and Hull Medical School, University of York, York, UK
  3. 3Department of Cardiology, York Teaching Hospital NHS Foundation Trust, York, UK
  4. 4Duke Clinical Research Institute, Duke University Medical Center, Durham, North Carolina, USA
  5. 5Department of Cardiology, Pinderfields General Hospital, Wakefield, UK
  6. 6National Institute for Cardiovascular Outcomes Research, University College London, London, UK
  7. 7Leeds Institute of Health Sciences, University of Leeds, Leeds, UK
  8. 8Centre for Cardiovascular Science, University of Edinburgh, Edinburgh, UK
  9. 9Department of Cardiology, Leeds Teaching Hospitals NHS Trust, Leeds, UK
  1. Correspondence to Dr A D Simms, Centre for Epidemiology and Biostatistics, Level 8, Worsley Building, University of Leeds, Clarendon Way, West Yorkshire, Leeds LS2 9JT, UK; simmsy{at}doctors.org.uk

Abstract

Objective To evaluate the performance of the National Institute for Health and Clinical Excellence (NICE) mini-Global Registry of Acute Coronary Events (GRACE) (MG) and adjusted mini-GRACE (AMG) risk scores.

Design Retrospective observational study.

Setting 215 acute hospitals in England and Wales.

Patients 137 084 patients discharged from hospital with a diagnosis of acute myocardial infarction (AMI) between 2003 and 2009, as recorded in the Myocardial Ischaemia National Audit Project (MINAP).

Main outcome measures Model performance indices of calibration accuracy, discriminative and explanatory performance, including net reclassification index (NRI) and integrated discrimination improvement.

Results Of 495 263 index patients hospitalised with AMI, there were 53 196 ST elevation myocardial infarction and 83 888 non-ST elevation myocardial infarction (NSTEMI) (27.7%) cases with complete data for all AMG variables. For AMI, AMG calibration was better than MG calibration (Hosmer–Lemeshow goodness of fit test: p=0.33 vs p<0.05). MG and AMG predictive accuracy and discriminative ability were good (Brier score: 0.10 vs 0.09; C statistic: 0.82 and 0.84, respectively). The NRI of AMG over MG was 8.1% (p<0.05). Model performance was reduced in patients with NSTEMI, chronic heart failure, chronic renal failure and in patients aged ≥85 years.

Conclusions The AMG and MG risk scores, utilised by NICE, demonstrated good performance across a range of indices using MINAP data, but performed less well in higher risk subgroups. Although indices were better for AMG, its application may be constrained by missing predictors.

  • Quality Of Care And Outcomes

https://doi.org/10.1136/heartjnl-2012-302632

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction

    The Global Registry of Acute Coronary Events (GRACE) risk score is recognised internationally as a tool for the risk stratification of non-ST elevation acute coronary syndromes,1–7 and its use in routine clinical practice is recommended by the European Society of Cardiology and the National Institute for Health and Clinical Excellence (NICE).8 ,9 Because it collects patient level data, an additional application of the GRACE risk score is case mix adjustment for comparative provider performance assessment.10 However, established acute coronary syndrome registries such as the Myocardial Ischaemia National Audit Project (MINAP) may not collect all eight GRACE predictor variables.11

    In light of this, and as part of the development of a national clinical guideline for the management of unstable angina and non-ST elevation myocardial infarction (NSTEMI) in England, NICE utilised a modified version of the GRACE risk score using MINAP data (2005–2008). This model excluded creatinine and Killip class, and was named the mini-GRACE (MG) risk score.12 The MG risk score was further modified with ‘prescription of a loop diuretic during admission’ as a surrogate for Killip class and creatinine concentration re-coded as a categorical variable, and termed the adjusted mini-GRACE (AMG) risk score.12

    The AMG and MG risk scores have not however been studied in detail in patients with ST elevation myocardial infarction (STEMI) and NSTEMI using data from the MINAP registry. Using a range of model performance indices, we aimed to evaluate AMG and MG, to ascertain whether they may be considered for patient level adjustment in studies of provider performance using MINAP data.

    Methods

    Study design

    The analyses were based on data from MINAP whose national database was established in 1999 to examine the quality of management of acute myocardial infarction (AMI) in England and Wales and to meet the audit requirements of the National Service Framework for Coronary Heart Disease.11–15 This study includes data collected on behalf of the British Cardiovascular Society under the auspices of the National Institute for Cardiovascular Outcomes Research (NICOR) in which patient identity was protected.

    Data for patients admitted with an acute coronary syndrome are collected prospectively at each acute hospital by a secure electronic system, developed by the Central Cardiac Audit Database, electronically encrypted and transferred online to a central database.11 ,16 The Central Cardiac Audit Database is part of the NICOR based at University College London. MINAP is overseen by a multi-professional steering group representing the stakeholders17 and the NICOR executive.18 Each patient entry offers details of the patient journey, including the method and timing of admission, inpatient investigations, treatment and date of all-cause death (from linkage to the Medical Research Information System, part of the National Health Service (NHS) Information Centre using a unique NHS number). Data entry is subject to routine online error checking. There is a mandatory annual data validation exercise for each hospital.19

    Ethics

    NICOR, which includes MINAP (Ref: NIGB: ECC 1-06 (d)/2011), has support under section 251 of the NHS Act 2006. On seeking advice from Leeds (West) Research Ethics Committee, formal ethical approval was not required under NHS research governance arrangements for the study.

    Cohort description

    The MINAP cohort comprised 495 263 index patients with a final diagnosis of AMI admitted to 236 acute hospitals in England and Wales between 1 January 2003 and 31 December 2009. Of those, 137 084 (27.7%) patient records held complete data to permit calculation of the AMG risk score, which formed the analytic cohort from 215 acute hospitals. Thus patients with complete AMG data had an MG and AMG score calculated and were compared.

    The final diagnosis was formed from the patient's presenting history, clinical examination and the results of inpatient investigations (including troponin measurement), and made by a senior member of the medical staff prior to entry of data into MINAP. The consensus document of the Joint European Society of Cardiology/American College of Cardiology20 was used as the diagnostic standard for AMI and provided the basis for categorisation into STEMI and NSTEMI.

    Statistical methods

    The MG risk score for 6 month mortality from hospitalisation with AMI comprised six of the eight GRACE variables: age, admission systolic blood pressure, heart rate, electrocardiographic ST segment deviation, cardiac arrest and elevated cardiac enzymes (defined as a cardiac troponin concentration >0.05 ng/ml). In line with NICE guidance,9 ,12 an MG score was calculated for each patient using the scoring system and coefficients described by the GRACE investigators.3 For the AMG risk score, ‘use of loop diuretic’ (as a surrogate for Killip class II, and assigned a score of 20 points) and creatinine concentration (using the coefficients described by GRACE) were added to the MG risk score, and the total score calculated for each patient.

    The AMG and MG variables were used to predict 6 month all-cause mortality risk using logistic regression with binomial distribution and a log link. The relationship between the scores and the predicted 6 month mortality risk was quantified using Pearson's correlation coefficient. They were also graphically compared with the published corresponding 24 categories of the GRACE predicted 6 month mortality.3

    Model performance indices

    The Hosmer–Lemeshow goodness of fit test (HLT) was used to assess calibration of the models—whether the observed mortality rates matched the expected mortality rates in subgroups of 100 patients. A high HLT χ2 value and p value <0.05 suggests a significant disagreement between observed and expected rates and therefore a poor model fit.21 A McFadden's pseudo R2 was used to quantify the proportion of the variation in the data explained by the linear expression for the mean part of the model. A pseudo R2<0.2 suggests a poor model fit, 0.2–0.4 a fair model fit and >0.4 a good model fit.22 The Brier score was used to measure the average squared deviation between the predicted probabilities of death and the observed mortality rates—the accuracy of the probability assessment. A Brier score of 1 suggests no accuracy, a score of 0 suggests perfect accuracy and a score of 0.25 suggests half the predicted events are correct and the other half not.23 The C statistic was used to evaluate the discriminative performance of the models. A C statistic close to 1 suggests near perfect discrimination and close to 0.5 almost no discrimination.23

    The increased discriminative value of AMG over MG was examined with the net reclassification index (NRI) and the integrated discrimination improvement (IDI).23 The NRI was used to quantify the proportion of patients moving up or down 6 month mortality risk categories of <3% (low), 3.0% to <6% (intermediate) and ≥6% (high)12 between the two models. The IDI considers the change in the estimated prediction probabilities as a continuous rather than prespecified categorical variable.

    Model performance was studied in the analytic cohort of AMI, including STEMI and NSTEMI, and within subgroups of patients with diabetes mellitus, chronic heart failure, chronic renal failure, patients aged ≥85 years and by sex. All analyses were performed using Stata V.11.2 (2009).

    Results

    Study characteristics

    Of the analytic cohort with complete data for AMG, 53 196 (38.8%) were STEMI and 83 888 (61.2%) were NSTEMI, from 215 hospitals in England and Wales. The 6 month mortality rates (95% CI) from admission were STEMI 12.3% (12.0% to 12.6%) and NSTEMI 15.6% (15.3% to 15.8%). The baseline characteristics illustrate older age and greater comorbidity in NSTEMI than STEMI patients (table 1). Although there appeared to be limited systematic differences between missing and complete AMG variable data, higher 6 month mortality rates were observed for both STEMI and NSTEMI missing AMG variable cases compared with complete cases.

    View this table:
    Table 1

    Baseline characteristics of the analytic cohort by STEMI and NSTEMI for the adjusted mini-GRACE risk score

    Predicted mortality risk

    The mean (SD) MG and AMG 6 month risk scores were: AMI 111.8 (29.8) and 126.0 (35.2); STEMI 113.1 (29.8) and 125.2 (34.2); and NSTEMI 111.0 (29.8) and 126.5 (35.8), respectively. Each risk score showed good correlation with its respective 6 month mortality prediction in patients with AMI (MG: r=0.82, p<0.001; AMG: r=0.80, p<0.001) (figure 1).

    Figure 1
    Figure 1

    Kernel weighted local polynomial smoothing plots of (A) mini-GRACE risk score and (B) adjusted mini-GRACE risk score against respective predicted 6 month mortality following acute myocardial infarction. GRACE, Global Registry of Acute Coronary Events.

    The MG risk score overpredicted 6 month mortality, although at lower risk (<5% risk of 6 month mortality) it correlated well with the published GRACE predicted mortality risk3 (figure 2). Overall there was a strong correlation between the AMG risk score and GRACE 6 month mortality prediction. For patients with a GRACE predicted 6 month mortality risk ≤20% (which encompassed around 75% of the analytic cohort) however, the AMG score underpredicted 6 month mortality.

    Figure 2
    Figure 2

    (A) Scatterplot of GRACE versus mini-GRACE (MG) predicted 6 month mortality risk. (B) Scatterplot of GRACE versus MG predicted 6 month mortality risk restricted to risk ≤20%. (C) Scatterplot of GRACE versus adjusted mini-GRACE (AMG) predicted 6 month mortality risk. (D) Scatterplot of GRACE versus AMG predicted 6 month mortality risk restricted to risk ≤20%. GRACE, Global Registry of Acute Coronary Events.

    Model performance

    While both risk scores demonstrated good accuracy, and discriminative and explanatory performance, the MG risk score showed significant disagreement between observed and expected 6 month mortality rates for AMI and STEMI, and AMG for STEMI and NSTEMI (table 2). However, MG and AMG accuracy, and discriminative and explanatory performance were better for STEMI.

    View this table:
    Table 2

    Mini-GRACE and adjusted mini-GRACE model performance

    Model reclassification

    The increased discriminative value of AMG over MG was shown by IDI=2.6% (p<0.05) and NRI=8.1% (p<0.05). The AMG risk score reclassified 1247 (4.0%) and 212 (0.7%) low risk patients to intermediate and high risk, respectively. It reclassified 5954 (27.4%) and 2519 (11.6%) intermediate risk patients to low and high risk, respectively, and only 7425 (9.4%) high risk patients to intermediate risk. For STEMI the IDI was 2.4% (p<0.05) and the NRI was 6.5% (p<0.05), and for NSTEMI the IDI was 2.6% (p<0.05) and the NRI was 8.4% (p<0.05). A confusion matrix of MG and AMG risk category against observed 6 month mortality is shown in table 3.

    View this table:
    Table 3

    Confusion matrix of observed 6 month mortality against mini-GRACE and adjusted mini-GRACE predicted 6 month risk group

    Model performance in higher risk subgroups

    For both AMG and MG, accuracy, and discriminative and explanatory performance was reduced in patients with chronic heart failure, chronic renal failure and in patients aged ≥85 years, and increased in women, STEMI and diabetes (table 4).

    View this table:
    Table 4

    Mini-GRACE and adjusted mini-GRACE model calibration and discrimination by acute myocardial infarction type and subgroup

    Discussion

    The coalition Government has recalibrated the NHS, emphasising that the commissioning and delivery of healthcare should be based on the quality of care delivered rather than metrics of delivery, such as waiting times.24–26 Mortality rates after hospitalisation with AMI, in part, reflect hospital quality of care. Methods such as patient level adjustment allow more considered quantification of hospital level mortality rates.10 This study shows that the MG and AMG risk scores, utilised by NICE,9 ,12 have good model performance across a range of indices and may be used to calculate 6 month mortality risk after hospitalisation with AMI. Consequently, this may allow hospital level AMI care to be studied more appropriately.

    The NICE guidance for unstable angina and NSTEMI recommends coronary intervention and antithrombotic treatments in higher risk patients hospitalised with NSTEMI.9 Yet a well document risk–treatment paradox exists.27–29 The AMG and MG risk scores allow case mix adjustment thereby enabling the comparison of quality of care in England and Wales according to patient risk. That is, the AMG and MG risk scores perform well in MINAP data and simplify the potential for wider clinical application, which may, in turn, improve care.

    No other studies have evaluated the NICE MG and AMG risk scores in patient subgroups across the wider spectrum of AMI phenotypes. The GRACE risk score has been validated in many studies,1 ,2 ,4–6 ,30 ,31 including a study using MINAP data.4 While this study suggested good discriminative performance (C statistic, 95% CI) for 6 month mortality (0.80, 0.79 to 0.80), the model was limited in light of data collected by MINAP at that time. Our study adds to the literature, suggesting that the use of surrogate data for missing GRACE predictors does not result in poor model performance.

    While overall MG and AMG performance was reduced in patients with chronic heart failure, chronic renal failure and in patients aged ≥85 years, overall performance appeared better than that of the GRACE model used in the earlier MINAP validation study (C statistic (95% CI)): 0.80 (0.79 to 0.80) versus MG 0.82 (0.81 to 0.82) and AMG 0.84 (0.83 to 0.84). The disparity may be explained by the differences between the two studies: firstly, the present study utilised the published GRACE risk score coefficients to mimic the score in the real world while Gale et al generated coefficients from the MINAP dataset. Secondly, we modelled additional GRACE variables: cardiac arrest and creatinine, which also may have improved model performance.

    On balance, the AMG risk score appeared to be superior to the MG risk score (reclassifying over 8% of patients of the 6 month MG risk prediction and showing better correlation with published GRACE 6 month risk prediction). AMG however relied on the inclusion of two further variables, which reduced the numbers of available complete cases by 58.9% from 333 515 for complete MG cases to 137 084 for complete AMG cases. Even though modelling data missingness10 and multiple imputation32 are methods by which concerns over data quality may be countered, encouraging good data collection at source is preferential.33 Moreover, mandatory collection of AMG variables in MINAP would allow more appropriate adjustment of case mix so that hospital performance may be represented fairly and relinquish the drive to utilise administrative data such as hospital episode statistics.

    Limitations

    Only 27.7% of cases of AMI had complete data to allow the calculation of the AMG score. A reduced number of cases due to data missingness (or analysis of subgroup populations such as those with diabetes) may have biased model performance and the results may not, therefore, be representative of the wider AMI population. However, systematic differences between those with and without missing data for STEMI and NSTEMI were not substantial (table 1). The HLT was used as an estimate for model calibration when it is influenced by the numbers of cases (and therefore the number of groups) in the study. It is possible that alternative measures of model performance may report different outcomes.

    Conclusion

    NICE advocates the use of the GRACE risk score to predict 6 month mortality for NSTEMI.9 ,12 This study demonstrates that two surrogate (MG and AGM) risk scores of the GRACE model, derived from MINAP data, are applicable to patients hospitalised with AMI in England and Wales. Furthermore, the MG and AMG perform well at generating scores and predicting 6 month mortality and, therefore, are appropriate for use in MINAP for the retrospective adjustment of case mix to allow the comparison of hospital provider performance. While performance indices were better for AMG, the degree of missing data may limit its application.

    Key messages

    • The NICE mini-GRACE (MG) and adjusted mini-GRACE (AMG) risk scores perform well in MINAP data.

    • The AMG risk score performs better than the MG risk score, but its application may be constrained by missing predictors.

    • The AMG and MG risk score allow adjustment for case mix such that hospital care (including the use of evidence based therapies according to risk) may be studied in more detail.

    Acknowledgments

    The extract from the MINAP database was provided by the MINAP Academic Group, National Institute for Cardiovascular Outcomes, University College London. We acknowledge all of the hospitals in England and Wales for their contribution of data to MINAP.

    References

    1. ↵
      1. Bradshaw PJ,
      2. Ko DT,
      3. Newman AM,
      4. et al
      . Validity of the GRACE (Global Registry of Acute Coronary Events) acute coronary syndrome prediction model for six month post-discharge death in an independent data set. Heart 2006;92:905–9.
      OpenUrlAbstract/FREE Full Text
    2. ↵
      1. Fox KAA,
      2. Dabbous OH,
      3. Goldberg RJ,
      4. et al
      . Prediction of risk of death and myocardial infarction in the six months after presentation with acute coronary syndrome: prospective multinational observational study (GRACE). BMJ 2006;333:1091–4.
      OpenUrlAbstract/FREE Full Text
    3. ↵
      Center for Outcomes Research UoMMS. Methods and formulas used to calculate the GRACE risk scores for patients presenting to hospital with an acute coronary syndrome. Worcester, MA: University of Massachusetts Medical School, 2011.
    4. ↵
      1. Gale CP,
      2. Manda SOM,
      3. Weston CF,
      4. et al
      . Evaluation of risk scores for risk stratification of acute coronary syndromes in the Myocardial Infarction National Audit Project (MINAP) database. Heart 2009;95:221–7.
      OpenUrlAbstract/FREE Full Text
    5. ↵
      1. Granger CB,
      2. Goldberg RJ,
      3. Dabbous O,
      4. et al
      . Predictors of hospital mortality in the global registry of acute coronary events. Arch Intern Med 2003;163:2345–53.
      OpenUrlCrossRefPubMedWeb of Science
    6. ↵
      1. Yan AT,
      2. Yan RT,
      3. Tan M,
      4. et al
      . Risk scores for risk stratification in acute coronary syndromes: useful but simpler is not necessarily better. Eur Heart J 2007;28:1072–8.
      OpenUrlAbstract/FREE Full Text
    7. ↵
      1. Fox KA,
      2. Eagle KA,
      3. Gore JM,
      4. et al
      . The Global Registry of Acute Coronary Events, 1999 to 2009—GRACE. Heart 2010;96:1095–101.
      OpenUrlAbstract/FREE Full Text
    8. ↵
      1. Hamm CW,
      2. Bassand JP,
      3. Agewall S,
      4. et al
      . ESC guidelines for the management of acute coronary syndromes in patients presenting without persistent ST-segment elevation. Eur Heart J 2011;32:2999–3054.
      OpenUrlFREE Full Text
    9. ↵
      1. Gray HH,
      2. Henderson RA,
      3. de Belder MA,
      4. et al
      . Early management of unstable angina and non-ST-segment elevation myocardial infarction: summary of NICE guidance. Heart 2010;96:1662–8.
      OpenUrlFREE Full Text
    10. ↵
      1. Gale CP,
      2. Cattle BA,
      3. Moore J,
      4. et al
      . Impact of missing data on standardised mortality ratios for acute myocardial infarction: evidence from the Myocardial Ischaemia National Audit Project (MINAP) 2004–7. Heart 2011;97:1926–31.
      OpenUrlAbstract/FREE Full Text
    11. ↵
      1. Herrett E,
      2. Smeeth L,
      3. Walker L,
      4. et al
      . The Myocardial Ischaemia National Audit Project (MINAP). Heart 2010;96:1264–7.
      OpenUrlAbstract/FREE Full Text
    12. ↵
      Royal College of Physicians. Unstable angina and NSTEMI: the early management of unstable angina and non-ST-segment-elevation myocardial infarction: clinical guideline 94. London: National Institute for Health and Clinical Excellence, 2010.
    13. ↵
      Department of Health. The National Service Framework for coronary heart disease: modern standards and service models. London: DOH, 2000.
    14. ↵
      1. Birkhead J,
      2. Walker L
      . National audit of myocardial infarction (MINAP): a project in evolution. Hosp Med 2004;65:452–3.
      OpenUrlCrossRefPubMedWeb of Science
    15. ↵
      1. Birkhead JS
      . Responding to the requirements of the national service framework for coronary disease: a core data set for myocardial infarction. Heart 2000;84:116–17.
      OpenUrlFREE Full Text
    16. ↵
      1. Rickards A,
      2. Cunningham D
      . From quantity to quality: the central cardiac audit database project. Heart 1999;82:II18–22.
      OpenUrlPubMed
    17. ↵
      National Institute Cardiovascular Outcomes Research (NICOR). The Myocardial Ischaemia National Audit Project (MINAP). London: University College London, 6 November, 2011.
    18. ↵
      1. Gale CP,
      2. Weston C,
      3. Denaxas S,
      4. et al
      . Engaging with the clinical data transparency initiative: a view from the National Institute for Cardiovascular Outcomes Research (NICOR). Heart 2012;98:1040–3.
      OpenUrlFREE Full Text
    19. ↵
      National Institute Cardiovascular Outcomes Research (NICOR). Myocardial Ischaemia National Audit Project (MINAP): How the NHS manages heart attacks. Tenth Public Report 2011. London: University College London, 2011.
    20. ↵
      Myocardial infarction redefined. A consensus document of the Joint European Society of Cardiology/American College of Cardiology Committee for the redefinition of myocardial infarction. Eur Heart J 2000;21:1502–13.
      OpenUrlAbstract/FREE Full Text
    21. ↵
      1. Hosmer DW,
      2. Hosmer T,
      3. Le CS,
      4. et al
      . A comparison of goodness-of-fit tests for the logistic regression model. Stat Med 1997;16:965–80.
      OpenUrlCrossRefPubMedWeb of Science
    22. ↵
      1. McFadden D.
      . Conditional logit analysis of qualitative choice behaviour. In: Zarembka P Frontiers of econometrics. New York: Acad Press, 1974:105–42.
    23. ↵
      1. Steyerberg EW,
      2. Vickers AJ,
      3. Cook NR,
      4. et al
      . Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology 2010;21:128–38.
      OpenUrlCrossRefPubMedWeb of Science
    24. ↵
      Great Britain. Health and social care act. London: The Stationary Office, 2012.
    25. ↵
      Department of Health. Equity and excellence: liberating the NHS. London: The Stationary Office, 2010.
    26. ↵
      Department of Health. Transparency in outcomes—a framework for the NHS. A consultation on proposals. London: The Stationary Office, 2010.
    27. ↵
      1. Fox KA,
      2. Anderson FA Jr.,
      3. Dabbous OH,
      4. et al
      . Intervention in acute coronary syndromes: do patients undergo intervention on the basis of their risk characteristics? The Global Registry of Acute Coronary Events (GRACE). Heart 2007;93:177–82.
      OpenUrlAbstract/FREE Full Text
    28. ↵
      1. Bhatt DL,
      2. Roe MT,
      3. Peterson ED,
      4. et al
      . Utilization of early invasive management strategies for high-risk patients with non ST-segment elevation acute coronary syndromes. JAMA 2004;292:2096–104.
      OpenUrlCrossRefPubMedWeb of Science
    29. ↵
      1. Brieger D,
      2. Eagle KA,
      3. Goodman SG,
      4. et al
      . Acute coronary syndromes without chest pain, an underdiagnosed and undertreated high-risk group: insights from the Global Registry of Acute Coronary Events. Chest 2004;126:461–9.
      OpenUrlCrossRefPubMedWeb of Science
    30. ↵
      1. Eagle KA,
      2. Lim MJ,
      3. Dabbous OH,
      4. et al
      . A validated prediction model for all forms of acute coronary syndrome. JAMA 2004;291:2727–33.
      OpenUrlCrossRefPubMedWeb of Science
    31. ↵
      1. Elbarouni B,
      2. Goodman SG,
      3. Yan RT,
      4. et al
      . Validation of the Global Registry of Acute Coronary Event (GRACE) risk score for in-hospital mortality in patients with acute coronary syndrome in Canada. Am Heart J 2009;158:392–9.
      OpenUrlCrossRefPubMedWeb of Science
    32. ↵
      1. Cattle BA,
      2. Baxter PD,
      3. Greenwood DC,
      4. et al
      . Multiple imputation for completion of a national clinical audit dataset. Stat Med 2011;30:2736–53.
      OpenUrlCrossRefPubMed
    33. ↵
      1. Lilford R,
      2. Mohammed MA,
      3. Spiegelhalter D,
      4. et al
      . Use and misuse of process and outcome data in managing performance of acute medical care: avoiding institutional stigma. Lancet 2004;363:1147–54.
      OpenUrlCrossRefPubMedWeb of Science

    Footnotes

    • Contributors ADS and CPG: concept, design, analysis, interpretation of the data, and drafting and revision of the manuscript. SR: design and analysis of the data. PDB, BAC, RMW and KP: statistical support, analysis, interpretation of the data and revision of the manuscript. PDB, JIW, JED, KAAF and ASH: analysis, interpretation of the data, and drafting and revision of the manuscript.

    • Funding British Heart Foundation grant (PG/07/057/23215) and the National Institute for Health Research grant (NIHR/CS/009/004).

    • Competing interests None.

    • Provenance and peer review Not commissioned; externally peer reviewed.