The evaluation of an individual’s risk of experiencing a future cardiovascular event increasingly forms the basis of clinical guidelines for the prevention of cardiovascular diseases worldwide.¹ Several tools have been developed for risk estimation, most of which are derived from the Framingham Study.^2–4 The Framingham risk equations were developed from a US population during the peak of cardiovascular disease incidence in that country. These equations perform well when applied to populations with a similarly high background risk of cardiovascular disease.^5,6 However, application of Framingham-based risk equations to cohorts in Europe and Asia, as well as to “newer” cohorts in the US, has shown that these tools often overestimate global cardiovascular risk in external populations.^{7,8,9,10,11,12,13,14} Hence, new risk prediction equations have been developed for specific use in European populations, using cohort studies from Europe.¹⁵ However, to date, there has been no similar work for Asian populations, most of which are experiencing a rapidly increasing incidence of cardiovascular diseases.

Resources for risk factor screening are limited in many Asian countries. Consequently, a “low-information” cardiovascular risk prediction equation may be more useful in Asia. Here, we derive such an equation in two ways: from the Framingham Study and from Asian data derived from the Asia Pacific Cohort Studies Collaboration (APCSC). We compare the performance of these equations, after “recalibrating” the Framingham equation, in estimating cardiovascular risk in China, and show how the recalibrated equation may be applied in practice.

METHODS

Participants

The Framingham Study

The Framingham Study has been described in detail previously.³ Published multivariable equations that form the basis of many tools for the estimation of absolute cardiovascular risk have included age, sex, systolic blood pressure (SBP), total cholesterol, high-density lipoprotein cholesterol, diabetes status and smoking status. To ensure comparability with a low-information Asian risk prediction equation with fewer variables, new Framingham risk equations were developed rather than relying on published data. The Framingham population used in this analysis comprised participants who attended either the 11th examination of the original cohort or the baseline examination of the Offspring Cohort. We further restricted the analyses to individuals aged 30–75 years, and those with information, at the 1971 examination, on age, sex, SBP, total cholesterol and smoking status.

The Asia Pacific Cohort Studies Collaboration

The APCSC has also been described in detail previously.¹⁶ The major objective of this individual participant data meta-analysis of cohort studies is to provide reliable evidence about determinants of cardiovascular diseases in the Asia Pacific region. Studies were invited to join the collaboration if they had age, sex and blood pressure measured at baseline and at least 5000 person-years of follow-up, with the cause of death recorded during follow-up. This analysis included individuals aged 30–75 years, participating in cohorts from several Asian countries, with baseline information on age, sex, SBP, total cholesterol and smoking status. Only those cohorts that reported both fatal and non-fatal outcomes were included.

Outcome

The composite outcome of interest was any cardiovascular event, defined as cardiovascular death, non-fatal myocardial infarction or non-fatal cerebrovascular event. For the APCSC cohorts, relevant outcomes were identified using the International Classification of Disease (ICD) codes. As these studies were initiated and followed up during various decades, outcome classification referred to different ICD revisions, although the ninth revision was the most often used.

Comparison of risk factors and disease incidence in the Framingham and the Asian cohorts

A cardiovascular risk prediction model will perform well in a given population if three major characteristics are similar in both the test population and the population from which the model was developed. These characteristics are: (1) the nature and strength of the association between each risk factor included in the model and the risk of a cardiovascular event; (2) mean levels (or prevalences) of the risk factors; and (3) background incidence of cardiovascular disease. The background incidence is, particularly, the most important determinant of accuracy. We compared each of these characteristics in the Framingham cohort and the APCSC cohorts from Japan, Korea and Singapore (Asian cohorts).

Separately for the Framingham and the Asian cohorts, Cox’s proportional hazard models were used to determine the strength of the association between cardiovascular events and a 1-year increase in age, 1 mm Hg increase in SBP, 1 mmol/l increase in total cholesterol and current smoking compared with previous or never smoking. The hazards ratio (HR) and the corresponding 95% confidence interval (CI) for each risk factor were derived separately for men and women, by pooling the individual participant data, stratifying the Cox model by cohort and using all the follow-up data available. Stratification by cohort allows the baseline survival function to vary by cohort; however, relative risks do not vary. The HRs for the Framingham and the Asian cohorts were compared for equality by calculating the z test statistic for the null hypothesis of no difference between the logarithm of HRs (β coefficients):

where SE is the standard error of the log HR, to be compared with the standard normal distribution. The t tests were used to compare the mean age and mean levels of SBP and total cholesterol, separately for men and women, from the Framingham and Asian cohorts. For smoking prevalence, χ² tests were used. The background cardiovascular disease incidence (cardiovascular disease-free survival) was compared by the log rank test.

Low-information cardiovascular risk prediction equations

The log HRs derived from the Cox model for the Asian cohorts provided the weights for the risk equation for the Asian cohorts. Using this equation, each individual’s probability of experiencing a cardiovascular event within 8 years (approximate median follow-up period) is derived by substituting the individual risk profile into the risk equation. In general, the probability of survival to time t for the generic individual i with risk factors {X_i} is

where S(t, X̄) is the baseline survival function for the Asian cohorts estimated from a Cox model, which, in order to derive an overall estimate, was not stratified by cohort, and X̄ are the mean concentrations (or prevalences) of the risk factors observed in the Asian cohorts. Using the same methods, a “low-information” cardiovascular risk equation was also developed from the Framingham Study.

Comparison of the Asian and Framingham cardiovascular risk equations

The performance of the cardiovascular risk prediction equations was evaluated by applying each function to independent Chinese cohorts included in the APCSC. Among the APCSC participants from China, we assessed discrimination (ie, the ability of each equation to distinguish between those who do and those who do not experience a cardiovascular event) and calibration (ie, the degree of similarity between observed rates and predicted risks for each equation). Discrimination was assessed by calculation of the area under the receiver–operating characteristic curve. This represents the probability of the relevant equation assigning a higher risk of cardiovascular event to those who experience such an event than to those who do not. Calibration was evaluated by ranking the APCSC Chinese population into deciles of predicted risk. The Hosmer and Lemeshow goodness-of-fit χ² statistic was then used to measure the agreement between predicted and observed events across the deciles of predicted risk.¹⁷ The overall bias in predicted cardiovascular event rates was estimated as

(predicted rate−observed rate)×100/observed rate.

We further evaluated whether the Framingham equation could be adapted for use in China by a process termed “recalibration”,⁴ which involves substitution of the Framingham baseline survival at 8 years (the complement of the average incidence rate of cardiovascular disease) and mean values (or prevalences) of the risk factors from the target population; in this case, China. As reliable, independent and concurrent data on incidence of disease and levels of risk factors from China are not available, we used pooled data from the non-Chinese Asian cohorts in the APCSC to provide these data. The baseline survival in China was estimated fitting a Cox model on the pooled Chinese cohorts, without cohort stratification.

Country-specific use of the Framingham equation: an example

To show how the low-information Framingham risk equation could be used in Asia in practice today, we also recalibrated this model for a contemporary population in Thailand. To recalibrate, the 8-year baseline survival in the Framingham equation was substituted with the corresponding value from a recent Thai cohort, the Electricity Generating Authority of Thailand (EGAT) study.¹⁸ A population-based cross-sectional study of adults aged ⩾35 years conducted in Thailand in 2000, The International Collaborative Study of Cardiovascular Disease in Asia,¹⁹ provided mean values (and prevalences) of risk factors.

All the statistical analyses were performed using SAS/STAT software V.8.02 and STATA V.9.1.

RESULTS

Comparison of risk factors and disease incidence in the Framingham and the Asian cohorts

Table 1 shows the distribution of the baseline risk factors and the number of cardiovascular events in the Framingham Study and in each APCSC cohort. Mean age, SBP and total cholesterol were higher among men and women from the Framingham cohort than among their counterparts in the APCSC Asian cohorts. Prevalences of smoking were higher among Asian men than those from the Framingham cohort; however, the converse was true for women. During median follow-up periods of 4 years for men and women in the Asian cohorts, 1704 and 581 cardiovascular events were recorded, respectively. For the Framingham cohort, 797 cardiovascular events occurred during 16-years’ median follow-up among men; 571 events occurred over 19-years’ median follow-up among women. In the cohorts from China, 254 and 99 events were recorded in men and women, respectively, during the first 8 years of follow-up. As expected, the incidence of cardiovascular events was considerably higher in the Framingham cohort than in the Chinese and other Asian cohorts. For both men and women, the incidence in China was remarkably similar to that observed overall in the other Asian cohorts.

For age and smoking, the associations with cardiovascular disease were similar between the Framingham and the Asian cohorts, among both men and women (table 2). We found significant heterogeneity in the association between total cholesterol and cardiovascular events among women, although the p value for heterogeneity was borderline; this possibly reflects a higher proportion of haemorrhagic strokes in the Asian cohorts.²⁰ For SBP, the association with cardiovascular events was significantly stronger in Asian populations compared with Framingham.

Low-information cardiovascular risk prediction equations from the Asian and the Framingham cohorts

On the basis of the Asian cohorts in the APCSC, the corresponding generic equation for the calculation of the probability of experiencing a cardiovascular event within 8 years for Asian men is

and for Asian women is

Here S(8) is the 8-year average survival free from cardiovascular events, age, SBP and TC are the mean values of age, SBP and total cholesterol and smoke is the prevalence of current smoking.

Derived from the Framingham Study, the generic equation for the calculation of the probability of experiencing a cardiovascular event within 8 years for men is

and for women is

Comparison of the Asian and Framingham cardiovascular risk equations

The risk equations from the Framingham and the Asian cohorts ranked Chinese men and women similarly according to increasing risk of a cardiovascular event (fig 1). Risk discrimination was good for both equations. The area under the receiver–operating characteristic curve (95% CI) obtained with the Framingham risk equation was 0.75 (0.72 to 0.78) for men and 0.79 (0.74 to 0.83) for women. The corresponding values for the risk equation from the Asian cohorts were 0.76 (0.73 to 0.79) for men and 0.80 (0.75 to 0.84) for women.

Figure 2A shows that the Framingham risk equation systematically overestimated the probability of a cardiovascular event among men and women in China, across all the deciles of predicted risk. The average percentage of overestimation was 276% for men and 102% for women. The values of the Hosmer–Lemeshow χ² (557.5; p<0.001 for men; and 608; p<0.001 for women) reflect the poor ability of the equation to estimate the risk accurately in these Chinese populations. By contrast, the risk prediction equation derived from the Asian cohorts estimated the probabilities of cardiovascular events in the Chinese cohorts much more accurately, with 11% and 10% average overestimation in men and women, respectively (fig 2B). With the Asian risk prediction equation, the Hosmer–Lemeshow χ² was 16.7 (p = 0.033) for men and 12.2 (p = 0.15) for women. Recalibration resulted in substantial improvement in the ability of the Framingham risk equation to estimate probability accurately in the Chinese APCSC participants (fig 2C). The Hosmer–Lemeshow χ² for the recalibrated Framingham equation was 16.7 (p = 0.032) for men and 20.5 (p = 0.009) for women.

Use of the recalibrated Framingham equation in Thailand: an example

On the basis of the EGAT study, the cumulative incidence of cardiovascular disease among Thai men over 8 years was estimated to be 3.4%, resulting in an 8-year baseline cardiovascular disease-free survival (S(8)) of 0.966.¹⁸ The International Collaborative Study of Cardiovascular Disease in Asia¹⁹ provided mean data for Thai men on age (50.3 years), SBP (120 mm Hg) and total cholesterol (5.0 mmol/l), and the prevalence of smoking (48%). Using these data to recalibrate the Framingham cardiovascular risk equation for Thailand, we can estimate the future risk of a cardiovascular event for an individual Thai man. For example, the estimated 8-year cardiovascular risk of a currently smoking, 62-year old Thai man, with an SBP of 148 mm Hg and a total blood cholesterol concentration of 5.20 mmol/l is

That is, this individual has a 13% probability of experiencing a cardiovascular event within the next 8 years.

DISCUSSION

The use of risk prediction tools derived from the Framingham Study has been widely recommended to health professionals, including those outside the US, for the identification of individuals at high risk of cardiovascular disease. The basis of this recommendation is an understanding that the intensity of management of risk factors should be proportional to an individual’s absolute risk of experiencing a cardiovascular event. This paradigm acknowledges the multifactorial aetiology of vascular diseases, and that benefits of intervention are greatest among those at highest baseline risk. This is a cost-effective “medical” approach to the prevention of cardiovascular diseases, and may be particularly important in low-income and middle-income countries in Asia experiencing large increases in the burden of cardiovascular diseases.²¹ Thus, the availability of reliable and inexpensive tools to evaluate the cardiovascular risks among individuals in these populations is important.

In these analyses, we found that the HRs for the associations of most risk factors with cardiovascular disease in the Asian cohorts were broadly similar to those of the Framingham Study cohort. We observed significant differences for SBP and (among women only) total cholesterol, with stronger associations in Asia. For SBP, this is partially explained by a stronger association between blood pressure and cerebrovascular events, which occur more frequently than coronary events in many Asian populations.²² Despite these quantitative differences, we found that the risk equations from the Framingham and the Asian cohorts ranked individual risk similarly, resulting in comparable, good cardiovascular risk discrimination. This underlines the fact that, qualitatively, the major determinants of cardiovascular risk are broadly similar, and act similarly, across populations.

However, the low-information Framingham risk equation substantially and systematically overestimated cardiovascular risk among the Chinese APCSC participants. Published Framingham equations have shown to overestimate coronary heart disease risk in Italian, French, Irish, Hawaiian, English, Danish, German, Swedish and Chinese populations, as well as certain more contemporary subgroups of the US population.^{7–9,11,13,23,24} On the other hand, Framingham-based equations performed well among US populations with a similarly high background risk of cardiovascular risk, such as in the Cardiovascular Heart Study, the Physician Health Study and the Atherosclerosis Risk in Communities Study.⁵ In the Dubbo Study on an elderly cohort from Australia, there was good agreement between observed and predicted cardiovascular rates using Framingham equations.²⁵ An analysis of the cohorts included in the Diverse Population Collaboration confirmed that Framingham-based tools overestimate the risk for populations at low background risk of cardiovascular disease, whereas the reverse was true for populations with high rates of disease.⁸ Our approach differs from that of the project SCORE¹⁵ in allowing “background” survival rates to vary by country: not being explicitly enumerated, but requiring insertion of a country and period-specific rate, as in the example given for Thailand. The systematic coronary risk evaluation (SCORE) approach assumes that all of Europe is either at high or low risk, with a common average rate within each subgroup. It is inconceivable that a common rate would be applicable throughout Asia; no obvious dichotomous split of countries with similar rates exists. In any event, and even if it did, this would soon be out of date due to the epidemiological transition to Western lifestyles taking place at varying rates across the continent.

The poor performance of the Framingham risk equation used in the current analyses cannot be explained by the absence of variables, such as high-density lipoprotein cholesterol and diabetes status, which are included in published Framingham risk functions. Risk prediction in the APCSC Chinese population was substantially improved by using either the Asian cohorts risk equation or the recalibrated low-information Framingham equation. Prediction with the Asian cohorts risk equation was accurate because participants in the APCSC from across the Asian region, including China, shared similar overall background rates of cardiovascular disease and distributions of risk factors despite considerable variation in these characteristics between individual cohorts. This explains why recalibration of the “low-information” Framingham equation, using data from the non-Chinese Asian cohorts, also improved prediction among the Chinese participants. This similarity in accuracy of the recalibrated Framingham and the Asian cohort equations was observed despite quantitative differences in the HRs for the associations between certain risk factors (total cholesterol and SBP) and cardiovascular events between these two populations, suggesting that such differences may not be of major practical importance in risk prediction. Other studies have similarly shown that recalibration of the Framingham equations improves the accuracy of risk prediction to an acceptable level in various populations, including in China; however, in all these earlier studies the data for recalibration were obtained from the same cohorts on which the risk prediction equation was subsequently tested.^5,23,26 To avoid what may be termed a “self-fulfilling prophecy”, we have instead used independent data for recalibration of the Framingham risk equation. Ideally, we would have used independent population-based cross-sectional data on risk factors, and reliable national mortality and morbidity data on the incidence of cardiovascular disease for China. Such data are not available for the time at which many of the APCSC Chinese cohort studies began. Thus, to maintain concurrence and independence, data from other Asian cohorts included in the APCSC were used to provide this information. Concurrence is important; if current estimates of levels of risk factors and incidence of cardiovascular disease in China were used, application of a recalibrated risk equation would surely result in major overestimation of cardiovascular risk in the APCSC Chinese cohorts as a result of the rapid increase in cardiovascular disease in China in recent years.

Apart from the absence of independent and concurrent population-based data from China with which to recalibrate the Framingham risk equation, there are other limitations to these analyses. The use of ICD codes (rather than independent adjudication) and differences in the ascertainment and definition of outcomes may have influenced the comparisons, although the effects of any misclassification are likely to be small given that all fatal and non-fatal cardiovascular events were included. The restricted number of variables used in the low-information approach could also be a limitation, when further key cardiovascular risk factors, such as diabetes status, are available for use in prediction. In such cases, the extra variables would have to be used more informally—for example, it has been suggested that a prior diagnosis of diabetes might be treated as equivalent to prior coronary disease in treatment algorithms.^27,28 Further limitations are the relatively few events and the relatively short duration of follow-up in the Chinese cohorts, particularly among women.

Using data from the long-established Framingham Study has several advantages, including the availability of standardised and adjudicated outcomes, and the potential to develop equations for different cardiovascular outcomes. A recalibrated low-information Framingham equation has potential application anywhere in Asia, provided the appropriate local data are available on average survival, free of cardiovascular disease, and mean values of risk factors. This has been illustrated with an example, using contemporary Thai data on risk factors and incidence of disease in Thailand. However, this example also highlights the limitations of such an approach. The Thai cohort study within the APCSC, EGAT, was based on an occupational cohort recruited in 1985. The prevalence data from this study are both outdated and probably biased because of the “healthy worker” effect. Thus, we used an alternative, contemporary, national sample survey. However, the only data available on survival in Thailand were from the EGAT, and its survival rates have questionable generalisability. These are important problems when cohort studies are used to estimate prevalence and to recalibrate survival. Few cohort studies, even in resource-rich countries, use truly nationally representative baseline samples. Cohort studies are almost always “one-off”, whereas administrative systems are designed to be perpetual. Reliable and ongoing national sample surveys on health and surveillance of mortality and morbidity are needed. Although periodic national health surveys are fairly common, surveillance is incomplete or lacking in many Asian countries.

In conclusion, a recalibrated Framingham cardiovascular risk prediction equation performs as accurately as a similar equation developed from Asian cohorts among selected populations from China. Importantly, we have found that low-information equations, including few variables that could be relatively inexpensively measured in low-resource settings, estimated cardiovascular risk in this population with good accuracy. We describe a low-information Framingham cardiovascular risk prediction equation that, with appropriate recalibration to local incidence of cardiovascular disease in particular, is likely to be a reliable tool for the assessment of cardiovascular risk in Asian populations. Although local data are always useful, these findings suggest that establishing costly new cohorts that take many years to mature sufficiently in order to specifically provide country-specific risk prediction equations may not be a priority; rather, developing systems for reliable mortality, morbidity and risk factor surveillance for many of these countries may be a better investment of limited health research funds.