You are here

  • Home
  • Archive
  • Volume 2, Issue 1
  • When rheumatologists report that they agree with a guideline, does this mean that they practise the guideline in clinical practice? Results of the International Recommendation Implementation Study (IRIS)

Article Text

Article menu

Download PDFPDF

Treatment
Original article
When rheumatologists report that they agree with a guideline, does this mean that they practise the guideline in clinical practice? Results of the International Recommendation Implementation Study (IRIS)
  1. Emilia Gvozdenović1,
  2. Cornelia F Allaart1,
  3. Désirée van der Heijde1,
  4. Gianfranco Ferraccioli2,
  5. Josef S Smolen3,
  6. Tom W J Huizinga1 and
  7. Robert Landewé4
  1. 1Department of Rheumatology, Leiden University Medical Center, Leiden, The Netherlands
  2. 2Division of Rheumatology, on behalf of GISEA, Catholic University of the Sacred Heart, Rome, Italy
  3. 3Division of Rheumatology, Medical University of Vienna, Vienna, Austria
  4. 4Rheumatology Department, Academic Medical Center, Amsterdam & Atrium Medical Center, Heerlen, The Netherlands
  1. Correspondence to Dr Robert Landewé; landewe{at}rlandewe.nl

Abstract

Introduction The European League Against Rheumatism (EULAR) recommendations for the management of rheumatoid arthritis (RA) and the treat-to-target (T2T) principles have been developed in order to improve the treatment outcome of patients with RA, and have received broad attention. It is not clear, though, whether these recommendations are indeed followed up in clinical practice.

Objective To investigate if rheumatologists that report to agree with existing guidelines indeed follow them up in clinical practice.

Methods The International Recommendation Implementation Study (IRIS) included 132 participating rheumatologists from 14 countries. Participating rheumatologists received a questionnaire measuring their awareness/commitment with the EULAR/T2T recommendations and followed a dedicated educational programme. Subsequently, they were asked to enrol 5–10 patients with new-onset RA in the online IRIS database and monitor disease activity and treatment for a period of 1–2 years. Four recommendations (3 from the EULAR recommendations and one from the T2T recommendations) were selected on the basis of testability, and analysed with regard to compliance by participating rheumatologists.

Results In total, 72 of the 132 participating rheumatologists contributed 378 patients to the database. Of these participants, 70 (98%) agreed upfront with the recommendation that disease-modifying antirheumatic drug (DMARD) therapy should be started as soon as possible after diagnosis in every patient; 69 (96%) of the rheumatologists agreed with the recommendation that methotrexate (MTX) should be part of the first treatment strategy. When measuring the actual performance, it was found that the recommendation on early DMARD start was met in 253 (67%) of the recorded patients, and the recommendation on MTX in 225 (60%) of the recorded patients. Of the participants, 60 (83%) agreed that composite measures should be recorded regularly, but only in 134(54%) of the patients were composite scores actually recorded in ≥50% of patient visits.

Conclusion Reporting to be compliant with EULAR recommendations and T2T principles, even after dedicated education does not mean they actually comply with it in clinical practice.

  • DMARDs (synthetic)
  • Rheumatoid Arthritis
  • Treatment
  • Disease Activity
  • DMARDs (biologic)

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

https://doi.org/10.1136/rmdopen-2015-000221

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Key messages

    What is already known about this subject?

    • Rheumatologists report following guidelines in daily practice.

    • Some studies showed successful implementation of guidelines in clinical practice.

    What does this study add?

    • In our study, we aim to investigate whether rheumatologists who report following guidelines really do this in clinical practice.

    How might this impact on clinical practice?

    • This study may increase awareness of and improve adherence to guidelines and possibly also enhance implementation of guidelines in clinical practice.

    Introduction

    Over the past decades, rheumatoid arthritis (RA) guidelines and recommendations have been formulated with the aim of improving the quality of care.1–5 In rheumatology, international recommendations for the management of patients with RA include guidance about treatment decisions, such as the choice of initial and subsequent therapy, about monitoring disease activity as a measure of treatment success and about treatment adjustments. In addition, they emphasise the principles of shared decision-making, patient education, how to deal with comorbidities and the role of specialised nurses in the treatment care of RA.

    These RA recommendations include concepts of ‘treating-to-target’ (further referred to as T2T) and ‘tight control’. The T2T approach requires that antirheumatic therapy in a patient should be chosen and adjusted in such a manner that clinical remission or low disease activity is achieved. The ‘tight control’ concept requires frequent assessments of disease activity in order to check the treatment goals and to avoid delays in optimal treatment. It is recommended that monitoring disease activity should be done by composite measures (DAS, DAS28, SDAI and CDAI).6–8

    Surveys among rheumatologists have suggested that rheumatologists follow the recommendations for RA in clinical practice.9–11 Other studies, however, have suggested that recommendations are hardly practised outside clinical trials.12 ,13 These studies indicate that there is a discrepancy between reporting agreement with recommendations and the actual performance in clinical practice (implementation). Only a few studies have shown a successful implementation of recommendations, such as T2T and DAS steered therapy in clinical practice.14 ,15 Among the obstacles that may hinder successful implementation of recommendations are lack of awareness and lack of agreement,16–18 as well as the absence of proper treatment protocols.15 A previous study has suggested that dedicated educational programmes may help to implement clinical guidelines in practice.19

    To test if such an educational implementation initiative may improve practice performance, the International Recommendation Implementation Study (IRIS) has been initiated. The first step of this study was to investigate if rheumatologists that report complying with certain European League Against Rheumatism (EULAR) recommendations on the management of RA and/or the T2T recommendations for RA actually do so when measured in clinical practice. In order to increase the likelihood of success, the participants had been offered a dedicated web-delivered educational programme about the recommendations before the start of the study.

    Methods

    IRIS is an implementation study with a 2-year follow-up. From December 2011, rheumatologists worldwide were approached via their national societies to participate in the study. One hundred and thirty two rheumatologists from the following 14 countries (Bosnia, Brazil, Croatia, Cyprus, Greece, Italy, Malta, the Netherlands, Nigeria, Poland, Portugal, Russia, Spain and Turkey) agreed to participate.

    Participants received a questionnaire with questions about the awareness of: (1) agreement with (2) and adherence to (3) the EULAR and T2T recommendations formulated in 2010 (see online supplementary attachments I and II).6 ,8 Four recommendations were preselected by us for further testing, based on their appropriateness for measuring awareness (1) and agreement (2), but in addition also on the appropriateness of actually measuring performance in clinical practice (3) (database).

    Supplemental material

    The selected recommendations were:

    • ‘Treatment with synthetic disease-modifying antirheumatic drugs (DMARDs) should be started as soon as the diagnosis of RA is made’

    • ‘Methotrexate (MTX) is part of the first treatment strategy in patients with active RA’

    • ‘When MTX contraindications (or intolerance) are present, the following DMARDs should be used: leflunomide, sulfasalazine or injectable gold’

    • ‘Measures of disease activity must be obtained and documented regularly as frequently as monthly for patients with high/moderate disease activity or less frequently (such as every 3–6 months) for patients in sustained low disease activity or remission’

    Participating rheumatologists took part in an educational programme, that included (1) the appraisal of the ‘EULAR recommendations for the management of RA’ article and the ‘T2T recommendations’ article; and (2) watching an online video in which the recommendations and the aims of IRIS were further explained and elucidated by expert rheumatologists and researchers. In addition, participants could follow an online training programme about the Measurement of efficacy of Treatment in the Era of Outcome of Rheumatology (METEOR) registration tool.

    Next, participants were required to enrol 5–10 patients with newly diagnosed RA, and to record disease activity assessments and treatment adjustments as often as they thought this would be appropriate. Treatment choice and monitoring frequency was entirely at the discretion of the participating rheumatologist. Registration was performed in the METEOR database,20 ,21 a combination of a database and an online tool to register data from patients with RA and monitor them in daily practice. The patients were followed for 1–2 years. During follow-up, participating rheumatologists received one of the EULAR and/or T2T recommendations per email every week as a reminder to encourage them to comply with the recommendations in clinical practice.

    The first participant started with the educational programme in March 2012 and the final participant started in February 2014. The database is still ongoing and will close 2 years after the last participant has added the last patient (February 2016). The participants received a payment of €250 per included patient to compensate for the work in this study.

    Analysis

    The primary analysis was a comparison between the rate of agreement with each of the four selected recommendations and the rate of actual compliance with them when measured in the METEOR database. The recommendations were operationalised as follows:

    • EULAR recommendation 1: ‘Treatment with synthetic DMARDs should be started as soon as the diagnosis of RA is made’; was operationalised as: The proportion of patients in whom the time interval between the date of diagnosis and the date of start of a DMARD (disease modifying anti-rheumatic drug) was ≤4 weeks.

    • EULAR recommendation 2: ‘MTX is part of the first treatment strategy in patients with active RA’; was operationalised as: The proportion of RA patients in whom MTX was (part of) the first treatment strategy.

    • EULAR recommendation 3: ‘When MTX contraindications (or intolerance) are present, the following DMARDs should be used: Leflunomide, sulfasalazine or injectable gold’; was operationalised as: The proportion of patients who did not start with MTX in whom leflunomide, sulfasalazine or injectable gold was prescribed as an alternative.

    • Treat to target recommendation 5: ‘Measures of disease activity must be obtained and documented regularly as frequently as monthly for patients with high/moderate disease activity or less frequently (such as every 3–6 months) for patients in sustained low disease activity or remission’. Three categories were formed: (1) ‘T2T always’ represents the number of patients for whom a composite score (DAS, DAS28, CDAI or SDAI) was reported at least every 2 months if moderate- or high disease activity (DAS>2.4, DAS28>3.6 CDAI>10, SDAI>11) was present; and at least every 7 months when low disease activity was measured in 100% of the visits; (2) ‘T2T sometimes/never’ when ‘T2T always’ was not met; and (3). ‘not reported’ when composite scores were missing in all of the visits.

    Results

    Of the 132 participating rheumatologists who agreed to participate in IRIS, 122 (92%) completed the questionnaire on awareness and agreement, took part in the web-based educational programme, and agreed to start in the patient enrolment programme.

    During the follow-up period, 72 (55%) of the participating rheumatologists recorded 1.155 visits of 378 newly diagnosed patients in the database prospectively. The other participants declined participation (n=44) or were lost to contact (n=6) before including any patient in the METEOR database (figure 1). Reported reasons for declining were ‘lack of time’ and ‘consent withdrawn’. The remaining 72 rheumatologists were from eight countries (Bosnia, Cyprus, Greece, Italy, the Netherlands, Nigeria, Russia and Spain). We compared the results of the questionnaire from 72 rheumatologists who entered patients in the database with the 50 rheumatologists who did not, and found that agreement was similar between the two groups (see online supplementary attachments I and II).

    Figure 1
    Figure 1

    Flow-chart.

    Baseline characteristics of the participating patients were described in online supplementary attachment III table 1. Eight two per cent (n=300) of the patients were women. Mean (SD) age was 55 years and mean DAS was 3.1. Median time from diagnosis until first visit was 8 weeks.

    View this table:
    Table 1

    Comparison between reporting to follow the EULAR and T2T recommendations and managing patients in clinical practice

    1. ‘Early treatment start’

    Of the 72 participating rheumatologists, 70 (98%) had reported complying with the recommendation ‘Treatment with synthetic DMARDs should be started as soon as the diagnosis of RA is made’ (table 1). In 253 of the 378 (67%) patients who had been recorded in the database, a synthetic DMARD had indeed been prescribed within 4 weeks after the diagnosis.

    In 65 of the 378 patients (17%), a DMARD was not started within 4 weeks. 55 of them received a DMARD in a later period. The median (IQR) duration until these patients received a DMARD was 15 (5–81) weeks (table 2).

    View this table:
    Table 2

    Average time from diagnosis (weeks) until first DMARD in patients in whom a DMARD was NOT started within 4 weeks after diagnosis

    For 60 of the 378 patients (16%), essential information was missing (in these cases the date of diagnosis or the start date of the first DMARD was missing).

    2. ‘Start with MTX’

    Of the 72 participating rheumatologists, 69 (96%) had reported complying with the recommendation ‘MTX is part of the first treatment strategy in patients with active RA’. In 225 of the 378 patients (60%), MTX has indeed been prescribed as (part of) the first treatment.

    3. ‘Start with alternatives for MTX’

    Of the 93 patients (26%) who did not start with MTX,15 (19%) received leflunomide, sulfasalazine or injectable gold as first treatment. However, 78 patients (81%) started with other medications than these three preferred alternatives. Six of these patients started with biologicals (table 3).

    View this table:
    Table 3

    Medication prescribed as initial treatment and average time in weeks from diagnosis until start in patients in whom MTX, leflunomide, sulfasalazine or injectable gold was NOT started as first DMARD

    4. ‘Frequent monitoring’

    Of the participating 72 rheumatologists, 60 (83%) had reported compliance with the recommendation: ‘Measures of disease activity must be obtained and documented regularly as frequently as monthly for patients with high/moderate disease activity or less frequently (such as every 3–6 months) for patients in sustained low disease activity or remission’.

    Of the 378 patients, 131 had less than two visits available in the database. For those patients with more than one visit recorded (247), we checked whether patients had been monitored according to this recommendation. Of these 247 patients, only 68 (27%) had been monitored in full compliance with this recommendation (‘T2T always’). Of the remaining patients, 23 (9%) had been monitored in partial compliance with this recommendation (75–100% of the visits); 45 (18%) had been monitored insufficiently (in 50–75% of the visits); 27 (11%) had been poorly monitored (<50% of the visits). Finally, 30 (13%) had not been monitored at all. All these remaining patients were categorised in table 1 under the heading ‘Sometimes/never applied’.

    Discussion

    The main conclusion of this study among rheumatologists practising in different parts of the world is that reporting compliance with EULAR/T2T recommendations does not necessarily mean that these recommendations are actually applied in daily clinical practice. We have found discrepancies between what rheumatologists report doing versus how they actually treat and follow patients in clinical practice.

    Less than 60% of the recruited rheumatologists who agreed to participate, expectedly the more dedicated rheumatologists, finished the educational programme and included patients in the METEOR database, which is rather disappointing. Yet, even after participating in a dedicated educational programme, in which the participants were actively stimulated to follow their patients according to the recommendations, a strategy that has proven positive effects on implementation in many previous studies,19 ,22–24 rheumatologists still seem to be reluctant to practise that recommendation in real life and to register their performance. In fact, it is even worse: They report that they comply with recommendations but act differently in clinical practice.

    What could explain this discrepancy? First, trivial logistic explanations may account. For instance, a patient may not show up for a visit, which in turn could lead to missing disease activity data within the recommended time period. Furthermore, it may need time to fill the gap between obtaining knowledge about the need for recommendations and actually practising them. Rheumatologists may agree with recommendations and be convinced by educational programmes, but still need more time than 1–2 years to actually change their practice. There is some evidence for this statement: In the study by Forsetlund et al,25 physicians who were actively stimulated to treat patients according to evidence-based practice were compared with physicians who only received access to evidence-based libraries, but no significant differences between the group in behaviour of decision-making was found. Their follow-up was 1.5 years, which was argued to be too short to change decision-making among physicians. A Dutch study also showed similar discrepancies between compliance with and actual application of recommendations about mental health in practice. In this study, no educational programme was used to encourage physicians to follow the recommendations.26

    A more technical explanation for not following the recommendations in clinical practice is that we had to base our verdict about whether a recommendation was followed or not on the registration of rheumatologists' actions in the METEOR database. It is theoretically possible that recommendations were followed better than were actually recorded. However, all rheumatologists were explicitly informed about the procedures, as well as instructed to register their performance in the METEOR database and offered a training programme to optimally use that database. In addition, since we are not sure whether the participating rheumatologists have truly completed the offered educational programme, and we are not informed about the performance of rheumatologists without following the education, we cannot conclude from this analysis that the programme has influenced the behaviour of the rheumatologists. We did send out monthly emails with a recommendation in order to remind the rheumatologist on the project, which has been shown to be an effective tool in previous implementation studies,27 and may also have influenced performance. We have offered the educational programme via the internet, and a more effective approach could have been telephone interviews or meetings with physicians. Reviews have suggested that multifaceted strategies, such as educational meetings, educational resources and support from colleagues, are more successful to assist in implementation of recommendations,27–30 although another review has suggested that it is not clear yet which implementation strategies are best.31

    An intrinsic reluctance to record daily practice in a database, although being user-friendly,20 may explain why only 72 (55%) of the rheumatologists who followed the educational programme succeeded in including patients. When we compared the rates of agreement with the recommendations between the rheumatologists who dropped out after the educational programme (n=50) with the 72 participants who effectively included patients, the responses were similar, thus excluding spurious selections of participants as an explanation for disappointing results.

    Finally, technical data entry problems with the METEOR database may also have contributed to incompleteness of data. In 16–22% of patient data, information about the four selected recommendations was not reported. It is unclear if these patients are randomly missing, and if entered data are inconsistent. In addition, we can only speculate about the reasons for not complying with recommendations, as additional data on treatment steps, contraindications for medication, side effects and comorbidities are not recorded.

    This study has strengths and limitations. An important strength of this study is that we have recruited study participants from all over the world, which increases generalisability. There are also limitations to this study. First of all, we have investigated the agreement and adherence to the EULAR recommendations of 2010, which were new at the time of study initiation but have been updated since then (online publication in October 2013, while inclusion and instruction in the current study was finished in March 2013). However, the updated recommendations did not differ much with respect to the four recommendations studied. The only difference that is relevant for the interpretation of this study is that injectable gold is not recommended anymore when MTX is contraindicated.

    Another limitation is that we miss information about characteristics of the participating rheumatologists, due to privacy reasons. While rheumatologists from all over the world have participated in the study, it is still uncertain whether the study is fully generalisable to all rheumatologists.

    In conclusion, the results of this study show that there is a discrepancy between reporting agreement with well-known and broadly accepted recommendations on treating-to-target, timing and choice of initial treatment as well as tight control in the management of patients with RA on the one hand, and the actual performance in clinical practice when measured on the other. A dedicated internet-based educational programme is most likely insufficient to change the attitude of the rheumatologists in this regard, at least in the short term. This observation has implications for the broadly advocated recommendation to implement quality-control initiatives in order to make practice performance more transparent: It looks as if the development and publication of evidence-based and consensus-based treatment recommendations do not suffice to change practice performance in rheumatology. Since these recommendations are usually a trade-off between best evidence and cost-effectiveness, it can be argued if nowadays patients with RA are indeed optimally treated in clinical practice. Further studies should focus on factors that explain the reluctance of rheumatologists to follow evidence-based treatment recommendations, in particular reluctance of MTX prescription in patients with comorbidities, and on strategies to overcome this reluctance. In addition, future studies should focus on investigating what type of education is most effective for implementing guidelines in clinical practice.

    References

    1. ↵
      1. Grimshaw JM,
      2. Russell IT
      . Effect of clinical guidelines on medical practice: a systematic review of rigorous evaluations. Lancet 1993;342:1317–22. doi:10.1016/0140-6736(93)92244-N
      OpenUrlCrossRefPubMedWeb of Science
    2. ↵
      1. Woolf SH,
      2. Grol R,
      3. Hutchinson A, et al
      . Clinical guidelines: potential benefits, limitations, and harms of clinical guidelines. BMJ 1999;318:527–30. doi:10.1136/bmj.318.7182.527
      OpenUrlFREE Full Text
    3. ↵
      1. Amiel SA,
      2. Pursey N,
      3. Higgins B, et al
      . Diagnosis and management of type 1 diabetes in adults: summary of updated NICE guidance. BMJ 2015;351:h4188. doi:10.1136/bmj.h4188
      OpenUrlFREE Full Text
    4. ↵
      1. Gruffydd-Jones K,
      2. Jones MM
      . NICE guidelines for chronic obstructive pulmonary disease: implications for primary care. Br J Gen Pract 2011;61:91–2. doi:10.3399/bjgp11X556182
      OpenUrlFREE Full Text
    5. ↵
      1. Cressy DS,
      2. DeBoisblanc BP
      . Diagnosis and management of asthma: a summary of The National Asthma Education and Prevention Program guidelines. National Institutes of Health. J La State Med Soc 1998;150:611–17.
      OpenUrlPubMed
    6. ↵
      1. Smolen JS,
      2. Landewe R,
      3. Breedveld FC, et al
      . EULAR recommendations for the management of rheumatoid arthritis with synthetic and biological disease-modifying antirheumatic drugs. Ann Rheum Dis 2010;69:964–75. doi:10.1136/ard.2009.126532
      OpenUrlAbstract/FREE Full Text
    7. ↵
      1. Smolen JS,
      2. Landewe R,
      3. Breedveld FC, et al
      . EULAR recommendations for the management of rheumatoid arthritis with synthetic and biological disease-modifying antirheumatic drugs: 2013 update. Ann Rheum Dis 2014;73:492–509. doi:10.1136/annrheumdis-2013-204573
      OpenUrlAbstract/FREE Full Text
    8. ↵
      1. Smolen JS,
      2. Aletaha D,
      3. Bijlsma JW, et al
      . Treating rheumatoid arthritis to target: recommendations of an international task force. Ann Rheum Dis 2010;69:631–7. doi:10.1136/ard.2009.123919
      OpenUrlAbstract/FREE Full Text
    9. ↵
      1. Haraoui B,
      2. Bensen W,
      3. Bessette L, et al
      . Treating rheumatoid arthritis to target: a Canadian physician survey. J Rheumatol 2012;39:949–53. doi:10.3899/jrheum.111134
      OpenUrlAbstract/FREE Full Text
    10. ↵
      1. Park YB,
      2. Koh EM,
      3. Kim HY, et al
      . Treating rheumatoid arthritis to target: recommendations assessment questionnaire in Korea. Clin Rheumatol 2013;32:1791–7. doi:10.1007/s10067-013-2365-5
      OpenUrl
    11. ↵
      1. Schoels M,
      2. Aletaha D,
      3. Smolen JS, et al
      . Follow-up standards and treatment targets in rheumatoid arthritis: results of a questionnaire at the EULAR 2008. Ann Rheum Dis 2010;69:575–8. doi:10.1136/ard.2009.108472
      OpenUrlAbstract/FREE Full Text
    12. ↵
      1. Kruger K,
      2. Karberg K
      . [Treat-to-target from the perspective of office-based rheumatology]. Z Rheumatol 2011;70:664–9. doi:10.1007/s00393-011-0852-0
      OpenUrlPubMed
    13. ↵
      1. Littlejohn G,
      2. Roberts L,
      3. Arnold M, et al
      . A multi-center, observational study shows high proportion of Australian rheumatoid arthritis patients have inadequate disease control. Int J Rheum Dis 2013;16:532–8. doi:10.1111/1756-185X.12163
      OpenUrlCrossRefPubMed
    14. ↵
      1. Vermeer M,
      2. Kuper HH,
      3. Hoekstra M, et al
      . Implementation of a treat-to-target strategy in very early rheumatoid arthritis: results of the Dutch Rheumatoid Arthritis Monitoring remission induction cohort study. Arthritis Rheum 2011;63:2865–72. doi:10.1002/art.30494
      OpenUrlCrossRefPubMedWeb of Science
    15. ↵
      1. van Hulst LT,
      2. Creemers MC,
      3. Fransen J, et al
      . How to improve DAS28 use in daily clinical practice?—a pilot study of a nurse-led intervention. Rheumatology (Oxford) 2010;49:741–8. doi:10.1093/rheumatology/kep407
      OpenUrlAbstract/FREE Full Text
    16. ↵
      1. Michie S
      . Changing behavior: theoretical development needs protocol adherence. Health Psychol 2005;24:439. doi:10.1037/0278-6133.24.4.439
      OpenUrlCrossRefPubMedWeb of Science
    17. ↵
      1. Rashidian A,
      2. Eccles MP,
      3. Russell I
      . Falling on stony ground? A qualitative study of implementation of clinical guidelines’ prescribing recommendations in primary care. Health Policy 2008;85:148–61.
      OpenUrlCrossRefPubMedWeb of Science
    18. ↵
      1. Cabana MD,
      2. Rand CS,
      3. Powe NR, et al
      . Why Don't physicians follow clinical practice guidelines? A framework for improvement. JAMA 1999;282:1458–65.
      OpenUrlCrossRefPubMedWeb of Science
    19. ↵
      1. Zwerver F,
      2. Schellart AJ,
      3. Knol DL, et al
      . An implementation strategy to improve the guideline adherence of insurance physicians: an experiment in a controlled setting. Implement Sci 2011;6:131. doi:10.1186/1748-5908-6-131
      OpenUrlPubMed
    20. ↵
      1. Koevoets R,
      2. Allaart CF,
      3. van der Heijde DM, et al
      . Disease activity monitoring in rheumatoid arthritis in daily practice: experiences with METEOR, a free online tool. J Rheumatol 2010;37:2632–3. doi:10.3899/jrheum.100552
      OpenUrlFREE Full Text
    21. ↵
      1. Gvozdenovic E,
      2. Koevoets R,
      3. Wolterbeek R, et al
      . Assessment of global disease activity in RA patients monitored in the METEOR database: the patient's versus the rheumatologist's opinion. Clin Rheumatol 2014;33:461–6. doi:10.1007/s10067-013-2390-4
      OpenUrl
    22. ↵
      1. Sanci LA,
      2. Coffey CM,
      3. Veit FC, et al
      . Effects of an educational intervention for general practitioners in adolescent health care principles: a randomized controlled study. West J Med 2000;172:157–63. doi:10.1136/ewjm.172.3.157
      OpenUrlPubMed
    23. ↵
      1. Davis D,
      2. O'Brien MA,
      3. Freemantle N, et al
      . Impact of formal continuing medical education: do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes? JAMA 1999;282:867–74. doi:10.1001/jama.282.9.867
      OpenUrlCrossRefPubMedWeb of Science
    24. ↵
      1. Smits PB,
      2. Verbeek JH,
      3. van Dijk FJ, et al
      . Evaluation of a postgraduate educational programme for occupational physicians on work rehabilitation guidelines for patients with low back pain. Occup Environ Med 2000;57:645–6. doi:10.1136/oem.57.9.645
      OpenUrlAbstract/FREE Full Text
    25. ↵
      1. Forsetlund L,
      2. Bradley P,
      3. Forsen L, et al
      . Randomised controlled trial of a theoretically grounded tailored intervention to diffuse evidence-based public health practice [ISRCTN23257060]. BMC Med Educ 2003;3:2. doi:10.1186/1472-6920-3-2
      OpenUrlCrossRefPubMed
    26. ↵
      1. Rebergen D,
      2. Hoenen J,
      3. Heinemans A, et al
      . Adherence to mental health guidelines by Dutch occupational physicians. Occup Med (Lond) 2006;56:461–8. doi:10.1093/occmed/kql042
      OpenUrlAbstract/FREE Full Text
    27. ↵
      1. Prior M,
      2. Guerin M,
      3. Grimmer-Somers K
      . The effectiveness of clinical guideline implementation strategies—a synthesis of systematic review findings. J Eval Clin Pract 2008;14:888–97. doi:10.1111/j.1365-2753.2008.01014.x
      OpenUrlCrossRefPubMedWeb of Science
    28. ↵
      1. Grimshaw JM,
      2. Thomas RE,
      3. MacLennan G, et al
      . Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess 2004;8:iii–iv, 1–72. doi:10.3310/hta8060
      OpenUrlPubMed
    29. ↵
      1. Hadely KA,
      2. Power E,
      3. O'Halloran R
      . Speech pathologists’ experiences with stroke clinical practice guidelines and the barriers and facilitators influencing their use: a national descriptive study. BMC Health Serv Res 2014;14:110. doi:10.1186/1472-6963-14-110
      OpenUrl
    30. ↵
      1. Medves J,
      2. Godfrey C,
      3. Turner C, et al
      . Systematic review of practice guideline dissemination and implementation strategies for healthcare teams and team-based practice. Int J Evid Based Healthc 2010;8:79–89.
      OpenUrlPubMed
    31. ↵
      1. Grol R,
      2. Grimshaw J
      . From best evidence to best practice: effective implementation of change in patients’ care. Lancet 2003;362:1225–30. doi:10.1016/S0140-6736(03)14546-1
      OpenUrlCrossRefPubMedWeb of Science

    Footnotes

    • Contributors EG carried out the analyses and was in charge of writing the article. CFA, RL, TWJH, GF, JSS and DvdH contributed to revising the manuscript and brainstorming about the research questions. All authors read the manuscript and gave their final approval for submitting the manuscript to RMD open.

    • Funding This work was financially supported by Pfizer. Pfizer had no role in the data collection, data analysis or writing of the manuscript and publication of this paper was not contingent on their approval.

    • Competing interests None declared.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data sharing statement No additional data are available.