Article Text

Download PDFPDF

EULAR standardised operating procedures for the elaboration, evaluation, dissemination, and implementation of recommendations endorsed by the EULAR standing committees
  1. M Dougados1,
  2. N Betteridge2,
  3. G R Burmester3,
  4. L Euller-Ziegler4,
  5. F Guillemin5,
  6. J Hirvonen6,
  7. J Lloyd7,
  8. S Ozen8,
  9. J A P Da Silva9,
  10. P Emery10,
  11. J R Kalden11,
  12. T Kvien12,
  13. M Matucci-Cerinic13,
  14. J Smolen14
  1. 1Clinique de Rhumatologie B Groupe, Hospitalier Cochin, Paris 75014, France
  2. 2Arthritis Care, 18 Stephenson Way, London NW1 2HD, UK
  3. 3Department of Rheumatology and Clinical Immunology, Charite Humboldt University, Schumanstrasse 20–21, D 10117 Berlin, Germany
  4. 4Service de Rheumatologie, Archet Hospital, route de Saint-Antoine Ginestiere, 06202 Nice, Cedex 3, France
  5. 5Ecole de Santé Publique, Faculté de Médecine, BP 184, 54505 Vandoeuvre les Nancy, France
  6. 6Suomen Reumalittory, Iso Roobertinkatu 20–22A, Helsinki FIN-01200, Finland
  7. 7Brunel University, Department of Health Studies, Borough Rd, Isleworth TW7 5DU, UK
  8. 8Hacettepe University, Department of Paediatrics, 06100 Ankara, Turkey
  9. 9Nucleo de Reumatologia, Hospitalais de Universidade de Coimbra, Servicio de Medicina III et Reumatologia, 3000-075 Coimbra, Portugal
  10. 10Academic Unit of Musculoskeletal Diseases, Department of Rheumatology, 1st Floor, Leeds LS1 3EX, UK
  11. 11Institut for Clinical Immun & Rheumatism, University Erlangen-Nurenberg, Krakenhaustrasse 2, Erlangen 91054, Germany
  12. 12Oslo City Department of Rheumatology, Diakonhjemmet Hospital, PO Box 23 Vinderen, Oslo N 0319, Norway
  13. 13Department Medicine/Division of Rheumatology, University of Florence, Villa Monna Tessa, Viale Pleraccini 18, Florence 50139, Italy
  14. 14Aligemeines Krankenhaus, Klinik für Innere Medizin II, Klinische Abteilung für Rheumatologie, Wahringe Gürtel 18–20, Vienna 1090, Austria
  1. Correspondence to:
    Professor M Dougados

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

It is the objective of the EULAR executive committee to promote actions and/or projects aimed at improving the knowledge and/or the recognition of musculoskeletal disorders.

The chief aim is to contribute to the improvement of outcome of patients with rheumatic disorders. Apart from the projects devoted to education and research, projects aimed at facilitating the conduct of clinical studies or at improving the management of musculoskeletal disorders are welcome. Such studies can be categorised in four sections:

  • Studies dedicated to a proposal for classification and/or diagnostic criteria

  • Recommendations for designing and/or conducting clinical trials in specific musculoskeletal disorders

  • Recommendations for monitoring and/or management and/or treatment of specific musculoskeletal disorders

  • Standardisation of (laboratory and other) procedures.


It is the objective of the EULAR executive committee to maintain and to homogenise a high level of intrinsic quality and comparability of such studies.

To achieve such an objective it appeared that the definition and publication of standardised procedures for the elaboration, evaluation, dissemination, and implementation of recommendations might be a relevant and useful starting point.

Obviously these standardised operating procedures should not be a barrier to acceptance of a project if not all points are satisfied but might be important to consider before starting a project.

These recommendations are not mandatory in themselves but can be used flexibly.


At each step of such projects (application, elaboration, dissemination, etc), the individual items summarised in table 1 should be discussed. The reader is invited to visit the EULAR website ( to check the most recently updated version of these procedures.

A Which wording?

The three proposals are “points to consider”, “recommendations”, “guidelines”.

It is the opinion of the EULAR executive committee members that, on the one hand, “guidelines” might appear too constraining and that, on the other hand, “points to consider” might be considered as too open. Recommendations can be considered as advice for performing the task/action, when applicable, as a marker of quality.

The choice of wording should be based on the content of each project. For example, one can suggest that if an evidence based approach fully answers the question, “guidelines” can be proposed, but if an approach fails to reach any conclusion, “points to consider” should be preferred.

B Which category?

As previously mentioned, it is expected that these procedures could be applied to studies proposing:

  • Recommendations for conducting clinical studies and/or clinical trials

  • Recommendations for management, monitoring, or treatment in daily practice

  • Recommendations for standardisation of other procedures.

C Objectives

The objectives of the project should be made clear from the beginning.

The definition of the target population (the population interested in such project) will facilitate the different steps of the project from its design to its implementation.

For example, a target population might be defined as:

  • Rheumatologists

  • General practitioners

  • Health professionals

  • National and/or international drug agencies

  • Drug companies

  • Others.

In practice, one single project can have several target populations. However, the presentation, dissemination, evaluation, and implementation may be different for each subcategory of the target population.

D Steering group members

Four categories of people will have to be included in each project.

1 The convenor of the project

This person (one person), preferably not a member of the EULAR executive committee, will be the link between the project group and the EULAR organisation.

It is expected that such a convenor will be also the chairperson of the project because he/she has a high level of experience in the field of interest of the project.

2 The experts

Such people should be representatives of the European rheumatological community.

They should come from at least three different European countries. However, top leaders in the field, including non-rheumatologists, should be invited even if they come from outside the European community.

It is expected that these experts have an academic position, but they can also have other positions and be invited as experts because of their high level of expertise in this field (for example, national agencies and/or drug company representatives).

3 A clinical epidemiologist

To promote a high level of quality and homogeneity of methodological issues in all projects, each study steering group should include an expert in clinical epidemiology (preferably not a member of the EULAR executive committee). If needed, the chairman of the EULAR Standing Committee on Epidemiology or the chairman of ECSICIT will help in identifying such experts.

The clinical epidemiologist will attend at least the first meeting of the steering committee members and will be in charge of following up the project during its different steps.

4 The person in charge of the literature research

It is expected that for most projects a systematic literature research will be mandatory.

This research can be performed by a person outside the steering group under the supervision of the convenor and/or a designated member of the steering committee.

E Evidence based approach

1 Literature search strategy

If applicable, such strategy should include two parts, which have to be decided before performing the literature search.

a) Selection of specific modalities

For a specific project (for example, management of knee osteoarthritis), it has to be decided whether the literature search will be focused on specific domains (for example, intra-articular injections of steroids for knee osteoarthritis) or will be completely open.

This decision will permit keywords to be clearly defined, which will be the starting point of the literature research.

b) Techniques of the literature search

Each project should clearly describe the different databases explored in the literature search (for example, Medline, PubMed, etc). It is strongly recommended that the Cochrane Library is included in every search for treatments.

2 Quality scoring of the manuscripts

The methodology used for scoring the different evaluated manuscripts (quantity, quality) has to be precisely described.

The following practical decision can be expected:

  • To describe only the number of evaluated manuscripts

  • To categorise each evaluated manuscript (for example, placebo randomised controlled trial, randomised controlled trial, prospective versus retrospective, etc)

  • To score each evaluated manuscript. For this purpose, several scoring systems have been proposed—in particular, for evaluation of the report of therapeutic trials. The choice of scoring system should take into account the nature of the project (in particular, pharmacological versus non-pharmacological treatment modalities). A reference to a specific scoring system is given in Appendix 1.

3 Estimation of the relevance of the evaluated item

We can anticipate four situations.

a) Evaluation of treatment modalities

To obtain an objective evaluation of different treatment modalities one might consider it of interest to quantify treatment effects.

For example, in knee osteoarthritis, the decision has been taken to focus on a single variable (pain) and thereafter to present the results as either effect size (for the continuous variables) or number needed to treat (for the dichotomous variables). If the dichotomous variable is chosen (for example, responder yes/no, success yes/no) and if the domain (for example, pain) has been evaluated by using a continuous variable (for example, change in a 0–100 mm visual analogue scale), a cut off point has to be decided a priori (for example, improvement of at least 30% in pain will be defined as a success).

b) Evaluation of outcome variables

In a project aimed at proposing recommendations on the design and conduct of clinical trials in a specific musculoskeletal disease, one might consider it interesting to evaluate the performance of different proposed outcome measures (for example, face validity, reliability, sensitivity to change, and discriminant capacity, etc).

c) Evaluation of a proposed criterion

In a project aimed at proposing classification criteria (for example, definition of the disease at entry into a clinical trial), one might consider it important to evaluate also the performance of a set of criteria (for example, sensitivity, specificity, pre- and post-test probability).

d) Recommendations

It is strongly suggested that recommendation should only be made on the basis of homogeneous and quantifiable information.

4 Categorising evidence

Categorising evidence has been clearly defined for the treatment modalities. Table 2 summarises these categories.

Table 1 Points to consider for the application of a study focused on recommendations in order to obtain EULAR endorsement

Table 2

 Categories of evidence

This categorisation has to be given for each recommendation of the treatment modality

5 Strength of recommendations

The strength of recommendations is clearly defined for the important factors of the treatment modalities. Table 3 summarises these categories.

Table 3

 Strength of recommendations

The main difference between “categories of evidence” and “strength of recommendations” is that the category of evidence is only based on a systematic literature research and the strength of recommendations also takes into account the knowledge of the experts. The strategy permitting use of the category “strength of recommendations” should be clearly described in the project (for example, vote of the experts after getting the results of the literature research).

F Expert opinion approach

It is admitted that publication of the evidence based approach alone may be too complicated to be fully used by the target population. For example, interpretation of the effect size of a treatment modality and/or the κ coefficient for the reliability of an outcome measure requires a specific knowledge.

Thus to make it clear, the recommendations may include summary statements from the experts based on the reported evidence or personal experience. Such expert opinion should appear only after or in parallel to an evidence based approach but never alone. Contents derived from expert opinion should be clearly identified, together with the reasons for that approach.

G Presentation of recommendations

1 Example of recommendations for management/monitoring/treatment-specific disorders

Two main categories have been proposed:

  • Presenting the results using an algorithm (tree decision)

  • Presenting the results using different short sentences (bullets, take home messages).

Whatever the decision, dissemination of the recommendations will be highly facilitated if the presentation is as simple as possible.

2 Example of recommendations for conducting clinical studies

It is recommended that the structure of the protocol should be followed and the following points discussed if relevant to the project.

a) Inclusion and exclusion criteria
  • Definition of the disease (which set of criteria?).

  • Definition of the activity of the disease (which set of criteria?).

  • Definition of the severity of the disease (which set of criteria?).

  • Demographics (age, sex, etc).

  • Concomitant treatments (allowed, prohibited, washout period before entry, etc).

  • Concomitant disorders.

b) Outcome measures
  • Recommended primary outcome measure.

  • Detailed list of recommended outcome measures.

  • Time to collection.

c) Sample size calculation
  • The known (or unknown) expected placebo (and/or conventional treatment) effect.

  • The clinically relevant expected treatment effect (differences between the study treatment and the control treatment).

H Relevance of the recommendations

  • If applicable, this evaluation has to be planned from the beginning of the project.

  • An “external” evaluation can be easily performed according to the AGREE instrument. A paper version of this instrument is available on demand for current projects at the ECSICIT secretariat. Such an instrument is also available on the web at

  • The evaluation can also be performed at the level of the target population. For example, one survey can be performed to check whether the proposed recommendations of a treatment are in accordance with the daily practice of the target population (for example, general practitioners). For the conduct of clinical studies, these recommendations can be presented and discussed at a meeting to which health agencies and drug company representatives are invited.

  • The evaluation should also deal with the potential use of the proposed recommendations for teaching rheumatology (medical schools, postgraduate training, health professional schools, etc).

  • Such evaluation could be developed as a specific project.

I Dissemination of the recommendations

Strategies for disseminating the proposed recommendation to the target population should be included in the project. Whatever the project, the steering group is expected to:

  • Submit an abstract for presentation at the annual EULAR scientific meeting

  • Submit a manuscript for publication in the EULAR journal.

J Implementation of the recommendations

It is the goal of recommendations to change practices and allow them to converge towards harmonisation. Implementation is the process by which targeted users (researchers or clinicians) integrate the actions recommended into their practice. Efficient implementation leads to successful changes.

However, it is worth assessing the potential impact of the proposed recommendations in daily practice because this is dependent on the degree of implementation. On the one hand, one might consider that any project aimed at either evaluating and/or disseminating the proposed recommendations will have an impact on daily practice. On the other hand, some studies suggest that the dissemination of recommendations is not sufficient to achieve such an impact. Several other techniques have been proposed:

  • Opinion leaders

  • Outcome visits or academic detailing

  • Audit feedback

  • Continuing medical education

  • Reminder (paper print reminders, electronic reminders, phone call reminders).

Detailed information about the efficacy of the different techniques (including references related to the above techniques) in implementing the medical recommendations is available (in French but with references in English) on the web at the following address:

Such an evaluation might be developed as a specific project.

K Update policy of the recommendations

During the process of producing recommendations, one should expect to answer the following questions:

  • Do proponents expect an update of the proposed recommendations?

  • If yes, when? How? By whom?

L Practical and financial aspects of the project

1 Practical aspects

It is expected that each project will require the following:

a) Meeting(s) of the steering group members

For each project, the following information will be required at the start:

  • Number of planned meetings

  • Number of people who will attend such meeting(s)

  • A calendar of the planned meetings.

As soon as the project is endorsed by EULAR (see below), the secretariat of EULAR will take care of the practical aspects of these meetings (hotel reservation, meeting room reservation, travel expenses).

b) Fellowship for the literature research

If possible, this fellow has to be designated before the first meeting of the steering group. The amount of time (full versus part time) and the duration of his/her work has to be proposed.

1 Financial aspects

The budget of each project will include:

  • Organisation of the meetings of the steering group (hotel reservation, travel expenses)

  • A grant for the fellow in charge of the literature research, if necessary

  • A grant for the statistical analysis, if necessary

  • A grant for the secretariat of the steering committee, if necessary.

There will be no honorarium for participation of the experts.

M EULAR endorsement policy

Endorsement by EULAR should be sought according to the EULAR procedures (see the EULAR website). In summary, the first steps are to get such an endorsement after submission of an application to the appropriate EULAR standing committee taking into account the different points summarised in table 1.

This EULAR endorsement will permit the project to start but could still be cancelled if there are any deviations from the present procedure during the different steps of the project. For examination of possible deviations the clinical epidemiologist in charge of the project will have to send to the chairman of the appropriate standing committee a report after each meeting of the steering committee. Moreover, the material (abstract, manuscript) planned for publication should receive the approval of the chairman of the appropriate standing committee before any submission is made.


Table 4 shows an example of a scoring system of a manuscript summarising the results of a therapeutic trial.1

Table 4

 Scoring system