Statistics from Altmetric.com
We would like to thank Reuter et al for their interest in our study and the constructive spirit in which their comments were offered.1 ,2 We agree with their concerns about inferring causality, which we have not done. Our study was designed to help provide coherence to the plethora of online discussions about disease-modifying antirheumatic drugs (DMARDs) but was not intended to imply causality. We also agree with their concerns regarding the veracity of the data collected from social media forums and have listed that as a limitation of our study.
Our search strategy was to include any discussions on DMARDs (thus we searched for all available conventional synthetic, biologic and targeted synthetic DMARDs) in the context of rheumatoid arthritis. Treato’s patented algorithm (which is available for review online—US8612455B2) is designed to detect a lived experience as opposed to posts from bots/spam. Our manual review of the data did not reveal any type of spam or advertising; however, we agree that without going through each of the billion posts that were screened, it would be impossible to ensure that none of the data that were included/excluded were entirely user generated.
We considered kappa values of 0.49 and 0.52 as ‘moderate agreement’ based on well-established parameters, which are widely used in clinical research.3 While a high kappa is desirable in clinical studies, the same standards cannot be applied when gauging online sentiment. Indeed, the kappa in such studies is moderate at best even between human reviewers, thus achieving a moderate kappa between human and algorithm is a strength of the study.4
Patients’ concerns were estimated based on the most commonly mentioned phrases/statements across thousands of conversations. These, we felt, would be reflective of the underlying sentiment/beliefs of the patients.
We agree with Reuter et al that cautious interpretation of data, innovation and an iterative approach to research methodology in studies of this kind are important in the ongoing development of this exciting new field of research.
Handling editor Josef S Smolen
Contributors CS wrote the response which was reviewed and amended by all the coauthors.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Patient consent for publication Not required.
Provenance and peer review Commissioned; internally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.