How evidence-based are advertisements in journalsregarding the subspecialty of rheumatology?
P. van Winkelen, J. S. van Denderen, C. Y. Vossen, T. W. J. Huizinga1 andF. W. Dekker for the SEDUCE study group
Objectives. In rheumatology, ﬁve different billion-dollar drugs have emerged in recent years, making this subspecialty the focusof extensive advertising campaigns. Considering this development and the fact that the scientiﬁc content of advertisements hasbeen questioned, we initiated a study to determine how evidence-based advertisements are in four leading journals ofrheumatology. Methods. Advertisements were extracted from the journals Arthritis and Rheumatism, Rheumatology, Seminars in Arthritisand Rheumatism and the Journal of Rheumatology, and were evaluated using a standardized score form. Results. In total, 353 advertisements were obtained, of which 84 were unique. Of the 300 references provided by these uniqueadvertisements, 53 (18%) were considered ‘supporting’. In addition, 87 (29%) of the 300 references referred to randomizedcontrolled trials (RCTs), of which 49% supported the claim. The vast majority of RCTs (97%) were sponsored by theadvertising company. In the 84 unique advertisements 288 claims were made, of which 190 provided one or more references. Ofthese 190 claims, 33 (17%) were judged ‘well supported’. In total, only four (5%) of the 84 different advertisements were judged‘well supported’ and 11 (13%) of the 84 were judged ‘misleading’ because they contained one or more misleading claims,i.e. claims with references to articles contradicting the claim. Conclusions. This study indicates that few advertisements in rheumatology journals are entirely evidence-based.
KEY WORDS: Advertisements, Evidence-based, Rheumatology.
Medical journals are the major source of information for doctors,
claim more impressive, so other researchers focused on the
through both peer-reviewed articles and advertisements. The
references accompanying the claims. Cooper described the kinds
scientiﬁc content of advertisements, trying to meet commercial
of documents cited in support of claims and assessed the
and scientiﬁc standards at the same time, has been questioned
availability of these documents . Lankinen, Villanueva and
[1–9]. The World Health Organization (WHO) formulated
Loke examined references in more detail. The study of Lankinen
ethical and scientiﬁc criteria for pharmaceutical drug promotion
et al. indicated that 21% of references were either totally irrelevant
in 1968. According to these criteria, advertisements should
to the claim, or at least did not provide scientiﬁc support to it .
provide ‘scientiﬁcally based, balanced factual information without
In the study of Villanueva et al., the promotional statement was
ambiguities’ . Another attempt to establish norms for promo-
not supported by the reference in 41% of claims . Loke et al.
tional quality was made by the International Federation of
found that, of 855 quantitatively substantiated claims, 55%
referred to a randomized controlled trial or a systematic review
The code became operational in 1981 and the accompanying
as their source of information . Three studies [5, 6, 8] assessed
complaints procedure in 1982. Information is judged according to
the methodological quality of articles that were referred to, but
whether it is up to date, meets ethical standards, is not misleading,
only Lexchin and Holbrook used an ofﬁcial score to assess the
reveals its commercial nature, and provides qualiﬁcation of
methodological quality of articles cited . Further, of all studies
keywords such as ‘safe’. The promotional material should be
only two focused on a single medical discipline [5, 9].
‘based on an up-to-date evaluation of evidence’. A number of
None of the researchers focused on the discipline of rheuma-
pharmaceutical companies have adopted the IFPMA code as
tology. This subspecialty, however, has been the subject of
their guideline for advertising .
important changes during the last 6 yr. According to the 2005
Several studies have tried to substantiate the value of pharma-
president’s speech of the American College of Rheumatology,
ceutical advertisements as a source of information for doctors,
ﬁve different billion-dollar drugs have emerged in rheumatology
thereby focusing on a wide range of issues. Some researchers
in recent years, making this subspecialty the focus of the ‘most
focused on the presence of important information such as drug
Considering this development and the results of the studies
References supporting the promotional claims can make the
mentioned above, it is interesting to evaluate the quality of
Department of Clinical Epidemiology and 1Department of Rheumatology, Leiden University Medical Center, Leiden, The Netherlands.
Submitted 22 November 2005; revised version accepted 7 February 2006.
Correspondence to: F. W. Dekker, Department of Clinical Epidemiology, Leiden University Medical Center, PO Box 9600, 2300 RC Leiden,
The Netherlands. E-mail: [email protected]ß The Author 2006. Published by Oxford University Press on behalf of the British Society for Rheumatology. All rights reserved. For Permissions, please email: [email protected]
How evidence-based are advertisements in journals?
information given in these advertising campaigns. Therefore, the
with nothing but well supported claims were labelled ‘well
aim of the present study was to measure the quality of evidence
supported’. Advertisements with one or more misleading claims
behind advertisements in four leading journals of rheumatology.
were considered ‘misleading’. Score forms were given to two
This was done by focusing on claims and evaluating the
researchers, who entered the data in SPSS (Chicago, Illinois, USA).
accompanying references. References to trials were evaluatedwith regard to their methodological properties and potentialto support the claim. For the methodological properties of
randomized controlled trials, the modiﬁed score of Chalmerswas used.
From the sample of four journals, 353 advertisements wereobtained, representing 31 different products. After exclusion ofidentical advertisements, 84 advertisements remained. These 84advertisements provided a total of 300 references. These references
were evaluated; 53 (18%) were considered ‘supporting’, indicating
Advertisements used in this study were extracted from four
that the article referred to was a traceable, methodologically
journals: Arthritis and Rheumatism, January to October 2004;
correct RCT (n ¼ 43), a systematic review (n ¼ 2) or other research
Rheumatology, 2003; Seminars in Arthritis and Rheumatism, 2002
(n ¼ 8) with sufﬁcient methodological quality (neutral or positive
and 2003; and Journal of Rheumatology, 2004. A team of four
Chalmers’ score), which dealt with population, intervention and
researchers searched the above journals and included all drug-
control characteristics comparable to the claim. Of the 247 ‘non-
promoting advertisements with one or more claims. Identical
supporting’ references, 181 referred to ‘data on ﬁle’, ‘prescribing
advertisements were copied only once but the frequency of
information’ and ‘other sources (e.g. web sites, proceedings,
appearance was noted. The advertisements found were distributed
research not traceable on PubMed), 44 referred to RCTs and
among the research group and evaluated using a standardized
22 to other research. Of the 300 references, 87 referred to RCTs.
score form. Meetings to discuss deﬁnitions and qualiﬁcations were
However, 84 (97%) of the 87 RCTs were sponsored by the
organized to diminish inter-observer variability.
advertising company. In none of the 87 cases did the RCT referred
Each claim was evaluated. A claim was deﬁned as all statements
to get a negative Chalmers score: 57 RCTs got a positive score and
concerning the safety or efﬁcacy of a drug, and other statements
the remainder a neutral score. Forty-nine per cent of the references
if they included a reference. Prescribing information in advertise-
to RCTs were supporting the claim, indicating that the Chalmers
ments (always included in separate textboxes) was not checked
score was neutral or positive and no discordance was found when
for claims because this was regarded as non-promotional informa-
comparing the advertisement and the article referred to with
tion. The quality of evidence-based advertising was evaluated
respect to the populations, controls and interventions.
by determining the number of claims with references, their
The 353 advertisements obtained represented 31 different
products from the whole spectrum of drugs used in rheumatology.
query.fcgi?db¼PubMed), and by comparing the claim with the
A total of 116 advertisements (33%) were published on three
factual information in the references. Points of comparison were
different TNF-blockers. Four coxibs formed the second largest
those essential for the validity of the reference serving as evidence
group, with 59 advertisements (17%), followed by cholinergic
for the claim. Thus, we asked whether the population, the controls
agonists (10%), bisphosphonates, viscosupplements and NSAIDs
and the intervention in the advertisement corresponded to the
(7% each) and a group ‘other’, representing 19% of all advertise-
article referred to. Furthermore, all references were divided into
ments. Misleading advertisements were found for all the different
the following categories: systematic review; randomized controlled
drug types except coxibs and biphosphanates.
trial (RCT); research other than an RCT or systematic review;
In the 84 unique advertisements, 288 claims were found. Ninety-
prescribing information; data on ﬁle; and ‘other’ (e.g. web sites,
eight claims did not refer to any source, so they were categorized as
proceedings, articles not traceable on PubMed). The methodo-
‘poorly supported’. One hundred and ninety claims had one or
logical properties of the RCTs referred to were also assessed.
more references. Of these, 16 were judged ‘misleading’, 33 ‘well
For this purpose, a score described by Chalmers et al.  was
supported’ and 141 ‘poorly supported’. Ninety-nine of 141 poorly
used, which is a more detailed variant of the more widely used
supported claims were so judged because they provided only one
Jadad score . Both scoring systems rate the methodology of thestudy by focusing on the reports of randomization, blinding
and the handling of withdrawals. The Chalmers score rates these
three items on a scale of 0–3. The sum of the three subscores formsthe total score, which can be grouped into the following categories:<3, negative; 3–5, neutral; and 6þ, positive.
Conclusions were drawn using the following deﬁnitions.
When references were of neutral or positive methodologicalquality, and in accordance with the claim, they were considered
‘supporting’. When the evidence for the claim was poor
(differences in populations, controls or interventions), they werecalled ‘non-supporting’. Because only articles available onPubMed were considered accessible and scientiﬁcally interpretablefor practitioners, references were also called ‘non-supporting’when the reference category was ‘prescribing information’, ‘data
on ﬁle’ or ‘other’. At the claim level, the outcome could be ‘wellsupported’, ‘poorly supported’ or ‘misleading’. Claims referringto traceable systematic reviews or trials, which supported the
• 77 data on file (63x being only ref.)
claim, were labelled ‘well supported’. Claims without a traceable
• 35 prescribing inf. (23x being only ref)
reference or with an insufﬁciently supporting reference were labelled
‘poorly supported’. Claims that referred to an article that contra-
dicted the claim were labelled ‘misleading’. Advertisements were
• 13 other research (6x being only ref)
rated ‘well supported’, ‘poorly supported’, ‘misleading’ or ascontaining both well and poorly supported claims. Advertisements
reference, referring to data on ﬁle, prescribing information or other
RCTs got a neutral or positive Chalmers score. The major problem
data (the three poorly supporting categories of information)
was that the advertisements tended to extrapolate trial results
to other patient groups, dosages or treatments. In addition,
Five per cent of the 84 different advertisements were
the advertising companies sponsored 97% of the RCTs referred
labelled ‘well supported’, meaning that their claims were all well
to. This is relevant as the results from RCTs differ according to
supported by their references. Thirteen per cent of the 84 different
whether they are or are not company-sponsored trials .
advertisements were labelled ‘misleading’ because they contained
This study included four rheumatology journals. We have not
one or more misleading claims, i.e. claims with references to
formally tested whether the advertisements in these journals are
articles contradicting the claim (Table 1).
a proper reﬂection of all journals regarding rheumatology, butthere are no data to indicate that advertisements differ in thedifferent journals. Three of the four journals examined were from
the American side of the Atlantic. Because both the Americanand the European pharmaceutical industry have adopted the
This is the ﬁrst study that has assessed the quality of information
IFMPA Code of Pharmaceutical Marketing Practices as their
provided by advertisements in journals covering the subspecialty of
advertising guideline , we do not expect large differences
rheumatology. Remarkably, only four (5%) out of 84 advertise-
between American and European advertising practices as a result
ments were fully supported by high-quality evidence. Many of the
of differences in regulatory requirements. The volumes of the
references accompanying claims did not refer to a proper paper
journals we used formed a relative small sample, but an up-to-date
traceable on PubMed. Three categories—data on ﬁle, prescribinginformation, and other (e.g. web sites and readers)—accounted
one. The assessment itself was subjective in its core, but researchers
for 60% of the references; the category most referred to was data
used standardized score forms and discussed difﬁculties in
on ﬁle (30%). There is a possibility that data on ﬁle will be
interpretation. A small pilot study was performed to train
published later and will become available for validation of the
researchers in using the score form, and to discover shortcomings
claim. Therefore, the term ‘non-supporting’ should not be inter-
preted as a deﬁnitive judgement for this category of references.
In the present study, a rather small percentage (5%) of the
Moreover, the relatively high proportion of references to data on
advertisements proved to be evidence-based. In a comparable
ﬁle may be caused by the tendency of pharmaceutical companies
study, Gutknecht classiﬁed 18% of distinctive advertisements
to advertise for new indications for which the data have been
(n ¼ 187) as evidence-based. In his study, ‘evidence-based’ meant:
presented to the regulatory authorities but have not yet been
‘citing numerical data from controlled clinical trials, and pre-
published in the public literature. However, as a way to distribute
sumably seeking to inﬂuence readers through provision of that
evidence’ . We used a wider deﬁnition (not restricted to the citing
questioned. Cooper and Schriger investigated the availability of
of numerical data) and still found a smaller percentage, probably
cited documents: only 20% of the requests for data on ﬁle
because we made a comparison between the claim and the
produced a document, showing that data on ﬁle are not easy
information in the article and found many incompatibilities.
available . Thus, it may be more logical to refer to the formal
Only when a referred-to trial supported a claim was the claim
label of the drug instead of data on ﬁle, because this information is
regarded evidence-based. In conclusion, this study indicates that
there is room for improvement in the quality of information in
Companies do not present a balanced list of references
representing pro and contra articles in their advertisement. Thiscould lead to a biased opinion when practitioners do not realize
this enough. In the present study we did not assess the potentialinﬂuence on how evidence-based an advertisement is.
We only assessed the various forms of research (RCT,
journals are entirely evidence-based.
systematic review and other research methods) available onPubMed as possible evidence to support claims, because thesearticles give the opportunity to draw conclusions based on theresults given. However, in 44 (51%) of 87 references referring
to an RCT, the RCT did not support the claim properly. The problem was never the methodological quality, as all the
TABLE 1. Assessment of advertisements with and without referenced
No conﬂict of interest has been declared by the authors.
1. Lankinen KS, Levola T, Marttinen K, Puumalainen I, Helin-
Salmivaara A. Industry guidelines, laws and regulations ignored:
quality of drug advertising in medical journals. Pharmacoepidemiol
2. Gutknecht DR. Evidence-based advertising? A survey of four major
journals. J Am Board Fam Pract 2001;14:197–200.
3. Villanueva P, Peiro S, Librero J, Pereiro I. Accuracy of pharma-
ceutical advertisements in medical journals. Lancet 2003;36:27–32.
4. Loke TW, Koh FC, Ward JE. Pharmaceutical advertisement claims in
Australian medical publications. Med J Aust 2002;177:291–3.
5. Bhattacharyya T, Tornetta P 3rd, Healy WL, Einhorn TA. The
validity of claims made in orthopaedic print advertisements. J Bone
aClaims concerning efﬁcacy or safety, without references.
How evidence-based are advertisements in journals?
6. Wilkes MS, Doblin BH, Shapiro MF. Pharmaceutical advertisements
13. Chalmers TC, Smith H Jr, Blackburn B et al. A method for assessing
in leading medical journals: experts’ assessments. Ann Intern Med
the quality of a randomized control trial. Control Clin Trials
7. Cooper RJ, Schriger DL. The availability of references and the
14. Jadad AR, Moore RA, Carroll D et al. Assessing the quality of
sponsorship of original research cited in pharmaceutical advertise-
reports of randomized clinical trial: is blinding necessary? Control
8. Lexchin J, Holbrook A. Methodologic quality and relevance of
15. Bekelman JE, Li Y, Gross CP. Scope and impact of ﬁnancial conﬂicts
references in pharmaceutical advertisements in a Canadian medical
of interest in biomedical research: a systematic review. JAMA
9. Sutherland AG, Craig N, Maffulli N, Brooksbank A, Moir JS.
Accuracy of references in the orthopaedic literature. J Bone Joint SurgBr 2000;82:9–10.
10. WHO. Resolution WHA41.17 adopted by the Forty-ﬁrst World
Health Assembly, 13 May 1988. Geneve: World Health Organization,
The SEDUCE study groups consists of 14 medical students
of the Leiden University Medical Center: P. van Winkelen,
11. IFPMA. IFPMA code of pharmaceutical marketing practices.
J. S. van Denderen, R. A. van Adrichem, M. M. de Boer,
Geneva: International Federation of Pharmaceutical Manufacturers
M. S. Bruijn, D. J. Bruijn, P. D. Dijk, B. P. C. Hoppe,
C. J. Kortmann, S. van der Marel, A. G. van de Merwe,
J. van Rhenen, S. Saadatmand and V. Vennemann; and of
C. Y. Vossen, T. W. J. Huizinga and F. W. Dekker.
Diese Broschüre versucht, die häufigsten onkologischen Fachbegriffe zu er-klären. Sie soll zu einer besseren Kommunikation zwischen Patienten, An-gehörigen und Krankenhauspersonal beitragen. Antigen : eine Substanz, die vom Körper als fremd erkannt wird Abdomen: Bauch Abdomenleeraufnahme: Röntgen- Antikörper : Bluteiweißsubstanzen, Abstoßung: die Lymphozyten des Aorta : Kö
Définition Produits iod Produits iodés hydrosolubles permettant Les produits de contraste d’augmenter le contraste des structures vasculaires ou urinaires. en radiologie urinaire Deux grandes catégories: Vivien THOMSON Janvier 2004 Produits injectables Propriétés physico-chimiques Propriétés physico-chimiques OSMOLALITE OSMOLALIT