Flu Vaccine Studies Review

[Celebration]

Wow, what a surprising revelation~~

:sarcasm:

http://www.ncbi.nlm.nih.gov/pubmed/19213766?ordinalpos=1&itool=EntrezSystem2.PEntrez.Pubmed.Pubmed_ResultsPanel.Pubmed_DefaultReportPanel.Pubmed_RVDocSum

BMJ. 2009 Feb 12;338:b354. doi: 10.1136/bmj.b354. Links
Relation of study quality, concordance, take home message, funding, and impact in studies of influenza vaccines: systematic review.Jefferson T, Di Pietrantonj C, Debalini MG, Rivetti A, Demicheli V.
Cochrane Vaccines Field, ASL (Azienda Sanitaria Locale) AL 20, 15100 Alessandria, Italy. [e-mail]

OBJECTIVE: To explore the relation between study concordance, take home message, funding, and dissemination of comparative studies assessing the effects of influenza vaccines. DESIGN: Systematic review without meta-analysis. DATA EXTRACTION: Search of the Cochrane Library, PubMed, Embase, and the web, without language restriction, for any studies comparing the effects of influenza vaccines against placebo or no intervention. Abstraction and assessment of quality of methods were carried out. DATA SYNTHESIS: We identified 259 primary studies (274 datasets). Higher quality studies were significantly more likely to show concordance between data presented and conclusions (odds ratio 16.35, 95% confidence interval 4.24 to 63.04) and less likely to favour effectiveness of vaccines (0.04, 0.02 to 0.09). Government funded studies were less likely to have conclusions favouring the vaccines (0.45, 0.26 to 0.90). A higher mean journal impact factor was associated with complete or partial industry funding compared with government or private funding and no funding (differences between means 5.04). Study size was not associated with concordance, content of take home message, funding, and study quality. Higher citation index factor was associated with partial or complete industry funding. This was sensitive to the exclusion from the analysis of studies with undeclared funding. CONCLUSION: Publication in prestigious journals is associated with partial or total industry funding, and this association is not explained by study quality or size

PMID: 19213766

[trotsky]

1. But how does this explain flu vaccine shortages...

when American companies refused to make them because there wasn't enough money involved? And how does this tie into the conspiracy of the FDA, CDC, and NIH covering up for big pharma?

[Celebration]

2. Jefferson's email is included

You could always ask him the questions.

[trotsky]

3. Oh - is he a supporter of the idea...

that the government is engaged in a willful conspiracy to protect big pharma by censoring the kind of information that they just made available to the entire world on the Internet?

[Celebration]

4. how would I know? email him and ask

This seems a diversion from the topic posted, which was about flu vaccine studies showing more favorable results when they were funded by the vaccine makers, and getting more favorable placement in prestigious medical journals. Do you have anything important to add on topic with the original post?

[trotsky]

5. On the contrary, this is quite on topic.

A common theme in this forum is how the FDA and related government agencies are to be totally distrusted, they cover everything up for big pharma, push through treatments and vaccines that kill people, etc. That this study came out through the NIH would serve as evidence against such a conspiracy.

I mean, no one in here that I am aware of thinks that pharmaceuticals are all about rainbows and happy bunnies - money is a huge problem, as it is for just about every health-related topic. Supplement makers, quack treatments, all motivated by money. But while this study would only reinforce what most of us already knew, it is actually negative evidence for the kind of conspiracy that some promote.

[Celebration]

7. start another conspiracy op then n/t

[Why Syzygy]

6. BMJ - British Medical Journal

This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
http://www.pubmedcentral.nih.gov/articlerender.fcgi?tool=pubmed&pubmedid=19213766

BMJ. 2009; 338: b354.
Published online 2009 February 12. doi: 10.1136/bmj.b354.

PMCID: PMC2643439
Copyright © Jefferson et al 2009
Relation of study quality, concordance, take home message, funding, and impact in studies of influenza vaccines: systematic review
T Jefferson, coordinator,corresponding author C Di Pietrantonj, statistician, M G Debalini, researcher, A Rivetti, researcher , and V Demicheli, director of health, Piemonte region
1Cochrane Vaccines Field, ASL (Azienda Sanitaria Locale) AL 20, 15100 Alessandria, Italy
corresponding authorCorresponding author.
Correspondence to: T Jefferson Email: [e-mail]
Accepted November 3, 2008.

Introduction
Healthcare workers wanting to keep up to date with recent advances in their specialty must deal with the quantity and quality of information sources.1 2 One study estimated that every month journals publish 7287 items (studies, letters, and editorials) relevant to primary care. These would take physicians trained in epidemiology 627 hours to read and appraise. Few healthcare workers have the time and skills to carry out in depth critical appraisal of published articles.3 Given the pressures of everyday work, the time available for reading a scientific article might be as little as 22 minutes.4
To inform their conduct and update their knowledge physicians might rely on a brief (two minute) scan of material.5 6 This probably includes either browsing the abstract (the “shop window” of an article) or the conclusions paragraph (the “take home message”), especially those published in prestigious journals (that is, those with the highest journal impact factor) that are most readily available or of their digests. The impact factor was specifically developed to facilitate prioritisation of subscription resources, allowing targeting on the most cited journals, considered to be most read and the ones that publish articles of the best quality. These journals are thus more likely to be accessible and their content widely disseminated.7 8
Methodological quality of studies is the other important aspect. Several items, all affecting the reliability of what presents to healthcare workers, are related in a complex fashion. Such quality is associated with the size of the effects reported in a study. The lower the quality, the greater the effect of the intervention seems.9 10
Funding of meta-analyses of antihypertensive drugs by individual pharmaceutical companies has been shown to be inversely associated with whether data presented supported the authors’ conclusions (concordance)11 and directly associated with the probability that study conclusions did or did not support the drugs being evaluated (take home message).12 The relation between the items of quality, concordance, take home message, and funding of single studies, however, is thought to be specific to the context, drugs evaluated, and outcomes assessed. This implies that generalisation of quality across study designs, interventions, and outcomes cannot be made on current evidence. In addition we found no studies looking at the relation of all these variables with that of study dissemination and possible impact.13 14
We explored the relation between study concordance, take home message, funding, and citation (as a proxy for dissemination) in studies on influenza vaccines by reviewing all published comparative studies. Our choice was motivated by the global nature of the intervention, the almost exclusive analysis by analytical studies of quality items in randomised controlled trials or their meta-analyses (in influenza vaccines there is an abundance of non-randomised evidence), and by the recent publication of studies suggesting poor quality and contradictory evidence from field studies supporting vaccination.15 16 17 18 19 20


Methods
Evidence searches
We systematically reviewed comparative studies on influenza vaccines. We used the bibliography of available up to date systematic reviews of influenza vaccines to identify studies comparing the effects of influenza vaccines with placebo or no intervention (“primary studies”) and carried out supplementary searches for all other comparative primary studies published by the end of 2006 (see appendix on bmj.com for search strategy and procedure).
Data extraction and quality assessment
We extracted primary study data in duplicate according to a list of variables (box 1).
Box 1 Variables extracted from primary studies

* Study identification
* Year(s) of execution of the study (the end of the past influenza season cited in the study)
* Year of publication
* Study design and location
* Number of arms
* Comparison and type of outcome assessed within the same study (papers divided into separate studies or datasets if they contained more than one study or comparison)
* Study population
* Types of influenza vaccine used
* Content of influenza vaccines used
* Match between content and circulating viruses in the community
* Types of outcome assessed
* Intervention arm size
* Number of events observed in the intervention arms
* Types of control
* Size of control arms
* Number of events observed in the control arms
* Type of funder
* Impact factor assigned to the journal for the year after the study was published, number of times the articles have been cited by other sources (citation index factor), conclusions of article (classified according to the categories reported in box 2)

Two reviewers, independently unblinded to the authors and institutions of the article, extracted data from the studies in three phases. Firstly, they read each study and compiled an abstract describing study content. Data presented in the study were summarised on a computerised database based on the Cochrane vaccines field study register. This process involved understanding the design and intent of the authors as well as reading and reconciling each part of the study. We then compared our abstract with the authors’ presentation and summaries of results. To classify concordance and take home message definitions we asked whether data presented in the study supported the authors’ conclusions and then whether the conclusions did or did not support the vaccine(s) being evaluated. We classified take home message of all studies as supporting, neutral, or critical of the effects of influenza vaccines (box 2). The definitions were based on work carried out on study concordance and take home message.11 12 We grouped funding sources into government/private/unfunded or industry/mixed, concordance into yes or no/partially/unclear, and take home message into favourable or mixed/unfavourable/unclear.
Box 2 Classification of study take home message
Supportive

* The authors state the dominance of the vaccines even without supporting statistical conclusions
* The study reports statistically significant vaccine effects compared with control intervention(s)
* The authors report the main study end points as statistically significantly dominant (P<0.05) compared with the control arm(s)
* The authors state that vaccination reduces disease burden and/or treatment costs and/or resource consumption
* The authors stress the efficacy, effectiveness, and safety of the vaccines compared with control intervention(s)
* The authors conclude that vaccination should be undertaken
* The authors criticise dissenters on the effects of vaccination
* The authors support at least one of the producers’ or programme sponsors’ statements on the vaccines

Neutral

* The authors conclude that there are insufficient data to assess one or all of the effects of the vaccines
* The authors make no recommendations but seem fairly to weigh up pros and cons of each course of action
* The authors conclude that vaccination is cost neutral
* The authors conclude that the data presented are insufficient to answer the study question
* The authors state that after vaccination the evidence for reduction of disease burden and/or treatment costs and/or resource consumption is slight or non-existent
* The authors dissent from at least one of the producers’ or programme sponsors’ statements on the vaccines but do not reach a definite conclusion
* The authors conclude that the evidence shows efficacy, effectiveness, and safety of the vaccines compared with control intervention(s) for some outcomes but not for others
* The authors conclude that the evidence shows efficacy, effectiveness, and safety of one type of vaccine compared with another and with control intervention(s)

Critical/not supportive

* The authors report no statistically significant efficacy or effectiveness results and/or significant harms
* The authors clearly state the vaccines’ inferiority and present data showing significant vaccine inferiority compared with control(s)
* The authors state that vaccination does not reduce disease burden and/or treatment costs and/or resource consumption
* The authors emphasise concerns over efficacy and/or effectiveness and/or safety and/or recommend alternative interventions to control the burden of disease
* The authors criticise those supporting influenza vaccination or state that vaccination should not be introduced or continued
* The authors are critical of all the producers’ or programme sponsors’ statements on the vaccines

Finally, we assessed quality of randomised and non-randomised studies separately. Randomised studies were assessed according to randomisation method, generation of the allocation sequence, allocation concealment, blinding, and follow-up. Non-randomised studies were assessed for the presence of potential confounders with the appropriate Newcastle-Ottawa scales29 for case-control and cohort studies. We assigned risk of bias (low, moderate, and high risk of bias) as described by the Cochrane Handbook30 by design category (randomised and non-randomised).
Statistical methods
We used χ2 tests to assess the association between items such as quality of methods, funding sources, coherence, and take home message.
To assess the level of dissemination we identified the impact factor of the journals publishing each of the studies in the review. A journal’s impact factor is calculated from the number of citations in the current year to items published in the previous two years (numerator), divided by the number of substantive articles and reviews published in the same two years (the denominator). This is proposed as a proxy for journal quality and, by implication, for quality of the research articles in that journal.7 8 The impact factor was designed to identify journals worthy of subscription, and its rankings fit well with the perceived prestige of journals8 and their circulation.
We identified the citation index of each article (the cumulative total of the number of times the article has been cited in journals indexed in the ISI Web of Science database). The latest impact factor and citation indices were taken from the ISI website http://isiwebofknowledge.com/.
We also tested the association between delay in publication (difference between the publication date and the date the study was completed), the journal impact factor, or the citation index factor against the items described above by Wilcoxon and Kruskal-Wallis non-parametric tests (SAS/STAT 9.1, SAS, Cary, NC). Other studies have described an association between type of take home message and citation and type of funder and citation.12 21 22 23 24 25 26 27 28


Results
Characteristics of primary studies
The figure shows the flow of the included studies.
figure jeft612622.f1
Flow of primary studies in review (RCT=randomised controlled trial; CCT=clinical controlled trial, semi-randomised trial)
We included 274 comparative studies/datasets (from 259 papers) in our population. These were mostly randomised or semi-randomised (that is, the methods of allocation were alternation or birth date) studies (138/274, 50%) or prospective cohort studies (69/274, 25%). Most studies were carried out on healthy or general populations (203/274, 74%) and were government financed (48%). Seventy per cent of studies reported conclusions favourable to the vaccines but only 18% showed complete concordance between data reported and study conclusions. Over half (56%) of studies were at high risk of bias, with only 4% being at low risk. We did not include any vaccine registration trials.
Association between study quality and concordance, take home message, and funding
Table 1 summarises our main findings. All comparisons assessing the association with study quality were carried out after we excluded studies for which we had insufficient data (4/274). We calculated χ2 on the basis of the three methodological quality categories of high, moderate, and low risk of bias. We calculated the χ2 for all comparisons of funding sources for three categories (government, industry, missing). Because a high proportion of studies had missing funding information (64/274, 23%) we carried out a full sensitivity analysis to assess robustness of observed associations by systematically assigning studies with unclear funding source to the other known funding categories, thus covering all possible scenarios.
Table 1 Table 1
Summary of main findings of assessment of possible association between methodological quality, concordance between data presented and conclusions reported, take home message of article conclusions, and funding source in comparative studies of (more ...)
Our analysis shows the presence of a strong positive association between methodological quality and concordance between results presented and study conclusions (when we aggregated studies with moderate and high risk of bias: odds ratio 16.35, 95% confidence interval 4.24 to 63.04). In other words, the higher the study quality (and the lower the risk of bias), the higher the probability of concordance. In addition, the higher the probability of concordance, the lower the probability that a study’s conclusions were in favour of vaccines’ effectiveness (0.04, 0.02 to 0.09); table 2 shows examples of discordance between results and conclusions and take home message. We found no association between funding source and methodological quality (excluding studies with unknown funding: 0.74, 0.19 to 2.84), but there was an inverse association between conclusions in favour of the vaccines’ effectiveness and government funding (0.45, 0.26 to 0.90). This finding confirms the established association between funding source and type of study conclusions.21 22 23 24 25 28
Table 2 Table 2
Examples of discrepancy between results and conclusions
Association between study quality items, funding, and dissemination
Statistical tests show no evidence of association between the mean time lapsed from the end of the study to the date of publication (publication delay), methodological quality, type of study design, concordance, coherence, and funder. There was no evidence of association between either journal impact factor or citation index factor and study design, methodological quality, and concordance.
The mean journal impact factor of 92 government funded studies was 3.74, and their mean citation index factor was 33.75. We calculated the mean impact factor on 114 studies, as the oldest studies were published in journals for which no impact factor was available. For the 52 studies wholly or partly funded by industry, the mean impact factor was 8.78 and the mean citation index factor was 58.39. We calculated the mean citation index for 74 studies as the oldest studies were published in journals for which no citation index was available.
Non-parametric testing shows that the impact factor score of studies with complete or partial industry funding was significantly higher than that of studies funded by government and other funders (table 3).
Table 3 Table 3
Analysis of relation between journal impact factor, citation index factor, study sample size, publication delay, methodological quality, take home message content, concordance, and source of funding
Non-parametric testing for article citation index showed a similar significant difference, which was, however, sensitive to the exclusion of studies with no declared funding source. In this case we found no difference in citation between studies with complete or partial funding from industry and studies funded by government and other funders (table 3).
There was no evidence of association between sample size of comparator and study design, methodological quality, concordance, and take home message.
We conclude that study size is an unlikely explanation for the association between industry sponsorship and publication in journals with higher impact factors and that industry funding is associated with a higher probability of publication in journals with a higher impact factor and possibly a higher citation index.
The differences in impact factor means between studies funded by government and those funded by industry or mixed sponsors were significant and sufficiently robust. Whole or part industrial funding was associated with publication in journals with higher impact factors.


Discussion
In the studies we included, poor methodological quality was associated with a discrepancy between results and conclusions, and this in turn was associated with optimistic conclusions in non-government sponsored studies. We found no direct association between these findings and industry funding, but this might have been affected by the sizeable number of studies with undeclared sponsors (23%).
Studies partly or completely sponsored by industry, however, were published in more prestigious journals and are probably cited more, although their methodological quality and size were similar. Some of these findings might help to explain the continuation of a near global policy, despite growing doubts as to its scientific basis.
We reasoned that the combined impact factor and citation index would give us a good idea of the circulation and dissemination of the study (that is, the interest its publication generated). Most of our studies (70%) were of poor quality with overoptimistic conclusions—that is, not supported by the data presented. Those sponsored by industry had greater visibility as they were more likely to be published by high impact factor journals and were likely to be given higher prominence by the international scientific and lay media, despite their apparent equivalent methodological quality and size compared with studies with other funders. Although differences in citation index by study funding are sensitive to the inclusion of the large number of studies with undeclared sources of funding, the higher impact factor and citation index are probably a reflection of a higher profile of industry sponsored studies and a more thorough dissemination of their content.
There are two possible mechanisms involved. Firstly, the same studies are cited more than the others, possibly because of the systematic nature of the dissemination of their results by industry. We have personally observed this on three recent occasions in which industry representatives presented abstracts or reprints of industry sponsored influenza vaccine studies to decision makers, their advisors, and local researchers in an effort to influence their decisions. Symposiums, conferences, and other types of publication further enhance the dissemination process. Often the abstracts were expensively bound and translated into the local language, a tangible sign of their importance to industry.
We cannot say for certain why industry sponsored studies are more attractive to more prestigious journals, but such journals are preferentially targeted by all studies because of their prominence and prestige, so industry sponsored studies might have a higher probability of acceptance. The two mechanisms might be linked, but further research, especially in other specialties, is required. As a measure of transparency for readers and authors, however, we recommend that once a year editors and publishers should post all sources of income related to the running of the journal.
Our finding of lack of concordance between results and conclusions is similar to those of Yank and colleagues in an industry sponsored meta-analysis of antihypertensive drugs.11 In our studies, however, we found a lack of concordance between results and conclusions associated with poor quality rather than any specific sponsorship. Given our findings of lack of concordance in primary studies, the content of current policy might reflect the gap between results and conclusions—that is, synthesis of evidence for policy making might be carried out at two independent levels: that of results and that of conclusions. In most cases, what you see is not necessarily what you get.
What is already known on this topic

* Study sponsorship is associated with optimistic results
* Influenza vaccination continues to be recommended globally, despite growing doubts about the validity of the scientific evidence underpinning policy recommendations

What this study adds

* Evidence is of poor quality, and studies with conclusions in favour of vaccines are of significantly lower methodological quality
* Influenza vaccines studies sponsored by industry are published in journals with higher impact factors and are cited more but are of similar size and quality to the others


Notes
Karen Davies, Peter Doshi, Peter Gøtzsche, and Iain Chalmers commented on previous drafts.
Contributors: TJ, CDP, and VD designed the study. TJ and CDP wrote the protocol. AR and MGD carried out supplementary searches. TJ, AR, and MGD applied inclusion criteria and extracted data. CDP checked data extraction and carried out data analysis and statistical testing. TJ, CDP, and VD wrote the final report. All authors contributed to both protocol and final report. VD is guarantor.
Funding: This study was funded by ASL AL, Alessandria, Piemonte, Italy. The funding source had no role in the study design, collection, analysis, and interpretation of results, in the writing of the report, or in the decision to submit the paper for publication.
Competing interests: TJ owned shares in GlaxoSmithKline and has received consultancy fees from Sanofi-Synthelabo (2002) and Roche (1997-9).
Ethics approval: Not required.
From various sources.12 21 22 23 24 25 26 27 28

Notes
Cite this as: BMJ 2009;338:b354

Footnotes
From various sources.12 21 22 23 24 25 26 27 28
Top
Abstract
Introduction
Methods
Results
Discussion
>References

References
1.
Davies K. The information-seeking behavior of doctors: a review of the evidence. Health Info Libr J 2007;24:78-94. .
2.
Dawes M, Sampson U. Knowledge management in clinical practice: a systematic review of information seeking behavior in physicians. Int J Med Inf 2003;71:9-15.
3.
Alper BS, Hand JA, Elliott SG, Kinkade S, Hauan MJ, Onion DK, et al. How much effort is needed to keep up with the literature relevant for primary care? J Med Libr Assoc 2004;92:429-37. .
4.
Tenopir C, King DW, Clarke MT, Na K, Zhou X. Journal reading patterns and preferences of pediatricians. J Med Libr Assoc 2007;95:56-63. .
5.
Ely JW, Osheroff JA, Ebell MH, Bergus GR, Levy BT, Chambliss ML, et al. Analysis of questions asked by family doctors regarding patient care. BMJ 1999;319:358-61. .
6.
Saint S, Christakis DA, Saha S, Elmore JG, Welsh DE, Baker P, et al. Journal reading habits of internists. J Gen Intern Med 2000;15:881-4. .
7.
Garfield E. The journal impact factor: a brief review. CMAJ 1999;161:979-80. .
8.
Garfield E. The history and meaning of the journal impact factor. JAMA 2006;295:90-3. .
9.
Moher D, Pham B, Jones A, Cook DJ, Jadad AR, Moher M, et al. Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet 1998;352:609-13. .
10.
Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 1995;273:408-12. .
11.
Yank V, Rennie D, Bero LA. Financial ties and concordance between results and conclusions in meta-analyses: retrospective cohort study. BMJ 2007;335:1202-5. .
12.
Lexchin J, Bero LA, Djulbegovic B, Clark O. Pharmaceutical industry sponsorship and research outcome and quality: systematic review. BMJ 2003;326:1167-70. .
13.
Als-Nielsen B, Gluud LL, Gluud C. Methodological quality and treatment effects in randomised trials: a review of six empirical studies. 12th Cochrane Colloquium: bridging the gaps; 2004:88-89. www.mrw.interscience.wiley.com/cochrane/clcmr/articles/CMR-6628/frame.html.
14.
Wood L, Egger M, Gluud LL, Schulz K, Altman D. Juni P, et al. The association of allocation concealment and blinding with estimated treatment effect varies according to type of outcome: a combined analysis of meta-epidemiological studies. 14th Cochrane Colloquium; 2006 Oct 23-26; Dublin, Ireland. www.mrw.interscience.wiley.com/cochrane/clcmr/articles/CMR-10003/frame.html.
15.
Jefferson T. The prevention of seasonal influenza—policy versus evidence. BMJ 2006;333:912-5. .
16.
Jordan R, Connock M, Albon E, Fry-Smith A, Olowokure B, Hawker J, et al. Universal vaccination of children against influenza: are there indirect benefits to the community? A systematic review of the evidence. Vaccine 2006;24:1047-62. .
17.
Thomas RE, Jefferson T, Demicheli V, Rivetti D. Influenza vaccination for healthcare workers who work with the elderly. Cochrane Database Syst Rev 2006;3:CD005187 .
18.
Simonsen L, Taylor RJ, Viboud C, Miller MA, Jackson LA. Mortality benefits of influenza vaccination in elderly people: an ongoing controversy. Lancet Infect Dis 2007;7:658-66. .
19.
Jackson LA, Jackson ML, Nelson JC, Neuzil KM, Weiss NS. Evidence of bias in estimates of influenza vaccine effectiveness in seniors. Int J Epidemiol 2006;35:337-44. .
20.
Eurich DT, Marrie TJ, Johnstone J, Majumdar SR. Mortality reduction with influenza vaccine in patients with pneumonia outside “flu” season: pleiotropic benefits or residual confounding? Am J Respir Crit Care Med 2008;178:527-33. .
21.
Stelfox HT, Chua G, O’Rourke K, Detsky AS. Conflict of interest in the debate over calcium-channel antagonist. N Engl J Med 1998;338:101-6. .
22.
Yaphe J, Edman R, Knishkowy B, Herman J. The association between funding by commercial interests and study outcome in randomized controlled drug trials. Fam Pract 2001;18:565-8. .
23.
Leopold SS, Warme WJ, Braunlich EF, Shott S. Association between funding source and study outcome in orthopaedic research. Clin Orthop Relat Res 2003;415:293-301. .
24.
Bekelman JE, Li Y, Gross CP. Scope and impact of financial conflicts of interest in biomedical research: a systematic review. JAMA 2003;289:454-65. .
25.
Kjaergard L, Als-Nielsen B. Association between competing interests and authors’ conclusions: epidemiological study of randomised clinical trials published in the BMJ. BMJ 2002;325:249-53. .
26.
Friedman LS, Elihu BA, Richter ED. Relationship between conflicts of interest and research results. J Gen Intern Med 2004;19:51-6. .
27.
Bero LA, Galbraith A, Rennie D. Sponsored symposia on environmental tobacco smoke. JAMA 1994;271:612-7. .
28.
Als-Nielsen B, Chen W, Gluud C, Kjaergard LL. Association of funding and conclusions in randomized drug trials: a reflection of treatment effect or adverse events? JAMA 2003;290:921-8. .
29.
Wells GA, Shea B, O’Connell D, Peterson J, Welch V, Losos M, et al. The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. 2005. www.ohri.ca/programs/clinical_epidemiology/oxford.htm.
30.
Higgins JPT, Green S, eds. Cochrane handbook for systematic reviews of Interventions Version 5.0.0. Oxford: Cochrane Collaboration, 2008 www.cochrane-handbook.org.
31.
Wongsurakiat P, Maranetra KN, Wasi C, Kositano U, Dejsomritrutai W, Charoenratanakul S. Acute respiratory illness in patients with COPD and the effectiveness of influenza vaccination: a randomized controlled study. Chest 2004;125:2011-20. .
32.
Wilde JA, McMillan JA, Serwint J, Butta J, O’Riordan MA, Steinhoff MC. Effectiveness of influenza vaccine in health care professionals: a randomized trial. JAMA 1999;281:908-13. .
33.
Carman WF, Elder AG, Wallace LA, McAulay K, Walker A, Murray GD, et al. Effects of influenza vaccination of health-care workers on mortality of elderly people in long-term care: a randomised controlled trial. Lancet 2000;355:93-7. .

*not edited for parenthesis/smilies