Transparency in research and reporting

Read this article about research transparency and how the responsibility for promoting greater openness in research falls on everyone.

By: Patricia K. Baskin, MS, Executive Editor, Neurology Journals and Robert A. Gross, MD, PhD, FAAN, FAHA, Editor in Chief, Neurology journals, Strong Epilepsy Center and University of Rochester Medical Center

A premise of science is that research is meticulous and objective so the results are valid and credible. Published articles should provide clearly written, transparent descriptions of how the research was conducted, results were obtained, and conclusions were reached based on appropriate uses of analytical tools. Reporting of research should be truthful, free of bias, and provide enough information about how the work was performed to allow others to replicate the work and to be useful for further analyses. In the case of medical research, the information (especially from clinical trials) influences decisions regarding patient care and health policy. However, factors such as individual biases, competition among research groups for funding, interest in career advancement, and, for drug companies, profitability, often discourage openness and transparency. This lack of transparency is often evident not only in lack of clarity or completeness in the writing of a report, but also in incomplete reporting of data, analytic tools, and materials, hindering replication efforts.

The responsibility for promoting greater openness in research falls not only to the individuals performing the work, but to the funders of the work (including government, foundation, and industry sponsors), institutions where the work is being done, and to journal editors and peer reviewers, who do the final check on the quality of the research before it is released to readers. Many journals have encouraged higher quality research by creating more useful, specific information for authors, requiring registration of clinical trials, and requiring adherence to published guidelines for reporting specific types of medical studies. Most journals enforce consequences for scientific misconduct or ethical breeches; however, transparency is often lacking. Fortunately, new guidelines are being developed to move scientific reporting toward greater openness.

TOPS Initiative

In November 2014, a joint meeting of the Berkeley Initiative for Transparency in the Social Sciences, Science magazine, and the Center for Open Science (COS) (a non-profit technology company that provides free and open services to increase research transparency and align more closely with scientific values) resulted in the creation of a set of guidelines using several categories of openness as requirements for publication.¹ The Transparency and Openness Promotion (TOP) Committee, sponsored by COS, created these guidelines to promote transparency, openness, reproducibility of scientific research, and, in the process, public credibility. These guidelines stipulate varying levels of openness based on the mission of a specific journal, leading to increased credibility and, at the highest levels, reproducibility. An introduction to these guidelines, authored by a group of researchers, journal editors, funders, and society leaders was published on June 26, 2015 in Science. The guidelines cover eight standards of transparency in the research process, with three levels of transparency for each standard, so journals can adopt standards with a level of stringency most appropriate for their own missions.

The eight transparency standards addressed by the TOP guidelines include (1) citation standards for citing articles and data, thus recognizing original contributions; (2) data transparency, stating the level of availability of data; (3) analytic methods, stating the statistical methods and software used; (4) research materials, stating the level of sharing; (5) reporting research design and analysis about the research process and completeness of reporting of the methodology; (6) preregistration of studies to make research more discoverable even if it is not ultimately published; (7) preregistration of analysis plans to verify whether the research is hypothesis-testing or hypothesis-generating, and (8) replication, which addresses whether or at what level the journal requires independent replications of a study before publication. The TOP Committee suggests that journals select the standards they wish to adopt and at which level. Some editors may believe that this type of reform should come from the research community itself rather than editors, especially because editors cannot easily enforce compliance. However, the adoption of such standards is an important effort that provides publishers with tools to communicate with researchers about expectations. Numerous endorsements by journals can be viewed on the COS website.

The EQUATOR Network

The EQUATOR (Enhancing the QUAlity and Transparency Of health Research) Network is a source for scientists, editors, and institutions wanting to define best practices in reporting of medical research. It is an international initiative that was launched in 2008, and at this writing, curates 276 guidelines with more in development. The goal of this organization is “to improve the reliability and value of published health research literature by promoting transparent and accurate reporting and wider use of robust reporting guidelines.” EQUATOR has developed design-specific guidelines for scientific reporting and offers resources for editors, authors, and educators. Last year, the EQUATOR Network launched three centers (in the UK, France, and Canada) to expand their activities in supporting adoption of good research reporting practices.

The practices that caused most concern to the EQUATOR guidelines developers included non-reporting of negative studies, selective reporting of outcomes in studies, omission of information in describing research methods and interventions, inadequate reporting of adverse events, and misleading presentations of results and data—all potentially leading to “spin” in the medical literature. ^2,3 The reporting guidelines aim to alleviate these shortcomings by providing specific advice and educational tools for authors, editors, and educators. The reporting guidelines for the main study types include CONSORT (for randomized clinical trials, requiring a checklist and prospective trial registration), which also has a number of more specific extensions pertaining to such areas as trial designs that differ from the standard trial or on reporting harms; STROBE (for observational studies) and its extensions; PRIMSA (for systematic reviews) and its extensions; STARD (for diagnostic/prognostic studies); and CARE (for case reports).

The EQUATOR Network also curates toolkits for authors, editors, and educators, along with information and resources for guideline developers. For instance, authors are pointed to the UK NIHR Clinical Trials Toolkit, which gives researchers practical advice about designing and conducting publicly funded clinical trials in the UK through an interactive roadmap. Other toolkits provide advice on best design of studies, budgeting, legislative requirements related to human tissues, and systematic reviews, in addition to guidance on scientific writing, data sharing, and ethical conduct. Toolkits for editors focus on developing a journal’s policies on reporting of research (using the most relevant reporting guidelines) and guiding peer reviewers. The Teacher Toolkit provides information for trainers of researchers on good research reporting and the deleterious effects of poor reporting.

Author transparency

An article on transparency in research or the publishing of research would not be complete without a discussion on transparency as applied to authorship. Multiple influences affect the credibility of the authors themselves; everyone has biases. Many clinical trials are sponsored by drug companies, while other studies are funded by governmental agencies or foundations, and authors are eager to renew grants or secure tenure. It often is not clear who did what in the writing of subsequent reports: Who wrote the paper? Who actually performed the research? Who provided materials or performed statistical analyses? Who might have contributed intellectually but is not named in the byline (a “ghost author”) and conversely, who might have been placed in the byline as a courtesy by being associated with other members of an author group, e.g., a department chair, who is really a “guest author”? Who paid for the research? All this information needs to be available so readers can decide for themselves if bias exists in the research or the reporting of a study. Thus, it’s important that authors are accurately identified according to criteria prescribed by a journal and that journals publish the contributions and full disclosures of all identified authors. It’s worth noting an important distinction at this point: Disclosures offer insights into readily measurable conflicts of interest (usually, financial), but no disclosure adequately measures the biases of authors, reviewers, or editors. Bias is the problem, of course, for which disclosures of conflicts of interest are merely surrogate measures.^4,5

Although the International Committee of Medical Journal Editors has established widely endorsed criteria for authorship, it is unrealistic for author groups to follow them because all authors must fulfill four criteria; adherence becomes increasingly difficult as research collaborations have become more global and collaborators on large projects come from multiple specific disciplines. These criteria include (1) substantial contributions to the conception or design of the work, or acquisition, analysis, or interpretation of data for the work; AND (2) drafting the work or revising it critically for important intellectual content; AND (3) final approval of the version to be published; AND (4) agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.⁶ However, these criteria encourage listing of guest authors because it is difficult to omit names of investigators who contributed heavily but did not fulfill all criteria (for example, someone who designed experiments or performed analysis but did not take part in the writing) and leaves room for ghost authors—those who drafted or wrote papers but do not qualify otherwise. At Neurology, we modified the criteria to avoid guest and ghost authors and to enhance the professionalism of medical writers: We require those named as authors to have participated in (1) the design or conceptualization of the study; OR (2) analysis or interpretation of the data; OR (3) drafting or revising the manuscript for intellectual content.^7-12 It follows that professional writers employed by pharmaceutical companies or other academic, government, or commercial entities who have drafted or revised the intellectual content of the paper must be included as authors. All authors state their contributions in an electronic form and all contributions are published to provide the highest level of transparency.

Author contributions

Another group, Project CRediT, is working on developing digital taxonomies to help researchers in the biomedical and life-sciences community categorize their contributions to collaborative products.¹³ Because it is often difficult to see who did what during research and writing of a study when many collaborators are involved, a contributor roles taxonomy covering all roles of contributors may make recording contributions easier and more accurate for authors at the outset. One of the planned experiments in this effort is to integrate a digital taxonomy of contributorship with research-management systems, which could affect the processes of doing research. The group is collaborating with bodies such as the National Information Standards Organization to further develop the taxonomy.

The pharmaceutical industry

The pharmaceutical industry is also making attempts to create more transparency and credibility in publications generated by their research divisions. The Medical Publishing Insights and Practices Initiative is a collaboration among pharmaceutical companies aimed at elevating trust, transparency, and integrity in publishing industry-sponsored studies. The group has worked with journal editors, clinical investigators, academic collaborators, and representatives of industry to develop a five-step authorship framework for a more prospective, consistent, and transparent process to assign authorship in industry publications.¹⁴ Steps include establishing an authorship working group early in a trial, determining “substantial” contribution criteria, documenting trial contributions, determining those making substantial contributions, and ensuring that authors meet all authorship criteria required by a journal. Such rules might work to the benefit of groups working in academic centers as well.

Conflict of interest database

For maximum transparency in a research article, all authors should make full disclosure of potential for conflicts of interest, and these disclosures should be published with the article. Soon readers will be hearing about the development of a universal online disclosure program. A prototype of Convey, a new centralized database system for disclosures, was presented on May 18, 2015 at the meeting of the Council of Science Editors in Philadelphia. Fueled by a recommendation by the Institute of Medicine, the American Association of Medical Colleges has worked to develop a centralized disclosure system that provides relevant disclosure information based on organizational criteria and user assessments, which is intuitive to use for both users and subscribing organizations.

The efforts made by all these initiatives are to be applauded by authors as they help in some measure to increase the transparency and validity of their work. The further hope is that expectation of transparency and rigor in reporting will foster better research design and thus more credibility in the eyes of the reader. Editors are always looking for well-written papers containing innovative and paradigm-changing research that colleagues can build upon and replicate.

References

1. Nosek BA, Alter G, Banks GC, et al.. Promoting an open research culture. Science 2015; 348(6242): 1422-1425.
2. Simera I, Moher D, Hirst A, Hoey J, Schulz KF, Altman DG. Transparent and accurate reporting increases reliability, utility, and impact of our research: reporting guidelines and the EQUATOR network. BMC Med 2010; 8:24.
3. Gross RA. Style, spin, and science. Neurology 2015; 85:10–11.
4. Schwid SR, Gross RA. Bias, not conflict of interest, is the enemy. Neurology 2005; 64:1830-1831.
5. Knopman DS, Baskin PK, Pieper KM, Quimby SL, Gross RA. Reporting potential bias: Neurology’s evolving policies. Neurology 2011;76:110-112.
6. International Committee of Medical Journal Editors. Defining the role of authors and contributors. 2014. http://www.icmje.org/recommendations/browse/roles-and-responsibilities/defining-the-role-of-authors-and-contributors.html.
7. Neurology. Information for authors. 2014. neurology.org/site/misc/auth2.xhtml.
8. Baskin PK, Gross RA. Of faces seen, unseen, and behind the scenes. Neurology 2012; 80:1-3.
9. Baskin PK, Gross RA. Honorary and ghost authorship. BMJ 2011; 343:d6223.
10. Baskin PK, Gross RA. Ghosts in the machine: Better definition of author may stem bias. The Conversation 2012. https://theconversation.com/ghosts-in-the-machine-better-definition-of-author-may-stem-bias-4288.
11. Baskin PK, Gross RA. Publishing rigor: an extension of scientific rigor: minding the business of scientific reporting. Neurology 2014; 83:2-3.
12. Baskin PK. Authorship and contributorship: who did what? Science Editor 2014; 37:39,45.
13. Allen L, Brand A, Scott J, Altman M, Hlava M. Publishing: credit where credit is due. Top Picks from Nature News 2014. https://www.nature.com/news/publishing-credit-where-credit-is-due-1.15033.
14. Marusic A, Kren D, Mansi B, et al. Five-step authorship framework to improve transparency in disclosing contributors to industry-sponsored clinical trial publications. BMC Med 2014; 12:197.

Disclosures

Patricia K. Baskin, MS, is Executive Editor of the Neurology journals and employed by the American Academy of Neurology. She consulted with no stipend or reimbursements for the Medical Publishing Insights and Practices initiative at journal-pharma workshops in 2012 and 2014.

Robert A. Gross, MD, PhD, FAAN, is supported for educational endeavors from the University of Rochester Medical Center’s Clinical and Translational Science Award from the NIH. He receives an honorarium from AAN as Editor-in-Chief of Neurology.

Adapted from “Manuscript Rejection: How to Submit a Revision and Tips on Being a Good Peer Reviewer,” by Sandra V. Kotsis, MPH, and Kevin C. Chung, MD, MS. This article originally appeared in Plastic & Reconstructive Surgery, Vol. 133, Issue 4 © 2014 by the American Society of Plastic Surgeons.