Have You Heard of the “Journal Transparency Index”?

Posted on by

A new metric will be launching from the Center for Open Science focusing on scholarly journals and measuring their commitment to research transparency, according to a recent article entitled, “Journal transparency index will be ‘alternative’ to impact scores.” (January 29, 2020)

The background for this measurement can be found with the Transparency & Openness Promotion (TOP) guidelines, a metric that reports the steps that a journal takes to implement open science practices. In reviewing the list of TOP Implementors, as of June 2019, there were over 1,100 journals that have applied one or more TOP-compliant policies including Science (AAAS), and many Nature Springer titles.

The guidelines cover eight transparency standards (citation, data transparency, analytic methods (code) transparency, research materials transparency, design & analysis transparency, preregistration of studies, preregistration of analysis plans, and replication).  Within each standard, there are three levels of implementation (also referred to as levels of increasing stringency), as well as a “not implemented” category. The higher the score in this evaluation rubric, the higher level of support the journal publisher is providing in promoting and encouraging open science.

Among these measures of assessment is data sharing, wherein a journal could score up to three points.  In order for this to happen, a data availability statement must be included, authors would be required to share their data, and the study should be reproducible based on the data provided.

As with any measurement, time ultimately is required to determine the impact of this journal assessment or score.  This ranking system might very well contribute to research transparency and is a step in the right direction in supporting the reproducibility crisis.

Donna Gibson
Director of Library Services

Looking Back at Drug Discovery Success

Posted on by

MSK’s Dr. Charles Sawyers recently won the inaugural STAT Biomedical Innovation Award. Earlier this month, STAT published a video of an interview with Sawyers, during which he discusses stories behind his groundbreaking work and his hopes for the future of cancer treatment.

Dr. Sawyers is MSK’s Chair of the Human Oncology and Pathogenesis Program. Learn more about his discoveries of targeted cancer therapies in this MSK feature.

Systematic Review Manuscript Submission Requirements

Posted on by

The publication of systematic reviews has been on the rise over the last decade, a trend which has greatly supported evidence based practice. As with most things, however, too much of a good thing likely has a downside. In recent years, multiple authors have brought attention to the fact that the “production of systematic reviews and meta‐analyses has reached epidemic proportions” and that their publication may be in need of some “realignment” – see, for example, this article by John Ioannidis:

Ioannidis JP. The Mass Production of Redundant, Misleading, and Conflicted Systematic Reviews and Meta-analyses. Milbank Q. 2016 Sep;94(3):485-514. doi: 10.1111/1468-0009.12210. PMID: 27620683; PMCID: PMC5020151. 

One consequence of all of this has been that many journal publishers are making their manuscript submission requirements and minimum standards for consideration of systematic reviews and meta-analyses a bit more stringent. Below are some examples of requirements authors may come across in a journal’s authors’ instructions for manuscript submission.

It’s now pretty common for journals to not only require the inclusion of a PRISMA Flow diagram, but also the submission of a completed PRISMA checklist.

  • For example, from JAMA Oncology’s instructions for authors:

    “A PRISMA-style flow diagram showing this information should also be included as an online-only supplement. In addition, a completed PRISMA checklist should be submitted for the items completed that apply to systematic reviews (the checklist items that apply to meta-analyses do not need to be completed for systematic reviews without meta-analysis). The checklist will be used during review but will not be published.”

Item no. 5 on the PRISMA checklist asks for information about the existence of a protocol and its registration. Some journals, particularly ones based in Europe, have now actually made prospective registration of the systematic review on PROSPERO, the international prospective register of systematic reviews, or a similar database, compulsory for acceptance and publication (whereas it generally used to be suggested but optional for most journals).

Even more strict, there are publishers that have started being explicit about the minimum number of papers that should be included in the submitted synthesis in order for them to give it their attention and consideration.

  • From the journal Plastic and Reconstructive Surgery‘s instructions for authors:

    “Systematic Reviews & Meta-Analyses: Systematic reviews or meta-analyses that include ≤10 papers will be rejected without review. Manuscripts that review more than 10 papers but have uncertain conclusions (e.g., fatal heterogeneity of data, conclusions state that data are limited and better studies need to be done) will likely be rejected.”

The take-away: Knowing these journal-specific requirements in advance is useful for planning the systematic review or meta-analysis project. Researchers should consider target journal candidates and review their instructions for authors early on in the process.

For more information, be sure to check out the MSK Library’s Systematic Review Service LibGuide or Ask Us.