Advancing Authorship Webinar: The Impact of AI on the Scholarly Publishing Landscape

The release of ChatGPT has generated considerable attention to generative AI. Those involved with scholarly publishing must be vigilant in understanding and managing AI technological developments that impact their work.

How will publishers maintain the integrity of the scholarly record? Manage and weed out non-existent articles? Detect research faked by sophisticated AIs? Adopt and implement policies that address AI?

Join us for presentations and conversation on how JAMA (Journal of the American Medical Association) is evaluating generative AI. JAMA’s Editor in Chief and Executive Managing Editor will share their experiences about what they have been seeing in the editorial process and the new policies developed around use of large language models (LLMs) in manuscript development and peer review. In addition, hear a preliminary view of generative AI from a member of the Memorial Sloan Kettering’s Compliance and Internal Audit department.

Bring your questions about this growing issue as there will be time for attendees to participate in an interactive Q&A.

Date: Wednesday, October 4, 2023
Time: 12:00 PM to 1:00 PM, EST
Zoom Webinar: Register Now

Speaker Bios:

Kirsten Bibbins-Domingo, PhD, MD, MAS, is the 17th Editor in Chief of JAMA (Journal of the American Medical Association) and the JAMA Network. She is the Lee Goldman, MD Endowed Professor of Medicine and Professor of Epidemiology and Biostatistics at the University of California, San Francisco.

Dr. Bibbins-Domingo is a general internist, cardiovascular disease epidemiologist, and a national leader in prevention and interventions to address health disparities. She is a physician-scientist who has used observational studies, pragmatic trials, and simulation modeling to examine effective clinical, public health, and policy interventions aimed at prevention.

Dr. Bibbins-Domingo previously served as the inaugural Vice Dean for Population Health and Health Equity in the UCSF School of Medicine and the Chair of the Department of Epidemiology and Biostatistics at UCSF. She co-founded the UCSF Center for Vulnerable Populations at Zuckerberg San Francisco General Hospital that generates actionable research to advance health equity and reduce health disparities in the San Francisco Bay Area, California, and nationally.

Dr. Bibbins-Domingo was a member of the US Preventive Services Task Force from 2010-2017 and led the Task Force as the vice-chair and chair from 2014-2017. She is an elected member of the American Society for Clinical Investigation, the Association of American Physicians, the National Academy of Medicine, and the American Academy of Arts and Sciences.

Annette Flanagin, RN, MA, FAAN, is Executive Managing Editor and Vice President of Editorial Operations for JAMA and the JAMA Network.

Before joining JAMA, Ms. Flanagin held editorial positions with other journals published for health care professionals, and before that, she practiced cardiology nursing. She is a graduate of Georgetown University, with a BS in nursing and an MA in English Literature.

Ms, Flanagin is Past President of the Council of Science Editors (CSE) and has been honored with the CSE Award for Meritorious Achievement and the John P. McGovern Award for preeminent contribution to medical communication by the American Medical Writers Association. She serves as the Executive Director of the International Congresses on Peer Review and Scientific Publication and as Co-Director of the African Journal Partnership Program. She is a Fellow in the American Academy of Nursing and a member of the board of STM: International Association of Scientific, Technical and Medical Publishers. Ms. Flanagin has co-developed a number of guidelines and policies to guide authors, editors, and publishers in scientific publication and is a committee member and author of the AMA Manual of Style: A Guide for Authors and Editors. She participates in research, lectures, and publishes on issues related to scientific publication for authors, editors, and publishers.

Philip Watson, PhD, is the course director for the Multi-Institutional Responsible Conduct of Research (RCR) program sponsored by Memorial Sloan Kettering (MSK), Weill Cornell Medicine, Rockefeller University, and Hospital for Special Surgery. He is an Associate Director in Compliance and Internal Audit at MSK. Previously, he was an Assistant Lab Member (Assistant Attending Biologist) at MSK, with a research interest in prostate cancer.

 

ChatGPT and Fake Citations: MSK Library Edition

Since the launch of ChatGPT, an artificial intelligence chatbot developed by OpenAI, we at the MSK Library have seen an uptick in requests to track down what turn out to be fake citations for studies related to cancer research.

We decided to pick a topic we were recently asked to conduct a literature search on (survival outcomes, recurrence, and pathology characteristics of poorly differentiated thyroid carcinoma) to see how ChatGPT handled it. Below are screenshots from our conversation. 

Looks pretty good, right? We asked for the full citations. 

Voila, ChatGPT delivered! We then attempted to verify these citations. We first looked them up in databases and citation indexes like PubMed and Google Scholar. Then we checked the DOIs, or digital object identifiers. Finally, we went directly to the journals these “articles” were “published” in to see if they appeared in the same journal, issue, and volume ChatGPT cited, or if they appeared in these journals at all. These citations didn’t appear to be legitimate, so we let ChatpGPT know.

ChatGPT gave the same incorrect citations again. We asked if it was fabricating this information.

Still no dice. It appeared that ChatGPT was “hallucinating.” Learn more about this phenomenon here and here

We asked ChatGPT why it was creating these fake citations, and its response was illuminating. 

Our interaction with ChatGPT isn’t surprising – it’s a large language model and not a database or citation index. ChatGPT is great for some aspects of research, but not others. Check out Duke University Libraries’ blog post ChatGPT and Fake Citations for more information. 

Learn more about AI by visiting our Artificial Intelligence guide.  Need help finding evidence based information? Ask Us