Information Access and Scholarly Communication in Post-publication Peer Review Online Social Networks

Abstract

PubMed Commons was launched in 2013 by the National Institutes of Health and established an online community of medical and health science scholars to contribute post-publication peer-reviews to citations indexed within the PubMed literature database. The NIH expressed both satisfaction and disappointment at various times throughout the four year experiment, but on short notice in February 2018, the PubMed Commons service was permanently shutdown.

This thesis explores the online community that was PubMed Commons, and uses qualitative and quantitative methods to analyze the corpus of retrospective data that tracked the online contributions shared by the members. This research will address the following questions. How have online post-publication peer-review networks demonstrated success or faced challenges in sustaining an online community towards the goal of evaluating, augmenting and furthering the published scientific record? Was PubMed Commons deficient in these ways? What strengths did PubMed Commons offer that may have been overlooked before the decision to shut down?

This study will evaluate the full corpus of publication and comment data that was discussed in the four year term that PubMed Commons was active. A bibliometrics analysis of the articles will demonstrate common themes and publishing trends that prompted members to engage in post-publication review. Additionally, a random sample of 155 comments was evaluated and tagged to assess each review for its level of constructiveness, impartiality, objective aims, and timeliness. Interviews were held with several former members of PubMed Commons, and their feedback will be summarized in context. The results demonstrate a highly engaged and active community, and call into question why the PubMed Commons system was deemed unsuccessful.