Comments on the Ellen Roche story
Free Online Scholarship (FOS) Newsletter
August 23, 2001
by Peter SuberPubMed is a premier example of FOS, a contender for FOS at its best. So does the Ellen Roche case prove that FOS is inadequate, even hazardous? How just is this interpretation? What are the lessons of this case for FOS? Please post your thoughts to our discussion forum. Meantime, let me exercise my privilege as editor and offer a first take on this.
We should be careful in drawing lessons from this case. It's not true that limiting a search to *online* sources would hide the dangers of hexamethonium. We know this because the medical librarians on the MedLib listserv found the relevant information in online sources. Nor is it true that limiting a search to *free online* sources (like PubMed) would hide the dangers of hexamethonium. We know this because the JH internal investigators turned up articles on these dangers from Google and Yahoo searches. Today, free online sources may be a feeble subset of online sources, which are in turn a feeble subset of published research. But apparently even free online sources were adequate to ground good care in this case, if diligently consulted. However, this does not vindicate FOS. Even if free online sources turn out to be adequate in this case, there is clearly a large number of other cases where it will not yet be adequate.
If there is a problem with free online sources here, it is that they are not sufficiently connected (or in the industry jargon, "interoperable") with other sources, so that Dr. Togias' search on PubMed did not bring up results buried in other collections. As long as this is true, then researchers will have an obligation, especially in high-stakes research, to run parallel searches on multiple collections. In the meantime, architects of the free online scholarship system of the future are already at work on metadata standards that make interoperability and cross-archive searching commonplace and transparent. Moreover, others are hard at work on the retroactive digitization of selected bodies of print literature. In short, we're already on the right track to make FOS nearly as adequate as the general body of published literature, and more adequate insofar as electronic sources can generate answers more quickly than print resources.
How can we answer the objection that the Roche case proves the inadequacy, even the hazards, of free online scholarship? First I'd point out the evidence (above) that free online sources contained the relevant information in this case. Second, I'd admit that we are in a transition period in which there is significantly less literature online than in print, and less free than priced. The ratio of FOS to published literature (an approximate measure of adequacy) changes from discipline to discipline and from day to day. But we can't pretend that the corpus of FOS is as adequate as the corpus of all published scholarship when, on the contrary, it's a mere subset. Third, I'd admit that we are likely to be in this transition period for a long time. Fourth, I'd urge researchers to draw the conclusion that relevant information may exist without being online, let alone free and online. When easy searches fail, we do not have a bona fide negative result from which we can draw scientific conclusions. Instead we must spend the time and calories to undertake a more arduous search. Fifth, I'd wonder out loud whether there is a good scientist anywhere who didn't already know this.
Finally, I'd point out that part of the underlying problem here is that FOS is compellingly attractive. It is only hazardous because it tempts busy people to rely on it to the exclusion of other research methods. What makes it tempting is not false advertising but spectacular convenience. This is not a hazard of FOS, then, but of unresisted temptation.
Another variable here is that researchers often work under deadlines, pressures, and expectations that jack up the temptation to take short-cuts. The problem is still unresisted temptation. But we don't have to conclude that Dr. Togias didn't know how to do good research. He might have been too busy to use the research methods he knew were more reliable and comprehensive. I doubt that many scientists would be willing to throw the first stone here.
If FOS is spectacularly convenient, but not yet adequate, then it may be hazardous to put before people who don't understand its inadequacy or whose will has been weakened by various pressures. But the same is true of potato chips and money. If the problem is that FOS is available for use before it is adequate, then the real objection is to stepwise progress. Which is better, to make FOS repositories available as they are ready, and let them grow and interconnect in real time, or to hold them back until the network of such repositories encompasses all published literature? Serious scientists would be the first to object to the latter plan. Running a close second would be people with debilitating medical conditions anxious for research breakthroughs.
Let's admit that we're in a transition period. FOS is large and growing, but still small and inadequate compared to the domain of extant published literature. To misunderstand this is like believing that every house can be reached by a superhighway. Some houses can't even be reached by a paved road.
Let's also admit that PubMed doesn't pretend to be more than it is. Its scope does not go back to the 1950's or extend to the rulings of federal administrative agencies. What it does, it does very well, giving users free access to a searchable medical bibliography with over 11 million citations and abstracts. For 18 months, it has been supplemented by PubMed Central, which provides free online access to a growing number of full-text articles. If it appears to a researcher through a veil of illusion, the illusion of sufficiency, then it is the researcher's illusion. The same is true of a print library.
You've probably heard the joke about the person searching for a contact lens on the sidewalk under a street light. "Did you lose your lens here?" "No, but this is where the light is." Dr. Togias, and the JH board which reviewed and approved his experiment, committed the Street Light Fallacy. Once we admit that we're in a transition period, then we must take the responsibility to avoid the Street Light Fallacy in our own research, and teach our students to avoid it in theirs. FOS is growing toward adequacy in every discipline, at different rates, but is probably not adequate in any discipline today. We can advance this cause best by working to make it more adequate without inviting others to presume its present adequacy. Convenience and adequacy are independent variables. This is not esoteric. It's a lesson we learned long ago from McDonald's.
FOS discussion forum(Anyone may read; only subscribers may post; subscription is free.)
Read this issue online
http://dash.harvard.edu/bitstream/handle/1/4314489/suber_news8-23-01.htmlThe Free Online Scholarship Newsletter is supported by a grant from the Open Society Institute.
This is the Free Online Scholarship Newsletter (ISSN 1535-7848).
Please feel free to forward any issue of the newsletter to interested colleagues. If you are reading a forwarded copy of this issue, you may subscribe by signing up at the FOS home page.
Peter Suber's page of related information, including the newsletter editorial position
Newsletter, archived back issues
Forum, archived postings
Conferences Related to the Open Access Movement
Timeline of the Open Access Movement
Open Access Overview
Open Access News blog
SOAN is licensed under a Creative Commons Attribution 3.0 United States License.
Return to the Newsletter archive