Communications of the ACM

Viewpoint

Why You Can't Cite Wikipedia in My Class

By Neil L. Waters
Communications of the ACM, September 2007, Vol. 50 No. 9, Pages 15-17
10.1145/1284621.1284635
Comments

The case for an online open-source encyclopedia is enormously appealing. What's not to like? It gives the originators of entries a means to publish, albeit anonymously, in fields they care deeply about and provides editors the opportunity to improve, add to, and polish them, a capacity not afforded to in-print articles. Above all, open sourcing marshals legions of unpaid, eager, frequently knowledgeable volunteers, whose enormous aggregate labor and energy makes possible the creation of an entity—Wikipedia, which today boasts more than 1.6 million entries in its English edition alone—that would otherwise be far too costly and labor-intensive to see the light of day. In a sense it would have been technologically impossible just a few years ago; open sourcing is democracy in action, and Wikipedia is its most ubiquitous and accessible creation.

Yet I am a historian, schooled in the concept that scholarship requires accountability and trained in a discipline in which collaborative research is rare. The idea that the vector-sum products of tens or hundreds of anonymous collaborators could have much value is, to say the least, counterintuitive for most of us in my profession. We don't allow our students to cite printed general encyclopedias, much less open-source ones. Further, while Wikipedia compares favorably with other tertiary sources for articles in the sciences, approximately half of all entries are in some sense historical. Here the qualitative record is much spottier, with reliability decreasing in approximate proportion to distance from "hot topics" in American history [1]. For a Japan historian like me to perceive the positive side of Wikipedia requires an effort of will.

I made that effort after an innocuous series of events briefly and improbably propelled me and the history department at Middlebury College into the national, even international, spotlight. While grading a set of final examinations from my "History of Early Japan" class, I noticed that a half-dozen students had provided incorrect information about two topics—the Shimabara Rebellion of 1637–1638 and the Confucian thinker Ogyu Sorai—on which they were to write brief essays. Moreover, they used virtually identical language in doing so. A quick check on Google propelled me via popularity-driven algorithms to the Wikipedia entries on them, and there, quite plainly, was the erroneous information. To head off similar events in the future, I proposed a policy to the history department it promptly adopted: "(1) Students are responsible for the accuracy of information they provide, and they cannot point to Wikipedia or any similar source that may appear in the future to escape the consequences of errors. (2) Wikipedia is not an acceptable citation, even though it may lead one to a citable source."

The rest, as they say, is history. The Middlebury student newspaper ran a story on the new policy. That story was picked up online by The Burlington Free Press, a Vermont newspaper, which ran its own story. I was interviewed, first by Vermont radio and TV stations and newspapers, then by The New York Times, the Asahi Shimbun in Tokyo, and by radio and TV stations in Australia and throughout the U.S., culminating in a story on NBC Nightly News. Hundreds of other newspapers ran stories without interviews, based primarily on the Times article. I received dozens of phone calls, ranging from laudatory to actionably defamatory. A representative of the Wikimedia Foundation (www.wikipedia.org), the board that controls Wikipedia, stated that he agreed with the position taken by the Middlebury history department, noting that Wikipedia states in its guidelines that its contents are not suitable for academic citation, because Wikipedia is, like a print encyclopedia, a tertiary source. I repeated this information in all my subsequent interviews, but clearly the publication of the department's policy had hit a nerve, and many news outlets implied, erroneously, that the department was at war with Wikipedia itself, rather than with the uses to which students were putting it.

In the wake of my allotted 15 minutes of Andy Warhol-promised fame I have tried to figure out what all the fuss was about. There is a great deal of uneasiness about Wikipedia in the U.S., as well as in the rest of the computerized world, and a great deal of passion and energy have been spent in its defense.

It is clear to me that the good stuff is related to the bad stuff. Wikipedia owes its incredible growth to open-source editing, which is also the root of its greatest weakness. Dedicated and knowledgeable editors can and do effectively reverse the process of entropy by making entries better over time. Other editors, through ignorance, sloppy research, or, on occasion, malice or zeal, can and do introduce or perpetuate errors in fact or interpretation. The reader never knows whether the last editor was one of this latter group; most editors leave no trace save a whimsical cyber-handle.

Popular entries are less subject to enduring errors, innocent or otherwise, than the seldom-visited ones, because, as I understand it, the frequency of visits by a Wikipedia "policeman" is largely determined, once again, by algorithms that trace the number of hits and move the most popular sites to a higher priority. The same principle, I have come to realize, props up the whole of the Wiki-world. Once a critical mass of hits is reached, Google begins to guide those who consulted it to Wikipedia before all else. A new button on my version of Firefox goes directly to Wikipedia. Preferential access leads to yet more hits, generating a still higher priority in an endless loop of mutual reinforcement.

It seems to me that there is a major downside to the self-reinforcing cycle of popularity. Popularity begets ease of use, and ease of use begets the "democratization" of access to information. But all too often, democratization of access to information is equated with the democratization of the information itself, in the sense that it is subject to a vote. That last mental conflation may have origins that predate Wikipedia and indeed the whole of the Internet.

The quiz show "Family Feud" has been a fixture of daytime television for decades and is worth a quick look. Contestants are not rewarded for guessing the correct answer but rather for guessing the answer that the largest number of people have chosen as the correct answer. The show must tap into some sort of popular desire to democratize information. Validation is not conformity to verifiable facts or weighing of interpretations and evidence but conformity to popular opinion. Expertise plays practically no role at all.

Here is where all but the most hopelessly post-modernist scholars bridle. "Family Feud" is harmless enough, but most of us believe in a real, external world in which facts exist independently of popular opinion, and some interpretations of events, thoroughly grounded in disciplinary rigor and the weight of evidence, are at least more likely to be right than others that are not. I tell my students that Wikipedia is a fine place to search for a paper topic or begin the research process, but it absolutely cannot serve subsequent stages of research. Wikipedia is not the direct heir to "Family Feud," but both seem to share an element of faith—that if enough people agree on something, it is most likely so.

What can be done? The answer depends on the goal. If it is to make Wikipedia a truly authoritative source, suitable for citation, it cannot be done for any general tertiary source, including the Encyclopaedia Britannica. For an anonymous open-source encyclopedia, that goal is theoretically, as well as practically, impossible. If the goal is more modest—to make Wikipedia more reliable than it is—then it seems to me that any changes must come at the expense of its open-source nature. Some sort of accountability for editors, as well as for the originators of entries, would be a first step, and that, I think, means that editors must leave a record of their real names. A more rigorous fact-checking system might help, but are there enough volunteers to cover 1.6 million entries, or would checking be in effect reserved for popular entries?

Can one move beyond the world of cut-and-dried facts to check for logical consistency and reasonableness of interpretations in light of what is known about a particular society in a particular historical period? Can it be done without experts? If you rely on experts, do you pay them or depend on their voluntarism?

I suppose I should now go fix the Wikipedia entry for Ogyu Sorai (en.wikipedia.org/wiki/Ogyu_ Sorai). I have been waiting since January to see how long it might take for the system to correct it, which has indeed been altered slightly and is rather good overall. But the statement that Ogyu opposed the Tokugawa order is still there and still highly misleading [2]. Somehow the statement that equates the samurai with the lower class in Tokugawa Japan has escaped the editors' attention, though anyone with the slightest contact with Japanese history knows it is wrong. One down, 1.6 million to go.

Back to Top

Back to Top

The Digital Library is published by the Association for Computing Machinery. Copyright © 2007 ACM, Inc.

---

No entries found