Lots of digital ink has been spilled on the trustworthiness of Wikipedia, and the circumstances in which it’s appropriate to use it as a source. Much more interesting, in my view, is the opposite question: what sources does Wikipedia trust? In our age of Truthiness, sorting good information from bad may be more critical than ever. It’s for that reason that fact-checkers seem to be making a comeback. So how, exactly, does Wikipedia manage that sorting process? Do they differentiate between The New York Times and the National Review? Does the Congressional Budget Office count as more reliable than the Heritage Foundation? Than the Brookings Institution?
To try and find out, I visited Wikipedia’s Identifying Reliable Sources page. And while it didn’t answer many of my questions, I gleaned several interesting nuggets about Wikipedians’ idea of reliability. For instance:
- “In general, the more people engaged in checking facts, analyzing legal issues, and scrutinizing the writing, the more reliable the publication.”
- “When available, academic and peer-reviewed publications, scholarly monographs, and textbooks are usually the most reliable sources.”
- “Mainstream news sources are generally considered to be reliable. However, even the most reputable news outlets occasionally contain errors. Whether a specific news story is reliable for a specific fact or statement in a Wikipedia article is something that must be assessed on a case by case basis. When using news sources, care should be taken to distinguish opinion columns from news reporting.”
- “The statement that all or most scientists or scholars hold a certain view requires reliable sourcing that directly says that all or most scientists or scholars hold that view. Otherwise, individual opinions should be identified as those of particular, named sources… Stated simply, any statement in Wikipedia that academic consensus exists on a topic must be sourced rather than being based on the opinion or assessment of editors.”
- “Anyone can create a website or pay to have a book published, then claim to be an expert in a certain field. For that reason self-published media—whether books, newsletters, personal websites, open wikis, blogs, personal pages on social networking sites, Internet forum postings, or tweets—are largely not acceptable. This includes any website whose content is largely user-generated, including the Internet Movie Database, Cracked.com, CBDB.com, and so forth, with the exception of material on such sites that is labeled as originating from credentialed members of the sites’ editorial staff, rather than users.”
More than anything, I was struck by how conservative these guidelines are. Wikipedia wouldn’t trust itself, for instance, being a user-generated project. On the one hand, that should put many of its more traditionally-minded critics at ease. On the other, it offers few new ideas about reliability.
Wikipedia’s reliability guidelines raise as many questions as they answer. But I think it’s important, as we all struggle to determine what’s reliable and what’s not, to look to innovative and successful collaborative projects like Wikipedia for guidance. There may not be much new there, but it’s not a bad starting point for discussion.
One point of interest to me: not only do Wikipedians maintain significant skepticism towards the press, they specifically don’t go in for the classic reporter’s line “many economists think”.
Bonus: PolitiFact has a post outlining their fact checking system here. I wish they would have gone further in identifying how they deal with reliability of sources. Perhaps I’ll ask them more about it.