Kuro5hin.org: technology and culture, from the trenches
create account | help/FAQ | contact | links | search | IRC | site news
[ Everything | Diaries | Technology | Science | Culture | Politics | Media | News | Internet | Op-Ed | Fiction | Meta | MLP ]
We need your support: buy an ad | premium membership

[P]
Can an Open Content Encyclopedia Thrive?

By Adam Theo in Media
Mon Apr 08, 2002 at 07:39:04 AM EST
Tags: Internet (all tags)
Internet

Wikipedia is an online encyclopedia that is open for the general public to read, copy, and even modify right there on the website. There are no dedicated editors or reviewers, relying instead on the community of peers which anyone can instantly join. Can such an oddball concept survive on the Internet, even thrive and prosper, or is it doomed to failure from various reasons?


Wikipedia operates on the famous Open Source principle of "given enough eyeballs, all errors are shallow", except instead of code and fixing bugs, Wikipedia is about knowlege and correcting falsehoods. All of the articles are licensed under the GNU Free Documentation License, and are created and modified by anyone from the general public, not hired writers and editors. This is done by using what is called a "Wiki", a type of software that allows anyone to modify web pages using shorthand instead of HTML, all through their web browser. This means anyone is free (and in fact encouraged) to write and improve articles on subjects they know about. The improvements take effect immediately, and it is up to the other writers and users of the site to correct any of your mistakes, just as you should correct the mistakes of others before you. In this fashion Wikipedia attempts to spiral upwards into a high-quality encyclopedia that will give even the Britannica a serious run for its money.

That's the thesis, anyway.

There are some rules, of course, such as the Wikipedia Etiquette, the strict rules against biased articles, and being bold when it comes to changing articles. But even regardless of these firm yet light rules, can such a concept really work? There are many objections to it, among the obvious questions of

  • What is to stop a crank from replacing a perfectly good article with some complete nonsense?
  • It seems like there should be a giant "under construction" sign on every page of the website. It seems worthless as a reference. I don't see what the point is. And...
  • Wikipedia isn't selective; hence it will never be authoritative.

Geez, what's an Open Content project to do?

Sponsors

Voxel dot net
o Managed Hosting
o VoxCAST Content Delivery
o Raw Infrastructure

Login

Poll
Would you ever consider Wikipedia as a useful resource?
o Certainly! I live for stuff like this! 36%
o Yeah, as a quick reference, but nothing for class reports. 27%
o Only when it has better content. It's too sparse right now. 15%
o Never. I'd be ashamed of not using a traditional source. 8%
o Who the hell cares? Not me! 12%

Votes: 86
Results | Other Polls

Related Links
o Wikipedia
o oddball concept
o GNU Free Documentation License
o Wiki
o Britannica
o run for its money
o Wikipedia Etiquette
o rules against biased articles
o being bold when it comes to changing articles
o objections
o Also by Adam Theo


Display: Sort:
Can an Open Content Encyclopedia Thrive? | 44 comments (29 topical, 15 editorial, 0 hidden)
So what is an encyclopedia, anyway? (4.50 / 12) (#3)
by sudasana on Sat Apr 06, 2002 at 06:49:46 PM EST

IIRC, one of the things that happened during the French Revolution was that a bunch of writers got together and put together the first encyclopedia, a collection of knowledge availible to everyone. No longer would the secrets of the universe, the lastest discoveries and the history of the world be availible only to those who could afford huge libraries. All you had to do was buy this one book and the knowledge of the world could be yours.

Isn't this just applying this concept to our own time? Right now the publishing of knowledge is restricted to those who have the money to do so, either through print or highly-advertised content-bloated websites. Here's a chance for anyone with web access to play a part in the spreading of knowledge. I don't know if the details of the system are up to its lofty goal, but I think the basic concept of an open-source knowledge repository is sound.

I agree (5.00 / 1) (#4)
by Adam Theo on Sat Apr 06, 2002 at 06:57:35 PM EST

I see this as the next step in the evolution of content and knowlege for us, but wanted to get other people's opinions on the issue to gain any insight on fixing any flaws the Wikipedia model may have.

-- "A computer geek free-market socialist patriotic American Buddhist."
[ Parent ]

Insight given. (5.00 / 1) (#14)
by Sunir on Sat Apr 06, 2002 at 10:34:59 PM EST

Wikipedia isn't a model by itself. Why do you think it's called Wikipedia? Discuss.

Yes, I'm biased. ;)

"Look! You're free! Go, and be free!" and everyone hated it for that. --r
[ Parent ]

History of Encylopaedias (5.00 / 7) (#18)
by Sunir on Sun Apr 07, 2002 at 05:00:15 AM EST

If you think about it for a moment, the word "encyclopaedia" is Greek, not French. The first encyclopaedias in the Western world were Greek, but the form really took off during Roman times. Those idiot Romans couldn't find anything better to do with their time than summarize Greek learning, adding little, removing a lot.

L'Encyclopédie is another story. It was created by Diderot's clique in order to schmuck the Church and the aristocracy in preperation for Le Revolution. It was successful if you measure success by how many times you get thrown into the Bastille and how much you enrage the populace. Certainly it didn't exhibit that neutral point of view that Wikipedia aims to achieve.

Consequently, your lumping in of Wikipedia with the sordid history of political texts just doesn't rest well. For that I must put aside one of your other statements.

Wikipedia just is not an example of information being free. It's not cheap to host it. Just because it's inexpensive for you because Bomis is subsidizing the venture, that doesn't mean that it's a great experiment in publishing by the people. Even if it is protected by the GFDL, if Wikipedia runs out of money, it will die. Wikipedia is not just a big document. It's a community too.

Indeed, that makes Wikipedia very much like L'Encyclopédie. It's paid for by patronage, although this time by a company, not aristocrats who've lost their heads.

Finally, here's an encyclopaedia article from Encarta on encyclopaedias. I could think of nothing more ironic. ;)

"Look! You're free! Go, and be free!" and everyone hated it for that. --r
[ Parent ]

Wikipedia is useless (3.22 / 9) (#5)
by rickward on Sat Apr 06, 2002 at 07:14:10 PM EST

I can't search for words with less than four letters. Do you know what that means? This encyclopedia has no information about sex, war, DVD, or ink. Screw that.

The moon laughs knowingly. The moon laughs. The moon. The.

Metawiki (4.66 / 3) (#8)
by Sunir on Sat Apr 06, 2002 at 07:54:57 PM EST

sex war DVD ink

"Look! You're free! Go, and be free!" and everyone hated it for that. --r
[ Parent ]

Can't be searched, but still there (4.20 / 5) (#10)
by Adam Theo on Sat Apr 06, 2002 at 08:19:34 PM EST

Most of those cannot be found via their search engine, true. But the articles still exist. Just try typing them into the URL.

Wikipedia does need to fix that flaw in their search engine, but can you say that is the entire reason you don't like it? Come on, I was asking about the feasability of how it works, not the inability to search for "sex".

-- "A computer geek free-market socialist patriotic American Buddhist."
[ Parent ]

The search function is... under construction (5.00 / 4) (#24)
by brion on Sun Apr 07, 2002 at 09:11:34 PM EST

Blame MySQL's full-text index; words shorter than 4 characters are rejected, and very common words return *no* results instead of many. Workaround are being put into place...

Chu vi parolas Vikipedion?
[ Parent ]
What workarounds? (2.50 / 2) (#41)
by dennis on Wed Apr 10, 2002 at 10:50:57 AM EST

I've been thinking about using MySQL with fulltext search for a project. Could you give details on the workarounds?

[ Parent ]
Follow the links! ((ed & topical, +1 FP) (5.00 / 9) (#7)
by BadDoggie on Sat Apr 06, 2002 at 07:46:49 PM EST

I recall a few start-up on-line encyclopedia wanna-bes, and for various reasons, they were crap. This one didn't look a lot better at first glance, but reading through the links did change my mind about feasibility.

Granted, as the amount of content increases linearly, the maintenance needs will increase exponentially, but the methods they have in place seem to be fairly workable and scalable. At some point, however, they will have to add some extra control layers.

The more exposure, the better their chances at growing. GPL. Free (beer, speech) Information. Attempts to remove bias. What more do you want[1]?

Unlike Britannica, this encyclopedia is less likely to be biased due to national culture, character and general opinion. I still remember the first time I saw the Great Soviet Encyclopedia (in English) -- pre-1989. Admittedly, finding anything was tough since, although the articles were all well-translated, they were still in the order they'd be in with Russian spelling, but the information and presentation often provided a lot of insight and sometimes even things my schools and government didn't want me to know about, much less discuss. Different levels of knowledge and viewpoints on the same subject, and isn't that what this place is all about?

Oh, and while this comment is editorial, it's also topical. If the articles survives, I don't want to type most of this again. Now excuse me while I go be a geek and update a few pages at Wiki with information people should not have in their heads and in which fewer still will ever be interested (unless they write questions for quiz shows).

woof.

[1] I meain besides more pr0n!

Truth is stranger than fiction because fiction has to make sense.

My Experience (4.20 / 5) (#16)
by Dolohov on Sat Apr 06, 2002 at 11:26:57 PM EST

Maybe I'm just not very knowledgeable, but my experience with Wikipedia has mostly been that I read the articles, but never am able to contribute to any of them, except to correct spelling and grammar. This is, I think, not because I don't know much, but because I possess a set of skills and interests that are firmly average for the type of person who contributes to Wikipedia with any regularity.

Don't discount your contribution (4.75 / 4) (#26)
by Solus on Sun Apr 07, 2002 at 10:07:27 PM EST

Maybe I'm just not very knowledgeable, but my experience with Wikipedia has mostly been that I read the articles, but never am able to contribute to any of them, except to correct spelling and grammar.

Traditional editing is vital to any endeavor where the unwashed masses are free to post anything and everything. Readability matters - if you can help with that you're adding a lot of value, in my opinion.

[ Parent ]

My (Limited) Contribution, and Some Misc Pondering (none / 0) (#42)
by some homeless guy on Mon Apr 15, 2002 at 10:16:18 AM EST

My measly contribution was adding to the gunpowder entry: the basic chemistry & ratio, and the blurb about the "vein of explosives".

So, not all is editing... ;-)

I stopped contributing and frequenting it awhile back -- the issue of reliability, and the fact that its hosting is so bloody slow, served as major deterrents!

Oh... i just had an idea... I don't know if this was discussed before (in the article, or anywhere else), but what if there was an encyclopedia section on K5? Since there are no anonymous posts, floods of crap wouldn't be a huge concern, and if they were rated, on whether they seem factual enough to be included, or something... it could become a very nice source, after a few months of contributions... just a thought... It would be painfully small in the beginning, but earlier contributions could just be pulled from public domain info... and having contributers be named, at least puts limited responsibility on submissions

Discuss?

[ Parent ]
Another thing to make it more useful... (none / 0) (#43)
by some homeless guy on Mon Apr 15, 2002 at 10:31:37 AM EST

... would be posters having credentials, and possibly "domains", or "territories" to which they can post, based on their knowledge/background... or at least the former (having credentials/a history of contributions/degrees/etc) - that way readers would be able to know when to be especially critical/wary if they know the author is unknown/not-trusted as of yet... or something...

Just a thought

[ Parent ]
Slightly related Advogato discussion... (4.66 / 6) (#17)
by leviramsey on Sat Apr 06, 2002 at 11:58:11 PM EST

Yesterday, Radagast posted his conception of a system for common pools of free data from the various free information projects (Wikipedia, dmoz, Project Gutenberg, etc.).

With this in place, we could have free counterparts to IMDb or All Music Guide.

It's a very thought-provoking article.



Well? (4.33 / 3) (#21)
by greenrd on Sun Apr 07, 2002 at 03:21:01 PM EST

Good question. What is to stop a crank from replacing all the articles in a Wiki with nonsense?

(My conscience, yes, but not much else.)


"Capitalism is the absurd belief that the worst of men, for the worst of reasons, will somehow work for the benefit of us all." -- John Maynard Keynes

Further question (4.00 / 2) (#30)
by streetlawyer on Mon Apr 08, 2002 at 06:25:15 AM EST

How would we tell?

--
Just because things have been nonergodic so far, doesn't mean that they'll be nonergodic forever
[ Parent ]
Simple answer (4.66 / 3) (#33)
by dennis on Mon Apr 08, 2002 at 02:36:12 PM EST

Nothing. But they entire revision history is visible to all, so anyone can restore the previous version. It happens.

[ Parent ]
Important stuff (4.00 / 1) (#22)
by anthrem on Sun Apr 07, 2002 at 05:57:32 PM EST

We need to continue to develop free resources like this. Infomation wants to be free, but we need to also have a conduit by which we communicate with one another. Things like this will help to develop that kind of network of information. As for the idiots, I am confident projects of this nature can develop ways to weed them out.

Viva la causa!



Disclaimer: I am a Buddhist. I am a Social Worker. Filter all written above throught that.
Problems with Wikipedia and cranks. (4.55 / 9) (#31)
by Kasreyn on Mon Apr 08, 2002 at 01:18:45 PM EST

Cranks give themselves away. Careful revisionists are much more dangerous.

Example issue: The holocaust.

A crank will erase the article and say "There was no holocaust! The dirty Jews invented it to keep the Aryan people down!" Obviously, this sort of crap will be caught and eliminated very rapidly. Even those whose whose opinions on the Nazis and Jews vary wildly, will recognize this as a lie, and repair it (if they're good contributors).

A revisionist will be smarter. He won't erase the original article, or change it drastically. He'll change a word here, a sentence structure there, to alter the connotation and meaning of things more subtly. Most users, simply on the lookout for factual errors, will miss this sort of misrepresentation. Semantics and choice of language can really influence people more than most think.

Another kind of misrepresentation is to post a huge article full of many facts that all check out, with one extremely obscure, hard-to-check, or made-up fact used to bring it to an incorrect conclusion. For instance, inventing up an historical atrocity in some tiny village in Africa that doesn't exist. If people try to check it and can't find the village, they'll just assume they're not looking hard enough (after all, they don't exactly have a huge internet presence, those obscure African villages). When people don't know, themselves, they tend to assume that anyone who sounds like they know what they're talking about, does.

In my opinion, this kind of sabotage is much more dangerous to a project like Wikipedia. The great, widely known facts will all be filled in easily (probably have been already), while the less known facts will be much more subject to corruption by cranks and those with agendas, because not only must a person find them in the first place, then after they're changed ANOTHER person must know the data to notice the error and fix it again.

Another problem is, what if a "fact" known to the public is actually wrong? If Wikipedia were made back when everyone believed the Earth was flat, those writing a correct article about its near-spherical shape would have had it continuously "repaired" by those who think the Earth is flat ("EVERYONE knows that!" ;-). Thus, information in Wikipedia will tend to be the popularly accepted truth, rather than the hard factual truth, in cases where the issue is obscure.

Since cranks and misrepresentationists are probably at least slightly outnumbered by fair and unbiased contributors (perfectly well-meaning people can be so biased their contributions will be worthless), Wikipedia's been doing ok. But I see it dying down a bit when all the really well known stuff is documented, unless it can get a much larger base of contributors. Eventually it may fall into the rut of all new contributions merely being popular current events - and many of these contributions will be colored by media reporting.

One advantage Wikipedia has working in its favor (besides the GPL), is this: Where a publication like Britannica could be controlled entirely by biased interests with agendas, causing the entire contents to be questionable, Wikipedia cannot be controlled by a single point of view. So while there may be cranks and misrepresentationists contributing, its overall goal and direction will always be to be a free and accurate encyclopedia. In the end, with enough users, that will allow it to do so.


-Kasreyn

P.S. Of course, the real fun begins when someone writes an article about the CoS and gets Wikipedia sued. Or someone will post the DeCSS poem in an article on that topic. Anti-free-speech suits are probably going to be its greatest single threat to existence (unless lack of funding is a problem).


"Extenuating circumstance to be mentioned on Judgement Day:
We never asked to be born in the first place."

R.I.P. Kurt. You will be missed.
Editorial discussion (4.40 / 5) (#34)
by dennis on Mon Apr 08, 2002 at 02:40:55 PM EST

Each article has a discussion section, and judging by the flamewars I've seen in some of them over minor variances in phrasing, I'm guessing the wiki might not be as vulnerable to subtle sabotage as you think. Not over the long term, anyway.

The problem of common knowledge being incorrect is a real one, but I suspect ordinary encyclopedias are vulnerable to it as well. (As are newspapers, etc.)

[ Parent ]

A bigger problem (4.00 / 3) (#36)
by Solus on Mon Apr 08, 2002 at 08:41:57 PM EST

In my mind, the real problem isn't saboteurs or malicious users - those can at least be anticipated and mechanisms to counter their "contributions" put in place - but normal, well-intentioned users who aren't quite the experts they imagine themselves to be.

For example - if I go to the Met and there's a tour underway I'll sometimes tag along and listen to the tour-guide explain various things about the art in the collections. I'm willing to take the time to listen. Why? Because I believe the tour guide is an expert. Why? Because I trust the Met to make these determinations.

On the other hand, I'm NOT willing to give my attention when I can't be sure of the qualifications of the person who's talking. Why? Because concentrating is hard. Integrating new things into your brain is non-trivial, and has potential repurcussions. If somebody is analyzing some Egyptian artifact in terms of its historical context, and I listen, and then it turns out that the person didn't know what the hell she was talking about, I've just wasted a lot of time, subjected myself to pointless confusion, and integrated a bunch of questionable facts into my knowledge base.

For these reasons it's very important, at least to me, to trust the "authority." If I don't have that trust, it's all useless. Even well-meaning people can be way, way, way off. Experts can even be way off, but at least in traditional media with accountable authors and review processes there's a more rigorous attempt at verifying content.

I have no doubts that a panel of experts on Roman history could make a compelling Wiki page on Roman history. I find it unlikely that they'd be willing to put in the work required to do so.

I think Wiki is a cool idea but I don't know if a Wiki encyclopedia is the best expression of that coolness, though perhaps (and hopefully) time will prove me wrong.

[ Parent ]

Re: Problems with Wikipedia and cranks (4.50 / 2) (#39)
by ansible on Tue Apr 09, 2002 at 01:19:56 PM EST

I'm a semi-regular wikipedia contributor. I'm not too worried about the cranks and/or revisionists.

The ones you're most worried about (the subtle ones) are pretty rare, it seems. Most of the time, they can't help but let some emotion seep into their contributions, which gives them away.

I agree though, that free-speech issues are going to be the most difficult tests for wikipedia.

I'm also not worried about the lack of unbiased contributors. There is so much stuff out there, even documenting the common stuff will take a long, long time.

[ Parent ]

Computers can't validate submissions (3.75 / 4) (#32)
by pnadeau on Mon Apr 08, 2002 at 02:31:46 PM EST

I think the open source encyclopedia is a good idea, but I think that people are missing an important point about what makes open source software projects different.

The difference is that the computer validates code when it tries to execute it.

The point of an open source project is to produce programs. Those programs are executed by computers. Generally, if a submission is faulty or garbled the computer dumps core or fails in some way and other people notice it right away and fix it.

It's kind of like genetics. A gamma particle comes by and switches a base pair on a gene. That gene has to be modified in such a way for the trait to be at least not detrimental to the organism to be passed on. It will be passed on even more if it is beneficial

In an encyclopedia we don't have this kind of built in checking. The full burden of checking for errors falls on humans (who sometimes have agendas, even subconscious ones) and are notorious for being error prone, especially with details like dates etc.

Yes there can be differences of opinion in computer programs (or in the way they should be designed architechted) that can produce viable code that will pass the computer execution check, often they do need to conform to an interface though so there is still some checking done for free there.


"Can't buy what I want because it's free, can't be what they want because I'm..."  Eddie Vedder


Compilers can't validate usefulness. (4.66 / 3) (#37)
by Uberdog on Tue Apr 09, 2002 at 12:06:10 AM EST

Just because it's validated by the computer, doesn't make it useful. The community decides what programs are useful and lets the rest go away. With OSS, as with an Open Encyclopia (nee wikki), it is and will always be the community that does the quality control.

[ Parent ]
Wouldn't Wiki be better if... (5.00 / 2) (#35)
by afree87 on Mon Apr 08, 2002 at 07:52:52 PM EST

...changes to an article were voted up or down a la Kuro5hin? (Yeah, we all know this system is perfect) That way, most of the junk would be filtered out, and you could vote on which change you liked best.

Just a suggestion. As opposed to the current anarchy...
--
Ha... yeah.
revisionism is nothing new (3.00 / 1) (#38)
by blisspix on Tue Apr 09, 2002 at 12:11:51 AM EST

Those of us who use encyclopaedias and other reference resources on a daily basis knows that they are not perfect. Each editing group has its own bias, and each publication has a different subject emphasis. Let's not forget the kerfuffle over how Microsoft's Encarta revised the history of computing and other topics to suit themselves. The best way to eliminate bias is to check a range of resources that have been published by a number of different groups in different countries. I think wikipedia is a great idea to cover topics that may not otherwise make it into 'mainstream' encyclopaedias.

Everything2 (4.66 / 3) (#40)
by Urthpaw on Wed Apr 10, 2002 at 04:52:23 AM EST

Hasn't Everything2 (http://www.everything2.com) been doing this for years?

Admittedly, it can be content-light at times, but you could call it an encyclopedia, if you really pushed the definition.

you must admit though... (none / 0) (#44)
by PsychoFurryEwok on Sat Apr 20, 2002 at 11:54:07 PM EST

Everything2.com is rather...jokeish in a way at times. By now, they've passed up almost everything serious and have just started posting random internet happenings and jokes...at least half of it is now in my opinion. Although it's all relevant to geek culture, which is mostly important ;-), I find it quite silly at times. Is silly the right word to use?

[ Parent ]
Can an Open Content Encyclopedia Thrive? | 44 comments (29 topical, 15 editorial, 0 hidden)
Display: Sort:

kuro5hin.org

[XML]
All trademarks and copyrights on this page are owned by their respective companies. The Rest © 2000 - Present Kuro5hin.org Inc.
See our legalese page for copyright policies. Please also read our Privacy Policy.
Kuro5hin.org is powered by Free Software, including Apache, Perl, and Linux, The Scoop Engine that runs this site is freely available, under the terms of the GPL.
Need some help? Email help@kuro5hin.org.
My heart's the long stairs.

Powered by Scoop create account | help/FAQ | mission | links | search | IRC | YOU choose the stories!