Kuro5hin.org: technology and culture, from the trenches
create account | help/FAQ | contact | links | search | IRC | site news
[ Everything | Diaries | Technology | Science | Culture | Politics | Media | News | Internet | Op-Ed | Fiction | Meta | MLP ]
We need your support: buy an ad | premium membership

[P]
Interfaces for User-Generated Content Sites

By Mabb in Internet
Tue Mar 13, 2001 at 08:26:00 PM EST
Tags: Software (all tags)
Software

I am currently scoping a personal project which comprises a usability analysis of what I believe are the two major contenders (Scoop and Slashcode) for what I call User-Generated Content sites and I'm calling for contribitions from people who use at least one of each type of site.


At this stage, I think the analysis will be confined to a comparison of the basic look and feel, story submission/acceptance and moderation processes, as well as the categorisation of content. Variations from the "standard interface" might also prove interesting.

Your contributions may point out other worthwhile comparisons in the course of the discussion.

Please note that this is NOT a comparison between K5 and /. In fact, I would really like to hear from users of diverse Scoop and Slashcode sites and I would like to keep the discussion focussed on the user interface, as opposed to the user and/or administrative culture. Of course, the interface can be a significant determinant in the resulting user culture so I don't want to discard this aspect entirely, just please keep in mind that the issue is the interface and the user experience it generates, not the users themselves.

Oh, and if you know of any other models for this kind of site, please also add the info - I'm happy to widen the scope to include other models. I do want to stay focussed on models with peer review of content.

Of course, the finished product will be posted to K5 and any other site where there is interest in the results. I may also publish in the HCI and Usability arenas - it just depends on how it pans out.

Thanks in advance for your comments on what might prove to be an interesting usability study.

Sponsors

Voxel dot net
o Managed Hosting
o VoxCAST Content Delivery
o Raw Infrastructure

Login

Related Links
o Scoop
o Also by Mabb


Display: Sort:
Interfaces for User-Generated Content Sites | 55 comments (38 topical, 17 editorial, 0 hidden)
What's the purpose? (4.50 / 4) (#3)
by jabber on Fri Mar 09, 2001 at 10:16:09 AM EST

Homework or Thesis?
Conference publication?
Personal interest?

Seems an odd thing to do without a covert motive. Not that I mind helping out. The net is about collaboration and if I can easily help you with your research, I'm game. But I'm curious of what the driver behind the project actually is.

[TINK5C] |"Is K5 my kapusta intellectual teddy bear?"| "Yes"

Purpose (4.00 / 3) (#7)
by Mabb on Fri Mar 09, 2001 at 10:50:27 AM EST

Mostly personal interest... honing up on my usability analysis skills. I'm a web project manager with a lot of interest and a fair amount of experience in web usability, so I tend to analyse everything I surf. This idea is something I've pondered upon since I began to regularly haunt Plastic, and then K5. Incidentally, I first read about K5 on Jakob Neilsen's site . There was a comment from rusty in a discussion about content creation.

If the data turns out to be interesting, there is the possibility of academic-type publication. Not sure who might be interested until I get a feel for the issues.

I guess I'm a little bored as well, being currently out of work due to the bad turn in IT employment in Australia. Formerly a contractor, I now find that the contract market is a dead dog and there are a bunch of out of work IT contractors all applying for the same jobs. The permanent market is still pretty strong and that's where I'm looking now. Something like this impresses the kind of employer I'd like to work with.



QuiltBlog: WIP, SEX, WOW, MQ, LQS, HST...

[ Parent ]

other models for this kind of site (4.00 / 3) (#6)
by anewc2 on Fri Mar 09, 2001 at 10:49:08 AM EST

Blogs in general and Metafilter in particular.


The world's biggest fool can say the sun is shining, but that doesn't make it dark out. -- Robert Pirsig

Other open source weblog projects (4.80 / 10) (#9)
by Dries on Fri Mar 09, 2001 at 11:37:27 AM EST

For what it's worth:

1. If you want I can temporary provide you administrator access to drop.org. Drop.org is powered by drupal, a fully-featured content management/discussion engine suitable to setup a news-driven community or portal site. It's primary written by me and built after Slashcode and Scoop. You can check out the drupal page or spend some time getting used to at drop.org.

2. Quite recently we had a user posting an excellent review comparing various Slash-clones including ThatWare, PHP-Nuke, PHPWeblog and drupal.


-- Dries

I really like tweakers.net (4.00 / 4) (#13)
by B'voYpenburg on Fri Mar 09, 2001 at 02:13:56 PM EST

I really like Tweakers.net, it has forii moderators on topics and it shows activity levels in the stories (red, yellow). It is Dutch, so you might not be able to read all of it.

I suggest... (4.00 / 6) (#14)
by k5er on Fri Mar 09, 2001 at 02:50:30 PM EST

I suggest you ad PHP-Nuke to that list. I for one have tried both scoop and slash code, but PHP-Nuke is simply amazing IMHO and that is why I decided to use it on my site. I was impressed from the start. It's powerful, feature full, and written in PHP/MySQL. Slash and Scoop are great weblogs, however I think you may want to check out PHP-Nuke. Does what Slash and Scoop do plus a little more.
Long live k5, down with CNN.
PHPnuke? PHP puke, more like. (4.00 / 4) (#16)
by Holloway on Fri Mar 09, 2001 at 06:32:09 PM EST

While doing a design for Access Mozilla I came across PHPNuke. The output code is quite bad (and PHPnuke themes only control several aspects of the design and layout, not this).

Most of the problems aren't actual faults if you consider it's goals - to be completely configurable through the web. As such it has fields like "title image" (whether or not you want one... or three) and "border colour" (what if I want no border?). It makes many assumptions about the layout in order to be configurable through the web... so it's easy to use, but inflexible.

All the TABLEs use 100% width - an incorrect assumption that it has that space available (if I float a div to the right, with this 100% widthed table on the left, it will overlap over the div). It makes many many more assumptions about the code it outputs - and the code it outputs looks like something designed for Netscape 3 (lots of nested tables to achieve effects).

This was about three months ago, so they might have tidied it up since then. But if you might ever want to use your own layout in the future I wouldn't use PHPnuke or you'll end up rewriting parts of it and tracking down why it didn't close a tag or whatever. I would use bender, or scoop. Probably Bender with it's rather nice support for themes.


== Human's wear pants, if they don't wear pants they stand out in a crowd. But if a monkey didn't wear pants it would be anonymous

[ Parent ]

Depends what you want (4.33 / 3) (#22)
by rusty on Sat Mar 10, 2001 at 08:52:08 AM EST

If you want a site that looks like what PHPNuke produces, and you're not too technical, it's probably a good fit. If you want a site that you can configure some, but that will likely have tens of millions of hits per month, Slash is probably the thing for you, having been tested on an extremely high-traffic site. Scoop's niche is mainly configurability, and user involvement. If you really want to go nuts customizing your site, and you don't mind getting into a little perl and a lot of html, Scoop's the thing. You really can change nearly every aspect of the look and feel, and future versions will soon remove that "nearly". Plus, with what we euphemistically call "boxes" you can code whole new functionality into it through the web interface. For example, I added forumzilla support to scoop by adding a few Vars and templates, and coding a couple boxes (ok, there's a backend module that holds some of the code, but there's no real need for it, other than it was 90% coded already by someone else. All of that could have gone in the boxes themselves).

Eventually, almost all layout control will be done through boxes. Scoop itself will be basically a box API, providing convenient methods and objects for doing common things. Unfortunately, we're still unique enough in doing things this way that not many people really understand how it all works (and naturally, being an open source project, the docs suck). But the downside to this is that if all you want is a simple weblog, it's probably overkill.

There is no "best system" for everyone, is the major point here. Though it's interesting to see what you thought of PHPNuke. It seems to be popular, so they must be doing something right. We got an amusing email at help@k5 recently, along the lines of "K5 is the best Nuke site I've ever seen! Can you send me your themes?" or something. We all had a good chuckle.

____
Not the real rusty
[ Parent ]

Some niggles with virtually all existing systems.. (4.28 / 7) (#15)
by jd on Fri Mar 09, 2001 at 04:58:22 PM EST

The moderation & scoring systems are linear, but deal with non-linear information. I can find a post funny AND inaccurate, for example, so do I mod it up or down? In a non-linear system, the answer is simple. Mod it up along the humour axis, mod it down on the factual axis.

Two, maybe three, axes should be sufficient to give an accurate mapping for where an article or post -truly- stands in the eyes of the readers.

A related problem is the use of N-ary trees for posts. Since posts may refer to more than one other post, this should be a graph, not a tree, with potentially more than one "parent".

If you take this to -one- possible conclusion, instead of a "flat" weblog, you'd have a "space" which you could navigate. Though that may be too complex for existing systems to handle efficiently.

Linear and non (5.00 / 5) (#21)
by rusty on Sat Mar 10, 2001 at 08:39:42 AM EST

The moderation & scoring systems are linear, but deal with non-linear information.

Careful with that. Our rating system is actually very non-linear, despite the appearance. Sure, each individual can only contribute one number, but there's no explicit restriction on what exactly you're rating. Ratings are averaged, so a post that is funny and inaccurate will get the rating that the community thinks it deserves. That is, if more people value accuracy over humor, the comment will be rated lower, and vice versa if the general preference is for humor.

The thing is, there's an innumerable number of potential axes, and right now, the system can get a usable result from any number of different ratings perspectives. Adding explicit axes would have the effect of first actually reducing the overall "rating space" (by limiting it), and second, would be a usability nightmare (one rating box is bad enough, three would likely convince most people to never rate).

And finally, what exactly are the axes involved? I can think of a dozen without even pausing in my typing. If I thought about it, I'd probably come up with three dozen more. Once you start down that road, there will be an infinite number of "why can't I rate based on (X)" questions. I really don't want to start trying to fit everyone's rating preference.

So, when you rate a post, and it's funny and innaccurate, rate it based on how you think it all balances out. There is no "objective" measure that you should be conforming to. If there was, we wouldn't need ratings to begin with. Basically, that part of rating is such a complex question that I think its much better left up to the gargantuan supercomputer sitting behind each keyboard.

____
Not the real rusty
[ Parent ]

Pick an axis, any axis... (2.50 / 2) (#32)
by sugarman on Tue Mar 13, 2001 at 02:28:09 PM EST

If you want to have multiple axis', then the addition of topics (in much the same way the user sections stories) may work. It is up to the user making the rating as to what they are rating the comment on.

If it was reall funny, even though it wasn't that accurate, give it a 5 on the Funny scale. If you found the inaccuracy to be too much to ignore, rate it on that. Everyone's personal buggaboos can get addressed. Of course, the "Show Rating page could show a synopsis. "5 users rated this a 3.95 on the funny scale, 12 users gave a 1.2 for Accuracy"

Whatever. Just a thought.

--sugarman--
[ Parent ]

Correnaton rating; Linear algebra (3.50 / 2) (#48)
by plastik55 on Thu Mar 15, 2001 at 07:21:23 AM EST

"Linear" and "non-linear" are not quite the best terms... If you want to get mathematical about it, Kuro5hin's system is very linear while Slashdot's is less linear. But I think the point is that these sites do not do a good job of separating out the various reasons why people rate comments differently. So a better term is "dimensionality;" k5 and slashdot are one-dimensional and exhibit a "tyrrany of the majority" problem: if I think differently from the average (k5: mean, /.: median) person who does ratings, I will find the ratings to be useless.

As you point out, simply making the rating system explicitly multi-dimensional just muddles everything. We want to have a scheme that preserves the dimensionality of the input (the different ratings made by different users) without complicating the interface or constraining the way people think about ratings.

The answer to this is correlation rating, whcih is how "recommendation services" like the original Firefly and Amazon.com's recommendation services work.

Basically how it works is that everyone rates on a one-dimensional scale, and the backend jiggles the numbers to find out which users tend to agree on their ratings.

So if Alice, Bob, and Charlie rate the same set of comments, and Charlie always mods down (say) Tr01l4aXxX0R's posts, but Alice and Bob try to at least read the comment to find out if he has something good to say, this will result in a grater correlation between Alice and Bob's ratings than between Alice and Charlie's. So in the future, when Alice is viewing a page, Bob's ratings will count more than Charlie's.

Benefits: You can feel free to mod however you like, you'll only affect the comments seen by people who rate like you do. This also provides all the "axes" you need; if you like to see funny but inaccurate comments, you'll be paired up with other people who like funny but inaccurate comments. The potential for moderation abuses is reduced, because (say) a script that moderates certain people down consistently will not be a close neighbor to any rasonable person. So you may be able to safely get away with hiding the ratings away from public view; this will eliminate karma/mojo whoring.

Drawbacks: This method requires more computation on the server side, and requires people to moderate much more than they normally do, otherwise the dataset is not large enough to make useful inferences. It's also not clear how quickly this will work with troll-bots (the stuff thst deserves to be at -1 in /. or 0 here.); there may need to be special provisions for that purpose.

As for "linearity," well, this scheme is implemented using linear algebra (matrix math) so it's about as "linear" as it gets.

I'd write a weblog that does this, but I've got psychophysics to do. Ta-ta...
w00t!
[ Parent ]

Where to put this story? (3.00 / 2) (#18)
by Mabb on Fri Mar 09, 2001 at 09:06:09 PM EST

Many thanks to everyone for the recommendations for other models. Will save the links just in case this story doesn't make it out of the queue.

Maybe a diary would work, but I have a feeling that it will not reach as many people as a fully-fledged story. I'll put it in there anyway, just in case.

What I want to do, once the story is published, is ask a series of questions to elicit responses in a more structured way. For example:

What was the first user-generated content site that you used (and what model was it based upon)? Are you still a regular? How long have you been using it? If you've moved on to other site/s, why did you abandon the first? How would you describe your first experience of this type of site?

I might also then ask a new set of questions regarding navigation, with similar promptings...

The more I think about it, perhaps the diary is the way to go, but my concern is that a lot of users don't look at diaries much, so how would I "publicise" its presence in order to get a decent number of responses? The survey wouldn't hold much water if there is only a small core of respondents.

K5 seemed a good place to put this because of the large user base. I also intended to do the same on /. if the idea seemed to take hold here.





QuiltBlog: WIP, SEX, WOW, MQ, LQS, HST...

2 reviews (4.50 / 2) (#19)
by Dries on Sat Mar 10, 2001 at 04:25:16 AM EST

As of a few hours ago there are now 2 interesting weblog reviews, comparing various different Slash-clones:
  1. http://drop.org/story.php?id=128: PHP-Nuke, ThatWare, PHPWeblog, GeekLog and drupal
  2. http://drop.org/story.php?id=111: PHP-Nuke, ThatWare, PHPWeblog and drupal
Feel free to re-post your questions at drop.org - it might slide down the gullet far more easy then this is the case with the K5 audience. ;-)


-- Dries
[ Parent ]

Democracy (4.00 / 4) (#23)
by emmanuel.charpentier on Sat Mar 10, 2001 at 03:54:53 PM EST

Hi
you can have a look at VeniVidiVoti.
It's an attempt at democratically evolving library, used by any group of persons to write any kind of text: book, newspaper, poem...
I still don't have any server space, so you will only be able to browse static pages, but the thing is working (although slowly).

Glasscode (4.00 / 5) (#24)
by nebby on Sat Mar 10, 2001 at 06:17:39 PM EST

If you don't mind running servlets, you could try Glasscode which runs half-empty.

Half-Empty: A global community of thoughts ideas and knowledge.
I like servlets, but... (1.00 / 1) (#42)
by delmoi on Wed Mar 14, 2001 at 10:49:26 AM EST

http://glasscode.half-empty.org/servlet/LoadPage. The page I get redirected to, dosn't seem to want to work :P
--
"'argumentation' is not a word, idiot." -- thelizman
[ Parent ]
A couple more (4.75 / 4) (#27)
by Sunir on Sun Mar 11, 2001 at 07:35:04 PM EST

I didn't see references to mod_virgule which runs Advogato. Advogato is chiefly interesting for its diary-centric community (progenitor of diaries on the other web logs I believe) and Raph Levien's trust metric.

I don't know if you're interested in non-weblog media, but you might want to contrast your sample set with the wikis. Some good examples of wikis are:

UseModWiki
The two biggest UseModWikis are Wikipedia by the good folks at NuPedia (the third largest wiki in the world), and MeatballWiki, my site on which you will find a lot about "wiki philosophy". UseModWiki is probably the best mix of power and simplicity, and it is designed primarily for online communities.

WikiWiki
The first and largest wiki; not necessarily the best to jump into though.

TWiki
The most powerful officeware wiki.
FoxForumWiki
The second largest wiki in the world, and one of the more innovative ones. Also, no comparison would be complete without Usenet and mailing lists as backgrounders.

"Look! You're free! Go, and be free!" and everyone hated it for that. --r

Usenets and Mailing Lists (3.00 / 1) (#29)
by Mabb on Mon Mar 12, 2001 at 08:27:44 AM EST

... fall outside the scope of what I'm doing.

I'm interested in web-based, user-generated, peer moderated content sites, which is why I looked first at Slash and Scoop.

Many thanks for the other pointers... will have a look at them!


QuiltBlog: WIP, SEX, WOW, MQ, LQS, HST...

[ Parent ]
Another excellent Wiki-like site (4.00 / 1) (#52)
by pin0cchio on Mon Mar 19, 2001 at 09:37:50 AM EST

Everything is another excellent Wiki clone, run by Blockstackers Intergalactic (who used to run Slashdot before Andover bought it). Profuse internal links are what make Everything Everything.
lj65
[ Parent ]
Ease of use! (4.00 / 1) (#31)
by loaf on Tue Mar 13, 2001 at 04:37:20 AM EST

It's not the answer that you wanted to hear, but I chose to investigate the Scoop method of site construction because it was way easier to install and maintain than Slashcode.

Nothing to do with the user interface, just pure convenience on the part of the webmaster.

I did look at using UserFriendly's back-end, but that just seems too esoteric for my needs, YMMV.

WebBoard (4.28 / 7) (#34)
by Andrew Dvorak on Tue Mar 13, 2001 at 05:39:05 PM EST

Webboard (url:http://webfx.eae.net/webboard/) is a nifty cgi forum which uses Microsoft Internet Explorer's proprietary DHTML. It does a good job of demonstrating the amazing potential of MSIE's DHTML.

For visitors which don't have MSIE, the board generates generic html for all browsers, but take a look at it in MSIE .. good stuff!

This caused me to realize the importance of all graphical browsers completely supporting a standard DHTML.



Hmmm... why is it proprietary? (4.25 / 4) (#39)
by /dev/niall on Wed Mar 14, 2001 at 08:55:36 AM EST

It's perfectly possible to achieve the same effect using W3C standards and a minimum of browser targeting. For instance, my newsreader (coming soon!) mockup at:

http://www.kst.com/test/

Works in IE and Mozilla based browsers. Great site though! ;)
--
"compared to the other apes, my genitals are gigantic" -- TheophileEscargot
[ Parent ]

Sad... (4.25 / 4) (#46)
by russgarrett on Wed Mar 14, 2001 at 04:54:25 PM EST

The sad (and hard to admit) fact is that Microsoft are market leader in fancy HTML. Netscape/Mozilla have always lagged behind even the standards in this respect, not respecting the <DIV> tag until version 6, while sticking with the BLINK tag. (to hand it to them, they're getting better. Slowly)

It's very hard to create a good dynamic graphical interface for a web page which looks good in both browsers (although DHTMLLib at www.siteexperts.com has a good try)

[ Parent ]
Cool stuff (3.00 / 3) (#47)
by jim.fr on Thu Mar 15, 2001 at 04:07:26 AM EST

The world needs a better web interface to NNTP !

[ Parent ]
Does Webboard have moderation? (2.50 / 2) (#50)
by Mabb on Thu Mar 15, 2001 at 11:42:17 PM EST

... I couldn't see anything that looked like moderation, but I didn't register or anything...


QuiltBlog: WIP, SEX, WOW, MQ, LQS, HST...

[ Parent ]
Accounting and Moderation (3.50 / 2) (#51)
by Andrew Dvorak on Fri Mar 16, 2001 at 04:45:31 PM EST

It has no sort of accounting, aside from the administration login (to modify the forums or posts, etc, which means you needn't register to post)

It doesn't have any sort of moderation--but who needs moderation in what appears to be a simple, threaded discussion board, very similar to email. I don't. I'm not everybody, however, so my opinion may not necessarily be that of yours or others. Milage may vary. Not valid outside the 48 contiguous states of the United States of America. Void where prohibited. Subject to verification. Fraudulent entries will be prosecuted to the fullest extent possible under law (that doesn't even cover what we'll do to the person commiting the fraud!). Retail Value: The cost of one small fry.

[ Parent ]

First Question in survey (2.00 / 1) (#35)
by Mabb on Tue Mar 13, 2001 at 07:57:47 PM EST

Ok, since the story has been published now, I'll keep going :-)

Here's the first question I wanted to ask everyone, which is a bit of a backgrounder, and multi-part. Respond as appropriate for you:

A. What was the first user-generated content site that you used (and what model was it based upon)?

B.Are you still a regular?

C. How long have you been using it? of, How long did you use it before "moving on" to another site?

D. If you've moved on to other site/s, why did you abandon the first?

E. How would you describe your first experience of this type of site? (ie were you attracted to the idea of being able to contribute? Did you like the idea of being able to moderate others' comments? etc)



QuiltBlog: WIP, SEX, WOW, MQ, LQS, HST...

ok, i'll go first... (3.00 / 1) (#36)
by Luke Francl on Tue Mar 13, 2001 at 11:56:37 PM EST

A. What was the first user-generated content site that you used (and what model was it based upon)?

Slashdot, pre-user registration period.

B.Are you still a regular?

Yes, though I lurk more than I post. Same with all the sites I frequent, really.

C. How long have you been using it? of, How long did you use it before "moving on" to another site?

I've been a multiple-daily reader of Slashdot since late 1997, when I entered college.

E. How would you describe your first experience of this type of site? (ie were you attracted to the idea of being able to contribute? Did you like the idea of being able to moderate others' comments? etc)

I was attracted to Slashdot because it was a forum reporting timely news to a geeky audience. At the time, Slashdot's readership was incredibly knowledgible. The signal-to-noise ratio was great, and it was amazing to see a thread get more than 100 comments.

Now, Slashdot must seperate the wheat from the chaff using moderation. I finally registered on Slashdot when it became possible to set your viewing threshold. While I occassionally contribute, I frequently moderate (I have low but positive karma). I try to do my best to push up the good stuff. I am embarassed to say that I initally fell for Signal 11's moderation trolling (I feel a certain kinship with him as we live in the same city). After that, I would ruthlessly mod him down (and slashdot-terminal, whatever happened to him? What a moron.) whenever I had points. It didn't really work.

The question you didn't ask was: What user-driven sites do you frequent?

I regularly read Slashdot, K5, ArsDigita's forums, Hack the Planet, and Infoanarchy. I occasionally read Libertyboard and Advogato.

[ Parent ]

Pre-Web (none / 0) (#55)
by strumco on Wed Apr 11, 2001 at 11:18:11 AM EST

The idea of the "user-generated content site" pre-dates the Web. My experience may shed some light on how the Web still has some catchup to do.

In 1989 I joined CIX - a British dial-up bulletin board, run on CoSy software. At that time, it had already been running for 4 years or so.

It was CLI-driven and (7-bit) text-only. But it did have a strong and worthwhile community (most of the British computer industry was there).

Because of the high phone costs in the UK, that community developed a range of "off-line readers". These connected briefly ("blinked") to the CLI system, grabbed new messages, posted new comments and got off the line, quick. The reader (i.e. me) then read and responded to the the new material at leisure.

This made an important cultural difference to the community and its discourse; people were prepared to spend longer composing (and winnowing) their contributions, they went off and researched the answers to questions, they even killed those flame messages before they were posted.

CIX became a (fairly) civilised place, and an even more useful one.

And it's still going. The CoSy system which runs it is ancient and klunky, but the OLRs which access it have presented the material in up-to-date formats - which are user-controlled.

Now, belatedly, CIX is intending to offer a Web front-end. The problem is - can it compete with a 15-year-old CLI?

DC
http://www.strum.co.uk

DC
http://www.strum.co.uk
[ Parent ]

Collection of 19 similar sites (4.00 / 2) (#40)
by lsommerer on Wed Mar 14, 2001 at 09:53:50 AM EST

I've been collecting links to sites like these for a few months. Here's a link to the 19 that are still "in the running" for my eternally eventual community site:

19 slashdot/kuro5hin/weblogish engines

The small editorial comments on each were simply for my benifit, and should be ignored by other visitors (which I was not planning on when I started the list).

Lloyd Sommerer
http://www.gatheringspot.com

Random Musings (4.75 / 4) (#41)
by mcelrath on Wed Mar 14, 2001 at 09:58:58 AM EST

Some comments on various weblog systems, and some drawbacks I've seen in them.

None of the rating systems really work all that well. With Scoop, something that has been rated once (at 5) is more highly rated than something that has been rated 9 times (at 5) and 1 time at 4 (for an average of...4.9). Scoop doesn't allow "categories" of rating (offtopic, flamebait, etc), which I think are important. Both Scoop and Slashcode have linear rating systems (one number, 0..5), and thus, have no kind of rating for "wrong, but will generate lots of discussion". Such a thing would be rated as "interesting" by one person (because it generated discussion or made them think), and "troll" by another (because it is factually wrong).

Here's some ideas that I haven't seen done yet:

  1. Moderation that doesn't require you to reload the page. This is very important. If I can click little buttons "interesting", "wrong", "agree", "disagree", etc, and continue reading uninterrupted, I would do it for every post. Use a bit of javascript, and a GET/POST request whose response is ignored. Kuro5hin is better than slashcode in this regard (with "rate all"), but it could still be better.
  2. Lots of moderation options. Not just a linear 1..5 scale like kuro5hin, but things like "interesting", "factually wrong", "agree", "disagree". I think lots of people mod things up or down because they agree or disagree with it, but this isn't really what the moderation system should be doing.
  3. Different kinds of ratings. Not just good/bad, 1..5, but "controversial" or "informative". A controversial post could be measured by the standard deviation of the agree/disagree rating, and this could be entirely separate from its interesting/boring rating. Rather than simply "highest rated", I could select "rate by controversy, highest first", or "rate by informativeness, highest first", or "rate by troll, highest first".
  4. The ability to "veto" factually wrong information. Weblogs are great at propegating factually wrong memes. Uninformed people read them, don't know any better, and think it's true. Informed people have no way of "vetoing" factually wrong information. To figure out it's wrong you have to read through all 47 comments and try to figure out who knows what they're talking about (because factually wrong posts always generate a lot of discussion). There are lots of topics that have very few "experts" and the general populous doesn't know much about (but is interested in), and therefore, ratings by non-experts on such posts will be almost meaningless. (i.e. physics, the human genome, law) To be really successful at this you need to categorize readers by "areas of expertise", and have some way of verifying their claim of expertise is true. If this were possible, then I could classify myself as an "expert" in physics, and vote down all the claims of antigravity, with more authority/power than a "regular user". Anyone who actually is a lawyer can moderate ignorant IANAL posts through the floor. (please, oh, please)
--Bob
1^2=1; (-1)^2=1; 1^2=(-1)^2; 1=-1; 2=0; 1=0.
#1, definitely (4.50 / 2) (#44)
by bjrubble on Wed Mar 14, 2001 at 03:39:48 PM EST

Your point #1 is something that's driven me crazy since first starting on Slashdot. The only drawback is feedback -- you either pop up a window that has to be dismissed, or give no feedback at all. But I have little doubt that there would be an order of magnitude more ratings done if it didn't require reloading the page.

[ Parent ]
Responding to points (none / 0) (#53)
by kmself on Fri Mar 30, 2001 at 06:03:22 PM EST

Points:

Page reloading. K5 does fairly well on this. Javascript is broken -- it's not supported by something like 25% of browsers (either the browser doesn't support it or users disable it), which is a big chunk of change. The development model is fundamentally flawed. Better, IMO, would be to have a single submission for all actions on a page. Rusty and I have discussed this before, there's some disagreement because various actions may lead to different results. My reading though is that Scoop should treat an article page as a large form which it submits. This is particularly an issue in the submission queue where you have two distinct operations: moderating a story, and moderating comments. Doing one operation negates (and wipes) the settings to the other. This is a UI bug.

Lots of moderation options. No. See my moderation backgrouder, as well as Rusty's comments here. Explicit multidimensionality falls apart. Without explicity, the relevance of multidimensionality becomes ambiguous. Even with multidimensionality, the relevence of what to do becomes unclear. Moderating is based on the principle of asking the question "is this post worth reading". We're relying on human intelligence (and, yes, bias) to collapse the logic for us.

There are two exceptions I'd make to this rule. Troll/spam posts might be considered less as being on a continuum than as being a categorical distinction. Though, that said, the current method for dealing with them seems to work fairly well -- few trolls are rated above 1, and few trolls are posted in the first place. Likewise, few legitimate posts are rated as trolls. I would like to know all posts which have had at least one zero rating, though, rather than those which are currently moderated below 1.

The other exception is the submission queue. Unlike moderation, here the objective is to fix problems. Categorical ratings may be appropriate. Essentially:

  • Unredeamable. This story ain't worth thinking about.
  • Major rewrite. Decent idea, but needs a hell of a lot of help.
  • Minor rewrite. Fix mechanical issues.
  • Post as is. Great job.

It's also possible that specific issues might be added as "fix it" items: spelling, grammer, links/tags, facts, bias. Though ultimately this gets rather open-ended. No need for complexity where it doesn't exist.

Section. IMO this should be dictated by the moderators in queue rather than the submitter, thoughh a section suggestion might be made.

Finally, an overall score. This is partially redundant, but also serves to establish story placement (front page, not).

Different ratings. No. See above. Specifically WRT additional metrics, many of these are emergent from the moderation system itself if it's analyzed in different ways. Particularly interest (number of moderations) and controversiality (standard deviation of moderations).

Vetoing factual errors. Maybe. I'd certainly like to address this in the submission queue, but don't know that it's necessary in comments. For the most part, though, it's easy enough to follow up a post with a statement "Wrong", supported by arguments and/or links. And, sometimes, you just have to read the follow-ups and make up your own mind.

Your point about readers' areas of expertise is well taken. This is a core weakness of the existing moderation system. I'm strongly inclined toward a topical basis for mojo, moderation, and content-submission. Also better means for evaluating a particular moderator's own cluefulness. Being able to set your own preference rating for how much you do (or don't) want to credit a particular user's moderation suggestions (on a +/- scale -- there can be negative corrolations).

--
Karsten M. Self
SCO -- backgrounder on Caldera/SCO vs IBM
Support the EFF!!
There is no K5 cabal.
[ Parent ]

Dunno about these systems, but... (3.00 / 2) (#43)
by pallex on Wed Mar 14, 2001 at 01:03:16 PM EST

...i`m a programmer who knows nothing about how you`d write a website in this style (users post comments, moderation system to keep the noise down).

But i have thought about how you`d design this sort of thing.

Wouldnt one approach be to have some sort of app running at the users end, being sent data (maybe partially filtered, such as only posts scoring >n points on the moderation system), but also be able to configure the app so that you could have killfiles, and other filters :

1)mod up/down/away people you trusted/disliked automatically, sort of like a credit rating - a karma, but which we decide for ourselves.

2)hide/show unlogged in people (ac`s in slashdot),

3)pattern matching so you could filter people who use a list of words you`d never get in a serious post (but which you`d get in trolls, such as portman, nigger, links to *.cx etc)

You could do stuff like compression/encryption on the data sent from the server if you could be bothered.
The app could take incremental updates so people who checked frequently wouldnt take as much time each time they visited, as happens now.

I`d probably bother with a little app for SlashDot or Kuro5hin if it increased speed and decreased lameness. Then again, Kuro5hin doesnt seem to have a problem with trolls. Perhaps its the lack of anonymous cowards? That works both ways too.


Two thoughts (4.00 / 1) (#45)
by rusty on Wed Mar 14, 2001 at 04:38:44 PM EST

  1. Wouldn't that be usenet?
  2. ForumZilla is a start at what you have in mind for web forums, I think.


____
Not the real rusty
[ Parent ]
Deluge (4.33 / 3) (#49)
by rinkjustice on Thu Mar 15, 2001 at 07:25:25 AM EST

Not all user-generated content sites have moderation systems, although most have some sort of story acceptance guidelines so the content fits the overall theme of the site. Take HardcoreLinux, the modest site I maintain. I've always welcomed reader contributions (articles, tutorials,reviews) and the submissions have added breadth to the site and helped better reflect the Linux/Open Source community at large. HardcoreLinux has no discussion forums, no reader moderation system and no account logins or reader datamining. There's already a deluge of /. clones vying for readers and contributers, and many of them simply aren't supported. This should be a dire concern of yours Mabb - because discussion-based sites need a steady influx of incoming content to stay afloat despite the tsunami's Slashdot and kuro5hin.
I wish you luck with your project, and I'll keep an eye out for it.



Secrets of getting stronger, faster, leaner - ZerotoSuperhero

Why no usenet-like interface? (none / 0) (#54)
by gjohnson on Sat Mar 31, 2001 at 06:00:50 PM EST

I've been wondering lately why there is no usenet-like interface for these kinds of sites. The usual interface is a real pain if you want to read all of the comments. There should be an option to see only "unread" comments, for example. I think a lot could be learned from news reading software.
--
I don't care. (0)
Interfaces for User-Generated Content Sites | 55 comments (38 topical, 17 editorial, 0 hidden)
Display: Sort:

kuro5hin.org

[XML]
All trademarks and copyrights on this page are owned by their respective companies. The Rest 2000 - Present Kuro5hin.org Inc.
See our legalese page for copyright policies. Please also read our Privacy Policy.
Kuro5hin.org is powered by Free Software, including Apache, Perl, and Linux, The Scoop Engine that runs this site is freely available, under the terms of the GPL.
Need some help? Email help@kuro5hin.org.
My heart's the long stairs.

Powered by Scoop create account | help/FAQ | mission | links | search | IRC | YOU choose the stories!