Kuro5hin.org: technology and culture, from the trenches
create account | help/FAQ | contact | links | search | IRC | site news
[ Everything | Diaries | Technology | Science | Culture | Politics | Media | News | Internet | Op-Ed | Fiction | Meta | MLP ]
We need your support: buy an ad | premium membership

[P]
Suggested change to moderation

By mattdm in Meta
Sat Sep 23, 2000 at 05:57:15 PM EST
Tags: Kuro5hin.org (all tags)
Kuro5hin.org

With the new "Rate All" buttons, people are finally using the moderation system. That's generally a good thing -- except it's not working well. A lot of decent comments get rated '1' apparently simply because someone disagrees. And there's a corresponding tendency to rate anything one agrees with as a '5'. Something's wrong.


The Lego community site LUGnet recently tried instituting a rating system, and it was a dismal failure. In fact, if it hadn't been fixed, it would have quickly killed all discussion. The situation isn't as bad here yet, but it's clearly broken. What to do about it? I'd suggest implementing a setup like the one LUGnet eventually arrived at: most posts get left alone, but you can also choose to "Highlight" a message as being particularly good, or "Spotlight" it as excellent. LUGnet has no negative ratings; perhaps we'd need to retain an "Offtopic" here.

I'm not advocating a bewildering array of choices a la slashdot. There's no particular reason to classify things as "Insightful" or "Funny" or whatever -- but it is useful to be able to mark something as "Should be noticed by a lot of people". LUGnet then uses these scores to generate a Spotlight Page, which shows the most-singled-out posts (using a fuzzy time vs. rating algorithm). This works amazingly well, and it'd be neat to see something like that here.

Sponsors

Voxel dot net
o Managed Hosting
o VoxCAST Content Delivery
o Raw Infrastructure

Login

Poll
Moderation good?
o +2 It's perfect as is 9%
o +1 It works ok 51%
o 0 It doesn't really add anything, but I don't care 19%
o -1 It's not workin' out like this 16%
o -2 Scrap the whole thing 2%
o -3 Troll! 1%

Votes: 152
Results | Other Polls

Related Links
o Slashdot
o LUGnet
o dismal failure
o slashdot
o Spotlight Page
o Also by mattdm


Display: Sort:
Suggested change to moderation | 93 comments (92 topical, 1 editorial, 0 hidden)
I disagree (3.61 / 31) (#1)
by Arkady on Sat Sep 23, 2000 at 03:27:33 PM EST

Hey, give the folks a chance, OK?

The Mojo stuff hasn't even been on for a week yet. It needs time for the users to get used to it before we should think about how to fine-tune it, much less replace it entirely.

Turning and turning in the widening gyre
The falcon cannot hear the falconer;
Things fall apart; the centre cannot hold;
Mere Anarchy is loosed upon the world.


Re: I disagree (2.70 / 10) (#2)
by mattdm on Sat Sep 23, 2000 at 03:51:02 PM EST

The mojo system is interesting, but it doesn't seem to address the core problem of bad comment moderation. This is why slashdot has metamoderation -- but that's obviously no solution either.

[ Parent ]
K5 moderation is metamoderation (4.25 / 8) (#13)
by kmself on Sat Sep 23, 2000 at 05:36:31 PM EST

Well, at least to the extent that every user with moderation access (and that's most of them) can voice his or her opinion.

Slash needs metamoderation to deal with the broken moderation scheme implemented. I'll post a bit from a comment I'd posted to Scoop.K5 regarding traits of a moderation system. Essentially, a Slashdot moderation largely takes into account only 5 people's votes (not strictly true, but close enough). Each person's moderation has a large impact on final score. Arbitrary limits (-1, 5) have to be implemented to keep the system from scaling open-endedly.

Scoop's system is bounded (range is 0-5) and convergent -- more moderation only servers to anchor the mean (relevant) score more solidly. A few divergent opinions disappear into the line noise. Metamoderation isn't necessary, though monitoring for consistantly contrarian moderators might be.

--
Karsten M. Self
SCO -- backgrounder on Caldera/SCO vs IBM
Support the EFF!!
There is no K5 cabal.
[ Parent ]

Re: I disagree (3.63 / 11) (#5)
by XScott on Sat Sep 23, 2000 at 04:08:11 PM EST

I disagree to your disagreement. :-)

I always go through and vote my opinion on articles in the queue. I never moderate comments on the articles.

The reasons are pretty simple:
  • There are a lot more comments than articles, and an individual comment is usually less significant.
  • It takes a lot of effort to go through every comment, and I'm lazy.
  • The numbers are pretty subjective. Without thinking about it for a bit, I'm not even sure if the ratings mean "We're #1" or "It's a 5 star comment". (I know higher is better, but you see my point.)
I like the author's suggestion. Highlight, and Spotlight are good ranks for better comments. I guess Offtopic would be appropriate sometimes, but sometimes a thread of discussion should go offtopic, so I'd hate to see that get used too strongly.

As for Mojo and being able to completely can a comment which is spam or trolling, I think that is pretty well done.


-- Of course I think I'm right. If I thought I was wrong, I'd change my mind.
[ Parent ]
Re: I disagree (1.00 / 5) (#38)
by psicE on Sat Sep 23, 2000 at 09:33:35 PM EST

How come I'm posting at (none)? I thought mojo changed my default rating... or has it not been long enough for mojo to even start... darn.

[ Parent ]
Give the system time. (3.42 / 21) (#3)
by nuntius on Sat Sep 23, 2000 at 03:56:28 PM EST

I agree that a lot of decent messages get scored as "1" -- hey, several of mine have. ;-)

However, I disagree that this is the indication of some sort of problem with the system. Whenever I see a comment which I feel was rated to low, I raise my rating of it until it reaches a level high enough to satisfy me. Likewise with comments which start out too high.

With this system, I'm expecting to start seeing a "bell-curve" phenomenon... Most controversial posts will get an average rating, bland posts will not get rated, trolls will get "1"'s or lower, and only a few things which everyone can appreciate will get high marks.

That's the way things should be--very few comments are exceptional, which is what a "5" should indicate.

Long live k5 quality!

*grin* (1.85 / 14) (#4)
by mattdm on Sat Sep 23, 2000 at 03:59:16 PM EST

Ok, we'll see how that works. :)

[ Parent ]
explanation (2.91 / 12) (#22)
by mattdm on Sat Sep 23, 2000 at 06:40:52 PM EST

In case it's now unclear: the previous comment is reference to a fact that I moderated its parent completely unfairly as a '1', to see what would happen. Looks like it's all turned out okay -- the comment did indeed get up to a reasonable 3.11. 'Course people are a bit sensitive to the issue in this particular story, so it's not a good general test.

I'm now going to go remove my '1' rating....



[ Parent ]
bug (2.66 / 9) (#23)
by mattdm on Sat Sep 23, 2000 at 06:45:43 PM EST

Huh. Just discovered a bug -- you can't change a rating back to "none".

[ Parent ]
not a bug (3.88 / 9) (#42)
by rusty on Sat Sep 23, 2000 at 10:27:39 PM EST

You can't unring a bell... You can't unbake a cake... You can't unrate a comment. This is by design-- we want more ratings, not fewer, so once you've rated, you're trapped. You can change your rating to reflect what you really think of the comment though.

____
Not the real rusty
[ Parent ]
Re: not a bug (2.20 / 5) (#53)
by mattdm on Sun Sep 24, 2000 at 12:07:55 AM EST

:) fair enough.

[ Parent ]
Re: Give the system time. (2.70 / 10) (#7)
by mattdm on Sat Sep 23, 2000 at 04:31:59 PM EST

Another thing LUGnet does which is good is give every post a base rating of "normal". This is averaged with any ratings from users. That way, it takes more than one rating for a post to jump far up or down.

[ Parent ]
Responsiveness to initial inputs, and inital score (3.81 / 11) (#15)
by kmself on Sat Sep 23, 2000 at 05:42:08 PM EST

One of the objectives of Scoop's moderation system is responsiveness to inital input. On average, we can assume that the first person to moderate a story is, well, average. If the average user is a troll, that's a problem. If they're a white hat, it's good.

This means that the first person's moderation on a post matters. A lot. It also means that late-entrant comments have a good chance of being flagged -- whether that's insightful, average, or bad. With Slash, discussions tend to die after a certain volume simply because a given comment has little chance of being moderated up. Actually, what happens is that top level discussion stagnates, while thread commentary in highly-rated TL comments propogates. This may actually be a good thing.

--
Karsten M. Self
SCO -- backgrounder on Caldera/SCO vs IBM
Support the EFF!!
There is no K5 cabal.
[ Parent ]

Additional metrics (4.25 / 12) (#21)
by kmself on Sat Sep 23, 2000 at 06:40:36 PM EST

You are pointing out a valid weakness of the current system, as it reports statistics. Median (the current "moderation" value posted) only gives the "on average, people feel x about this post". It tells you nothing about how many people cared enough to vote, or if there was strong disagreement (or agreement) among those who did.

What I'd like to see, or at least have the option to see, would be some additional statistics on comment moderation, namely:

  • n: number of times comment/story has been moderated.
  • standard deviation: the typical range of moderation. More controversial comments will tend to have middlin' means, but high standard deviations.
  • Other measures. Median would be a middle value, mode would be a most-commonly used value. Not sure if skew and kurtosis would matter, but they'd probably help some stats grad student use Scoop/K5 as a research project.

--
Karsten M. Self
SCO -- backgrounder on Caldera/SCO vs IBM
Support the EFF!!
There is no K5 cabal.
[ Parent ]

Maybe just a nudge... (2.71 / 21) (#6)
by argent on Sat Sep 23, 2000 at 04:16:44 PM EST

I don't mind the system as is. But maybe a individual should only be able to nudge a post one point in either direction. That way, I can't smush a post into the dirt, or kick it through the uprights either. Any thoughts to this?
cd /pub more Beer
Re: Maybe just a nudge... (3.12 / 8) (#27)
by Eimi on Sat Sep 23, 2000 at 07:09:33 PM EST

I'm sorry, but I think it's a very bad idea. Take /. as an example (and please note that the reason I'm against it isn't that /. does it, but that it doesn't work for them). So I see a comment in its default +1 state and say "Gee, that looks slightly funny, it's worth about +2" and mod it up. At the same time, three other ppl do the same. Now it just popped to the top of the stack. What we need is averages, and the way to overcome rogue moderation is with more moderation. Think something was unfairly moderated? Fix it.

[ Parent ]
Not a problem..yet (3.50 / 20) (#8)
by sugarman on Sat Sep 23, 2000 at 04:33:52 PM EST

I'm not sure about the rest of the populace here, but I've never felt the need to filter the comments based on rating.

Whether this is due to the lower, and more manageable, volume of comments than /. (my perosnal opinion), or due to some other factor, I'm not sure. But I have never felt it is an issue.

(As a counterpoint, I've found I need to cruise /. at +3 now, as so many of the zealots and karma-whores now appear in the +2 range by default that it makes it unreadable.)

Anyways, as I understand it, a 1 isn't necessarily a bad thing. There is no negative moderation, and I'm not sure if anyone here is actually able to use the "zero" option yet.


--sugarman--

What's that old AT&T slogan again... (3.71 / 7) (#12)
by kmself on Sat Sep 23, 2000 at 05:36:23 PM EST

..."you will".

Even with the relaunch and consequent Slashdotting, K5 hasn't seen the volume of commentary typical of other discussion boards. When traffic hits a certain level, we will need filtering capabilities.

There actually already are filters, but they're hardwired to the Mojo system -- spam rated < 1 isn't visible except to trusted users (who can reiterate the &tl; 1 vote or bump it up). Down the road, I think I'd like the opportunity to scan high-volume articles which I'm not terribly interested in at a high threshold to cherry pick. Or I might want to limit the peak threshold to look for lowball moderation abuse.

It's a matter of time.

--
Karsten M. Self
SCO -- backgrounder on Caldera/SCO vs IBM
Support the EFF!!
There is no K5 cabal.
[ Parent ]

Volume matters... (4.46 / 43) (#10)
by rusty on Sat Sep 23, 2000 at 05:00:29 PM EST

A few comments on how rating works, and how to best use it (we're working on a HOWTO for the site, by the way, and we'll try to be clear on this stuff):

The rating system relies on people using it as much as possible, first and foremost. Ideally, bad rating would be drowned out by more people rating stuff, since everyone can rate, and it's not just a +/-1 kind of thing.

The idea is that you should rate according to where on the scale you think a comment should wind up. I very frequently over or underrate things because I know that my rating will average with others to produce a certain outcome. If I see a comment rated 1, for example, that I think should be a 3, I would probably rate it five. This will average with the one and land the comment at what I think it's final score ought to be. No, this isn't considered "cheating." :-)

Assuming that enough people are rating, the ratings will converge to a reasonable value. Sometimes a comment might only get one rating, and it happens to not be one that most people would agree with. Ok, bad things happen. I don't see this as being the norm so far. In my reading, it seems that if anything, people are having a tendancy to overrate-- most articles don't have many comments below 4, and few seem to have any below 3. Then again, I don't see many comments here that are less than average, so maybe that's accurate. :-)

I do hope that people don't rate according to their opinion. I mean, rating someone down because you disagree with them is just silly. Rate according to how much thought was put into a comment, and how well it expresses what it's trying to say. For example:

Windows rules!

Linux Rules!

I would rate both of these '1'. Never mind that I agree with one and disagree with the other, neither one says anything particularly worthwhile.

And the final point is, don't put too much stock in what ratings your comments get. K5 is not the ultimate arbiter of you as a person. Ratings here, in the grand scheme of things, have no relevance outside this small closed software system. If some jerk mis-rates you, you can at least have the satisfaction of knowing that you don't do the same to others, and that ultimately, it means nothing.

I agree with those who said give it some more time, and see how it holds up. And meanwhile, the very best way to solve this is to rate more! Even if you don't normally rate comments, keep an eye out for ones which are drastically misrated, and rate those. It only works if we all do it, pretty much.

____
Not the real rusty

Re: Volume matters... (3.75 / 20) (#14)
by mattdm on Sat Sep 23, 2000 at 05:38:40 PM EST

I very frequently over or underrate things because I know that my rating will average with others to produce a certain outcome. If I see a comment rated 1, for example, that I think should be a 3, I would probably rate it five.

I'd encourage people (including you, Rusty *grin*) to not rate this way, unless you're going to go back and review your ratings later. If the system is working perfectly and everyone is rating, this just slants things weirdly.

Sometimes a comment might only get one rating, and it happens to not be one that most people would agree with.

Suggested fix: start all comments with a hidden rating of 2 or 3. That way it takes more than one vote to do something drastic to it.

I do hope that people don't rate according to their opinion. I mean, rating someone down because you disagree with them is just silly.

Unfortunately, that's what I see happening. I don't think there's a good technical fix to that, but there might be psychological ones.



[ Parent ]
Re: Volume matters... (3.25 / 12) (#19)
by nuntius on Sat Sep 23, 2000 at 06:08:34 PM EST

I like your starting rating of 2.5 idea.

That would mean that 2 "0"-wielding users would have to vote to sink a story (a good thing).

It would in general provide a weighting to keep stories from either end of the spectrum--furthering the idea that most posts are average.

[ Parent ]
Re: Volume matters... (3.25 / 4) (#84)
by ramses0 on Sun Sep 24, 2000 at 05:34:06 PM EST

I've been rated lower because somebody didn't like my opinion, and I've rated other comments low too because I didn't like their opinion.

...but... this is where the power of numbers comes in.

Rating an unrated comment to a 1 is the equivalent of blowing a virtual rasberry at somebody's post. However, if somebody else sees a lower-than-deserved rating, they should rate that comment up.

After 2 or 3 people excercise thier judgement, life is good, and the average rating should be pretty accurate. However, for those first 1 or 2 ratings, it might not be entirely accurate. Oh well. Like rusty said, it's really not that important ;^)=

--Robert
[ rate all comments , for great justice | sell.com ]
[ Parent ]

Re: Volume matters... (1.61 / 18) (#17)
by cesarb on Sat Sep 23, 2000 at 05:57:20 PM EST

You would get a 5 for the content, but got a 4 because you somehow managed to make the "Reply to This" link be in italic.

(This was just an example of a post I'd rate 2 for having almost no content...)

[ Parent ]
Quantity over Quality makes no sense. (2.62 / 16) (#34)
by Paradox on Sat Sep 23, 2000 at 08:34:39 PM EST

What you're suggesting is moderating up long winded people, because it looks like they've written more. As I said in my comment, I disagree with stuff I think is stupid, and rate it accordingly. Please explain to me how lots of people, voting down what they think is stupid and up what they think isn't, won't work. The masses will get their way, whichis what moderation is all about.

It sounds to me like a bunch of whiners are upset because they put the time into an article only to see it rated down. That's too bad. No one said using these sites is garunteed to give you a warm glow.
Dave "Paradox" Fayram

print print join q( ), split(q,q,,reverse qq;#qsti
qq)\;qlre;.q.pqevolqiqdog.);#1 reason to grin at Perl
print "\n";
[ Parent ]
Re: Quantity over Quality makes no sense. (2.45 / 11) (#47)
by mattdm on Sat Sep 23, 2000 at 11:55:36 PM EST

Are you talking about story moderation or comment moderation? I guess I didn't actually say, but my point was about the commend moderation.

[ Parent ]
T (1.84 / 13) (#48)
by mattdm on Sat Sep 23, 2000 at 11:56:28 PM EST

commenT

[ Parent ]
Re: Volume matters... (4.16 / 6) (#75)
by shaum on Sun Sep 24, 2000 at 01:57:57 PM EST

Sometimes a comment might only get one rating, and it happens to not be one that most people would agree with.
A possible remedy for this: if a comment has been rated fewer than three times, pad it with up to three "dummy" ratings with a score of 2. Thus a single rating will have a measurable effect, but won't immediately drive the comment to the top (or bottom) of the list. As real ratings roll in, they displace the dummy values, so once a comment has been rated three times, the dummy values are gone.

:wq!
[ Parent ]
Re: Volume matters... (none / 0) (#90)
by wholen1 on Mon Sep 25, 2000 at 04:21:35 PM EST

What exactly is wrong with rating something 1 that I, myself, do not agree with? I mean the whole idea (at least as I look at it from an unbiased view) is to determine which comments most accurately state what I agree/disagree with. I have no problem rating the 'opposing' point of view at a 5 if it is a valid argument, but I would rate it a 1 if it is repetitive, feeble, or just not to my standards or liking. I would expect others to do the same with comments that they disagree with, even if stated eloquently.

It seems that the consensus is to rate something based on thought, but could it not be just as accurate to rate something based on wether one judge's that comment to be a 'good' or 'bad' point of view? This isn't rocket science, and something that would really increase the ratings could possibly be having the default sort changed to: Unrated, then lowest, newest first. Let's face it, you might come in the middle of the conversation and have something very enlightening to say, but since I have already wasted an hour of my time reading the 5's and going, 'hell-yeah' you might as well be jumping off a cliff and writing your name on the way down....
Just a thought, they don't happen too often...
There are three kinds of people, those who can count, and those who can't.

[ Parent ]
Agree - too tedious (3.43 / 23) (#11)
by Delirium on Sat Sep 23, 2000 at 05:06:31 PM EST

I agree that a change needs to be made. The current system, to work properly, would need every person to rate every single comment on the page. This is extremely tedious (even with the "rate all" button), so I (and I assume others) usually just end up rating the ones which are particularly bad (as 1) and those which are particularly good (as 4 or 5).

Perhaps a system whereby a particularly insightful comment could be rated as "above-average" or something to that effect would work better (with the accompanying "off-topic" rating as suggested). This way, most comments would be left alone, with the few really insightful ones highlighted at the top and the (hopefully) few off-topic ones relegated to the end.

Re: Agree - too tedious (3.54 / 11) (#18)
by cesarb on Sat Sep 23, 2000 at 06:01:31 PM EST

Hm, nope.

I tend to rate whenever I read a comment. I do not go over a story just to moderate it -- moderation happens as a colateral effect of me reading a post. And at the end I press "Rate All".

This has the colateral effect of marking all the posts I didn't read yet as having a "none" in the moderation box...

[ Parent ]
Disagree - not tedious (4.10 / 10) (#33)
by MmmmJoel on Sat Sep 23, 2000 at 08:14:37 PM EST

Rating every comment you read is not tedious at all. It took no extra time for me to rate every comment in this thread that I read then it took to actually read it. I don't have to think hard every time; if it's a good comment, give it a good mark, and if not, don't. I feel that judging on a scale of 1-5 is not hard at all where I could imagine judging on a scale of 1-100 would be a little more thought provoking.

I like it how it is and if people would try using Rate All, things could be fine. Many people, like myself, gave up on rating posts as soon as the idea came about. Rate All has changed my mind.

[ Parent ]
Re: Disagree - not tedious (3.16 / 6) (#56)
by Delirium on Sun Sep 24, 2000 at 12:16:38 AM EST

I find judging on a scale of 1-5 to be less intuitive than you see to. Sometimes I see comments that are good, so I think "5", and sometimes I see off-topic or just pointless comments to which I think "1", but everything else is just kinda in between - on-topic good comments, but not eye-catchingly insightful or full of important information. In fact, I'd consider the majority of comments I've seen on this site so far to be "good" ones, with a few bad and a few excellent ones thrown in. Perhaps I should just moderate everything that's not a 1 and not a 5 as a 4 and get it over with.

What I'm driving at is that I don't really see the comments as having five gradations of quality - more like three.

[ Parent ]

Ok, perhaps not that tedious (none / 0) (#91)
by Delirium on Mon Sep 25, 2000 at 11:33:33 PM EST

Well, after trying it out for another few days, I've gotten used to moderating every comment I read, and it's really not as tedious as it had seemed at first. I suppose it's a good system after all. =)

[ Parent ]
Sign (3.58 / 29) (#16)
by Inoshiro on Sat Sep 23, 2000 at 05:53:55 PM EST

If some wanks are setting things to 1.0 and 5.0 without reason, give the same comments the appropriate rating to bring it in line with the fairness. It only works if everyone does it, so stop complaining about the odd weirdo and rate them all. That's how it works :-P



--
[ イノシロ ]
Oh, oops. (2.55 / 18) (#20)
by goosedaemon on Sat Sep 23, 2000 at 06:28:43 PM EST

I thought it was like, plus so many. So I rated most comments I thought were moderately good as 1. :p

LUGNet rating system specs? (3.50 / 14) (#24)
by kmself on Sat Sep 23, 2000 at 06:51:21 PM EST

Could the author or someone familiar with the LUGNet rating system please post a summary of the characteristics of the comment rating system? I am finding the link provided concludes that the system is bad without really describing what the system is. I'm afraid that doesn't give me much to go on.

--
Karsten M. Self
SCO -- backgrounder on Caldera/SCO vs IBM
Support the EFF!!
There is no K5 cabal.

Re: LUGNet rating system specs? (4.00 / 7) (#25)
by mattdm on Sat Sep 23, 2000 at 06:56:52 PM EST

Here's a detailed explaination of the current system.

[ Parent ]
mass opinion is not always valid ... (3.81 / 32) (#26)
by mx on Sat Sep 23, 2000 at 07:06:09 PM EST

while not a popular opinion, i do not see that the masses are always trustworthy as contributors to a moderation system (or free speech system for that matter). it seems that special interest groups (sigs) speak loudly, often hampering real free speech - that is, speech that can be spoken freely and not drowned out by the noise of opposition. really innovative thought usually comes from a small few who are not always part of the majority thought ... and in a world that seeks the approval of the masses (or the loudest), innovation in thought is often lost.

slashdot and k5 are a great example of sig based moderation systems. now don't get me wrong, i really enjoy slashdot (as i do k5) ... but the opinions that get through the moderation systems are quite slanted towards a number of special interests:

  • linux rules
  • windoze sucks
  • big corps suck
  • the patent office sucks
  • everything should be free
  • (on and on)

a set of interesting topics, but not always a terribly balanced set of thoughts. it isn't bad, but not balanced. and, it often limits many other useful views from ever getting serious attention.

attempt to discuss some useful use of a Micro$oft platform/product/technology on /. and you will see how limiting a LOUD set of sig voices can be.


- mx
Re: mass opinion is not always valid ... (3.55 / 9) (#28)
by Eimi on Sat Sep 23, 2000 at 07:16:02 PM EST

The question, then, becomes what can be done about it? Do we just give up on any moderation attempts (if so, just click the little "Ignore ratings" box), or do we just do the best we can to be impartial given that? Remember, in very few cases are comments actually being hidden from ppl, and hopefully those are genuine trolls (anyone have any stats on how many comments have gone below 1?). Just because there are one hundred voices shouting the party line, there's no one clubbing dissenters (sp?).

[ Parent ]
Re: mass opinion is not always valid ... (3.70 / 10) (#30)
by mx on Sat Sep 23, 2000 at 07:36:23 PM EST

it is easier to see the faults in a large system than to know the solution. i don't think k5 needs to worry for a while, the s/n ratio has been quite acceptable so far.

what could be done for larger, noisier systems?? some sort of benevolent editorial-ship group with good sensibilities (like moderated usenet?) ... or a rating system that could evaluate down-moderated (or undermoderated) for opposing views and increase the rating (and somehow still filter out spam, zelots, etc.). the rating system could use the rating-reasoning to determine if things were unfairly down-moderated by a large group of opposing views. most of these systems though, would be falible ... and let through the noise that has rendered many of the net's open forums near useless.

the general assumption that many free-speech supporters believe though, is that concensus (sp?) is required from the masses. it is not. often the masses cannot tell their head from there ass ... let alone decide in a useful way what is useful and what isn't.


- mx
[ Parent ]
Mass opinion (4.00 / 10) (#60)
by kmself on Sun Sep 24, 2000 at 01:15:19 AM EST

As Ashleigh Brilliant says, history may not always be right, but it's got the final word.

Mass opinion doesn't have to be infallible. It has to be better than no moderation (take a look at ZDNet screamback) or unbounded moderation (eg: Slash).

A true collaborative filter based on common preferences might be a more ideal solution. But the infrastructure required for this would be far, far higher. What I would like to see is an opportunity for people to see who's voting similarly to them on articles, and be able to define their own set of "editors", essentially coming up with a custom set of content selectors. This should be less intensive than having Scoop manage the process for you.

Anyone who feels that Scoop's moderation isn't working can always toggle it off by selecting "ignore ratings".

As for controversial or polarizing topics (God vs. atheism, abortion rights, politics, OS or editor preferences, GUI wars, favorite character on the Brady Bunch...), I've posted elsewhere that metrics to identify interest and controversialty might be useful.

--
Karsten M. Self
SCO -- backgrounder on Caldera/SCO vs IBM
Support the EFF!!
There is no K5 cabal.
[ Parent ]

Re: mass opinion is not always valid ... (3.85 / 7) (#62)
by Cariset on Sun Sep 24, 2000 at 03:37:03 AM EST

This can be true, but it's not what Kuro5hin is about...

Although I speak for no one but myself, it seems to me that this site is designed to give power to mass opinion. The users of this site, in toto, determine the stories, determine the discusssion, and rate it (for whatever purpose). True, we do operate under the benevolent dictatorship of Rusty/Inoshiro/etc., but they take a very hands-off approach and generally let the community run the site. So having any other form of rating would go against the general "gestalt". You might think of it as an experiment in web democracy...

But we should also keep in mind that we're not really "the masses". I mean, sure we're tech-savvy computer users, but that's the core of Slashdot, and look what happens to discussions there. I think K5 readers are generally a subgroup of that, who happen to be more conversationally inclined than the norm... And I think that all in all, since we seem to be doing a good job of story selection and discussion, we can assume we'll do a good job of comment rating as well.

[ Parent ]

Spotlight whores? (3.68 / 19) (#29)
by fluffy grue on Sat Sep 23, 2000 at 07:23:33 PM EST

What would be the appropriate term for that? Divas?

Basically, having a 'spotlight' sort of thing will lead to the same sort of problems as Slashdot's whole moderation system to begin with.

+5, Insightful.

Two words: Signal 11.

Personally, I don't give two whits about the whole moderation system here. This place isn't big enough to need it, and I don't even have my display preferences set to care about the moderation. What good does the moderation serve, anyway? It doesn't filter out cruft (except for the non-universal 0 rating), and the only reason to moderate someone is to affect their mojo. There's got to be better ways to do that, such as Advogato's meritocracy-by-association system thingy, which is always being discussed here as perfect - so why not just use it instead of saying how much nicer it is?
--
"Is not a quine" is not a quine.
I have a master's degree in science!

[ Hug Your Trikuare ]

Re: Spotlight whores? (2.00 / 8) (#49)
by mattdm on Sat Sep 23, 2000 at 11:57:11 PM EST

If it doesn't lead to a "score" like Karma, there's less incentive.

[ Parent ]
Article offers no solution but another problem (4.00 / 26) (#31)
by Carnage4Life on Sat Sep 23, 2000 at 08:11:31 PM EST

With the new "Rate All" buttons, people are finally using the moderation system. That's generally a good thing -- except it's not working well. A lot of decent comments get rated '1' apparently simply because someone disagrees. And there's a corresponding tendency to rate anything one agrees with as a '5'. Something's wrong.

I've never understood arguments like those posed by the author of this article. They correctly identify a problem then attack the symptoms of the problem instead of the cause.

Case in point, shifting from the max rating being +5 to a slashdot/LUGnet style highlighting does nothing to change the fact that people moderate up stuff they agree with and moderate down stuff they disagree with. All that would happen if the moderation system changed is that in a few weeks we'd be reading more Meta-articles about how moderation is still broken.

The problem is not moderation in any shape or form but instead human nature. It doesn't matter if there are a few moderators a la slashdot or a lot a la kuro5hin, people will generally appreciate an opinion that reflects theirs and be hostile to an opinion that is in conflict with theirs. If there was an open way of holding people accountable for their moderations, just as we can read vote histories in article submissions then maybe people would be more careful with their moderations (or it could become like slashdot where the many tyrannize the few via meta-moderation).

In my opinion all moderation is flawed because it relies on human nature which is inherrently flawed. I personally suggest reading without scores, after all USENET has no scores and this did not alter the quality of the discussions in several groups.

Re: Article offers no solution but another problem (3.40 / 10) (#40)
by Emacs on Sat Sep 23, 2000 at 09:46:17 PM EST

If there was an open way of holding people accountable for their moderations, just as we can read vote histories in article submissions then maybe people would be more careful with their moderations (or it could become like slashdot where the many tyrannize the few via meta-moderation).

I think that a form of "trusted" moderation might actually fit the bill. You have people vote for you based upon what they know about you like Advogato and those people gain "weight" for moderating. Thus if I've been certified by many people who think I'm level headed and trustworthy for moderation, well my moderation counts for more than someone who has not been certified.

Of course this would take a little while for the cream to rise to the top and could still be biased if a band of trolls got together and certified each other, but overall it might work well.

The bottom line for me is I prefer to scroll through all the comments rather than worry about what comments are rated, but it is an interesting problem.

[ Parent ]
Re: Article offers no solution but another problem (3.10 / 10) (#41)
by zencode on Sat Sep 23, 2000 at 10:25:24 PM EST

I think you're 100% correct in saying that the problem is damned near unfixable because human nature is what it is.

Secondly, I think that certain usenet groups are decent because how many kiddies really want to dick with rec.games.go? This place is a target if there ever was one.

And Thirdly, I happen to feel that Slashdot moderation works fairly well. I set my threshold to 3, I find this keeps a great deal of interesting stuff in while filtering out only a bit of what I would otherwise want to read.

But let's face it, any moderation system is basically going to regress to a "me too" forum.


my .02
zencode

http://www.iactivist.org/jason/
[ Parent ]

Re: Article offers no solution but another problem (3.22 / 9) (#51)
by mattdm on Sun Sep 24, 2000 at 12:02:38 AM EST

LUGnet highlighting and Slashdot moderation are very different. The LUGnet system was in fact designed with this in mind -- there is no way to mark something down at all, and the highlighting is exactly that -- highlighting. Unlike Slashdot, it doesn't give the poster a reward, and it doesn't define the post as "interesting" or "insightful" or whatever. This is psychologically different from moderation. Perhaps the people in LUGnet don't have human natures, but it seems to be working there.

[ Parent ]
USENET & Scores (3.57 / 7) (#63)
by matthead on Sun Sep 24, 2000 at 03:47:20 AM EST

I personally suggest reading without scores, after all USENET has no scores and this did not alter the quality of the discussions in several groups.

I agree, mostly. Moderation is useful, however, for filtering out flames, trolls, and such. I don't notice any spam here, but maybe it's just all deleted before I get here.

Anyhow, I do browse without sorting or filtering based on scores. I still moderate, however, because I respect those who do wish to censor some content.


--
- Matt
I'm at (0.3, -2.5). Where are you?
[ Parent ]
Spam on K5 (?) (5.00 / 1) (#88)
by Aquarius on Mon Sep 25, 2000 at 03:10:06 AM EST

Web discussion boards don't generally get spammed (in the sense of UBE)[1], I wouldn't have said. However, spam ought to be moderated to a below-1 score by mojo-ed moderators. Additionally, there is now no anonymous posting, which means that people posting a UCE can be LARTed somehow.

As a further point, anyone posting a UCE would probably just have it moved to the MLP section and voted on ;-)

Aq.

[1] ...although this is a possible area of future expansion for spammers...?


"The grand plan that is Aquarius proceeds apace" -- Ronin, Frank Miller
[ Parent ]
Re: Article offers no solution but another problem (none / 0) (#94)
by Mitheral on Wed Sep 27, 2000 at 11:48:33 AM EST

I personally suggest reading without scores, after all USENET has no scores and this did not alter the quality of the discussions in several groups.

And the vast majority of unmoderated groups are IMHO totally useless because of the vast quanities of UCE, spam, untalented or lame trolls, Jihads, and other noise. Mind you, it's been a few years since I actively read any unmoderated group; however experience tells me Usenet will have gotten worse not better.

I ignore the scores on K5; however, I also believe that the only reason that is possible is because of the active stance taken to rate inappropriate posts to 0 or to delete them all together

At some point, if a group/community continues to increase in popularity, the group attracts the kind of vandals whose whole reason for being seems to be to disrupt the communication of others. If they are unchecked then most of the group leaders leave and what is left is a shadow of the former.

rec.humor, soc.couples.weddings, and rec.photo.* went this way; The other site is well on it way and K5 would quickly go there if all moderation were abandoned.

[ Parent ]

Why is this a bad thing? (3.52 / 25) (#32)
by Paradox on Sat Sep 23, 2000 at 08:12:12 PM EST

I don't understand this, I never have. Why is the fact that I rate something down I think is crap BAD? What if someone says, "Linux sucks. It's worthless and slow." Then let's say he went on to make a huge argument for IRIX and all the wonder of IRIX. It could be a big, well-written criticism, but I think it's bullshit. I don't like IRIX. I've used it and it sucks. So I rate it down. Regardless of the presentation, the content itself annoyed me, so I rated it down.

Now if the moderation system is set up right, it should take this into account. Then, as everyone moderates, the most germae comments, the ones the talk about what people want to hear about, float up. Sure you may lose one or two, but nothing is perfect.

If I think someone is wrong, I'm going to rate them down. I don't feel any shame here. It's my prerogative as someone who CAN moderate to do so however I like within the rules. I can't understand why people think this is bad.
Dave "Paradox" Fayram

print print join q( ), split(q,q,,reverse qq;#qsti
qq)\;qlre;.q.pqevolqiqdog.);#1 reason to grin at Perl
print "\n";
Re: Why is this a bad thing? (4.23 / 13) (#35)
by Mr. Penguin on Sat Sep 23, 2000 at 08:49:22 PM EST

That's not the idea at all. If you disagree with what someone says, then argue (read: discuss) it with that person by replying to the comment. Moderation is for scoring down comments that shouldn't be where they are, whether they are offtopic, redundant, or trolling (I hate to use the /. terminology here, but those are the best adjectives available).

I'm not rating your comment down, I'm replying to tell you the reasons why I don't agree with you. That's the way you should play the game.



[ Parent ]
Re: Why is this a bad thing? (4.40 / 10) (#43)
by interiot on Sat Sep 23, 2000 at 11:12:31 PM EST

1) I agree. It's a very hollow feeling when you spend a lot of time trying to articulate your argument, reread the story to make sure you're on topic, make sure you're not being inflamatory, make sure you backed up all your points, etc... only to have the post marked down because the moderator didn't want to take the time to articulate the problems with your argument.

2) IMHO, the purpose of moderation is to: 2a) keep the S/N ratio high enough that the community isn't impacted (the horrible stuff is usually deleted anyway) and to 2b) make it easy for people to skim the comments if they don't have much time (this is more true of Slashdot. Here, there are usually few enough articles that you can read them all). So I don't think moderation of comments is useful here on K5.

3) Okay, maybe moderation is useful on K5. If so, then according to my previous points, it would mainly aid people who are skimming (those who arenít should read comments in unsorted order **). If this is the main goal, then it would help the skimmers if posts are modded down just because most people disagree with them.

** I don't practice what I preach about reading unsorted, but it seems to be a logical consequence of my assumptions, so I'm sure I'm missing something. Perhaps it's more efficient to consider the best argument first? Perhaps Iím just lazy?

[ Parent ]

Re: Why is this a bad thing? (4.50 / 10) (#57)
by Delirium on Sun Sep 24, 2000 at 12:23:55 AM EST

The problem is that the moderation system is supposed to weed out the "noise" while keeping the "signal" - get rid of off-topic or poorly thought out comments and promote well-thought out comments with interesting content.

If you happen to disagree with that content, you should reply saying why the poster is wrong - moderating the post down just serves to attempt to suppress their point of view, rather than actually refuting it.

For example, I rated your comment a 5, since it brings up a point of view that has not been discussed here yet and supports it, but I also disagree with it, so I replied saying why. By your logic, I should've just rated the comment as a 1 since I disagree with it.

[ Parent ]

Re: Why is this a bad thing? (4.62 / 8) (#61)
by ppanon on Sun Sep 24, 2000 at 02:59:34 AM EST

Unlike the others who replied to this post, I agree with it - with a qualifier. I think the difference is whether your disagreement with the poster is objective or subjective. If you make a perfectly logical argument, but you base it on completely false-to-facts assumptions, that argument is fundamentally flawed and deserves to be rated down. If, on the other hand, your assumptions are based on valid data but I assign more weight to seemingly contradictory data. then I will probably rate it highly and then still argue with it.

There is a difference between disagreement and squashing disinformation. The problem is that few people are sufficently discriminating and well informed to be able to make that distinction.

[ Parent ]
(2.60 / 15) (#37)
by psicE on Sat Sep 23, 2000 at 09:30:05 PM EST

This looks like a good place to rant. I don't wanna make it a story 'cos it's not important enough, so here goes:

I don't know how many times I've typed an article or comment, and post it, to see it become one paragraph because I forgot to make it HTML. Why do we need to support HTML stories and comments? You can just as easily make text bold <b>this way</b> as *this way*, or italics <i>this way</i> as _this way_, and most of the other tags are simply redundant/useless ( * here's a bullet, * there's a bullet). Can we fix this?

Do we need comment moderation at all? Something like what was suggested would be good, but make it even a one-step system. If you think a post is good, click a button. After a certain amount of time after the story is posted, the comments are totaled up and either all the comments surpassing a certain rating or a certain percentage of highest-rated comments (I'm leaning towards the latter) go up to a special section, and then the comment displayer defaults to showing "acclaimed" comments, and you need to specially set it to show "normal" comments. Obviously this would be a substantially long period of time to avoid the problem of new posts not being noticed.

Along the above lines, you could remove the story moderation queue and have all submitted stories go straight to the section pages. When a certain number of comments are posted, the story goes to the front page.

Choice (4.00 / 8) (#52)
by hurstdog on Sun Sep 24, 2000 at 12:04:57 AM EST

Why do we need to support HTML stories and comments?

To give people a choice. You may not like having the ability to post HTML formatted comments, but i sure do, and I know others do too. I post almost every comment using html. If you don't want to post as html, choose "Text Formatted" from the pull down menu below the comment. Also, preview your comment. I always preview at least once, to check links and formatting. Anyway, Why should we take away the choice to post html formatted because some people don't want to be able to post as html? I say, just choose "text formatted" and preview more.

[ Parent ]

Re: Choice (2.25 / 4) (#66)
by psicE on Sun Sep 24, 2000 at 09:01:09 AM EST

I can't see one thing in the message I'm replying to that uses anything more complex than paragraphs and italics, which can easily be handled with ASCII formatting. Even hyperlinks can be autoconverted, making http://www.microsoft.com automatically link to M$. Where else do you need the HTML?

[ Parent ]
(this is a test, please ignore. no text.) (1.40 / 5) (#76)
by psicE on Sun Sep 24, 2000 at 02:57:26 PM EST



[ Parent ]
Re: Choice (4.50 / 2) (#83)
by Carnage4Life on Sun Sep 24, 2000 at 05:17:07 PM EST

I can't see one thing in the message I'm replying to that uses anything more complex than paragraphs and italics, which can easily be handled with ASCII formatting. Even hyperlinks can be autoconverted, making http://www.microsoft.com automatically link to M$. Where else do you need the HTML?

What part of post with the "Text Formatted" option don't you understand?

I can't believe you have the gall to ask that a feature that is used by lots of people should be eliminated because you are too lazy to make a choice from a pull down menu.

[ Parent ]
Re: Choice (1.00 / 1) (#85)
by psicE on Sun Sep 24, 2000 at 06:22:29 PM EST

You need to learn that I like ranting, and you should never actually listen to me. Also, that every message by me in this thread was a Plain Text message, I'm not *that* lazy. I just think that you could make Scoop simpler by removing the HTML formatting feature. Of course, I forgot, people don't like simplicity.

I also want to remove story and comment moderation, but I expect that's a bit extreme. To me, the ideal engine has all stories posted go straight to the front page, no sections, no moderation, all comments are top-level. (Although one step better than that is mailing lists :) I wish there was a way we could get a site like slashdot or kuro5hin to go nntp or mailing list, because that's a much better format for a site like this.

[ Parent ]
Re: Choice (none / 0) (#93)
by Mitheral on Wed Sep 27, 2000 at 11:09:53 AM EST

I can't see how this would be any different than a pure Usenet group. Your welcome to create your own site with ports to nntp. alt. groups are easy to create and can be created by anyone. Of course doing that opens the discussion to all the noise that exists in practically every usenet group that isn't heavily moderated.

I think you will find that some form of control is required other wise popularity is inversely proportional to usefullness once the vandals start moving in.

[ Parent ]

Re: (2.25 / 8) (#55)
by mattdm on Sun Sep 24, 2000 at 12:11:04 AM EST

hyperlinks are good. blockquotes are good.

[ Parent ]
Moderation suggestion (2.50 / 12) (#39)
by evro on Sat Sep 23, 2000 at 09:39:10 PM EST

Keep the current rules for moderating a comment down, IE, people with the high mojo or whatever it is are the only ones who can mark down stories and see down-rated stories. However, only allow people to moderate 0 or 1. a 1 means good and a 0 means nothing. then list the total +1's next to the article rather than an average. Then there is only a "normal" and a "good" rating, and the "trusted" people can mark things as bad, etc. Basically the same way it is now except ditch 2,3,4,and 5.
---
"Asking me who to follow -- don't ask me, I don't know!"
Problems with a sum rather than an average (3.85 / 7) (#59)
by Delirium on Sun Sep 24, 2000 at 12:42:09 AM EST

The problem with this system is that late input would have almost no chance of being seen. If a story has been up for two days, some comments might have 50 or 60 +1 ratings, so it would take forever for a new comment to be able to get near the top, even if it's an excellent comment. In practice, a comment's rating would correlate more to how many people had read it than to how good it was, which isn't the point of moderation. The rating system implemented in Slash has a similar problem, only in this situation it would apply to an even greater extent, since at least with Slash, moderation can only go up to +5 (i.e. only 5 people, not 50 or more, need to see the new comment to push it up to the top). The current k5 system, using an average instead of a sum, gets rid of the problem of new comments having to "catch up" to the old comments - they can do so with a single rater. Also, ratings don't correllate at all to how many people have rated the comment (as long as at least one person has), which is good.

[ Parent ]
Related complaint (3.63 / 19) (#44)
by PresJPolk on Sat Sep 23, 2000 at 11:27:42 PM EST

Where are non-serious comments supposed to fit in this scheme?

Right now, it seems like anything that tries to be humorous, or is even slightly offtopic, gets an instant 1 or 2 rating.

If there's no place for fun in k5, it's going to be a rather dull community.

Speaking for myself... (3.88 / 9) (#58)
by kmself on Sun Sep 24, 2000 at 12:34:32 AM EST

I've moderated a couple of posts which were simply humerous. I've also slammed several when I though the jokes were tired.

People will moderate the way they care to, good humor may be rewarded. Noise is likely to be stomped.

--
Karsten M. Self
SCO -- backgrounder on Caldera/SCO vs IBM
Support the EFF!!
There is no K5 cabal.
[ Parent ]

Yes, you do speak for yourself. (3.00 / 3) (#82)
by simmons75 on Sun Sep 24, 2000 at 05:00:09 PM EST

Slamming someone for what you think is a tired joke is an abuse of moderation. Keep it up and we might end up with the atrocious moderation system of Slashdot :-(

clue stick employed: one person's tired joke is another person's extreme humor.
poot!
So there.

[ Parent ]
Re: Yes, you do speak for yourself. (none / 0) (#92)
by Arkady on Tue Sep 26, 2000 at 08:27:41 PM EST

That's what moderation is, though: applying your own criteria on whether a post is good or lame and to what degree. It's entirely appropriate to rate a joke you've heard umpteen times already at one, particularly since anyone who likes it and hasn't heard it too many times yet will be rating it up.

The vote you give to an ariticle in the queue or the rating you give to a comment is simply an expression of your own criteria. You can discuss what appropriate criteria are, certainly, but I think it's inappropriate to claim, "clue stick" status for that comment (or for this one). ;-)

-robin

Turning and turning in the widening gyre
The falcon cannot hear the falconer;
Things fall apart; the centre cannot hold;
Mere Anarchy is loosed upon the world.


[ Parent ]
Suggestion for psychological solution (4.20 / 25) (#45)
by terran on Sat Sep 23, 2000 at 11:45:54 PM EST

People have commented that it's human nature to highly rate things you agree with, and that telling people not to do it that way doesn't really help. I tend to agree with this sentiment.

I do have a suggestion for a possible way of improving this somewhat, and that is to add a second box next to the moderation selection for "agreement". Essentially, this turns every single comment into a poll. People can vote for whether or not they agree with the comment in addition to moderating it based on its value and quality; an average can be displayed somewhere in the header.

The assumption here is that, realistically, there will be situations (perhaps most of the time, even) where people will have an opinion, but will not have or want to take the time to write a reply expressing it. In the best case, perhaps they will agree with the comment, but have little to add, and not want to write a lame "me too" post. If we have an appropriate way of expressing agreement or disagreement in approximately three seconds, we may be less tempted to abuse the moderation system to perform this function.

Re: Suggestion for psychological solution (1.81 / 11) (#46)
by mattdm on Sat Sep 23, 2000 at 11:53:35 PM EST

I rated this '5' because I agree with it. :) Seriously, interesting idea.

[ Parent ]
Re: Suggestion for psychological solution (3.42 / 7) (#64)
by daani on Sun Sep 24, 2000 at 04:38:46 AM EST

...add a second box next to the moderation selection for "agreement". Essentially, this turns every single comment into a poll. People can vote for whether or not they agree with the comment...

Yeah that'd be cool. It would be really interesting to see what the majority agreed and disagreed with. Kind of like an automatic "me too..." counter. Or you could have the comment author nominate the categories for the poll.

After I previewed, I realised that if there was an agree button, I could of contributed my opinion without writing this comment, since this is a bit of a "me too...". That made me smile.

[ Parent ]

Re: Suggestion for psychological solution (3.80 / 5) (#70)
by FlinkDelDinky on Sun Sep 24, 2000 at 12:18:13 PM EST

The ideas is a good one.

But (Oh you knew there's was a But a comming ) it adds a bit more complexity and, if it's got it's own little box, it'll take even more time. For that reason I don't care for it.

I think that for the most part this is a non-problem. Anytime we get /.'d we get lots of newbies but the warp they cause in K5's culture is always temporary. The newbies that stick around "get it" and become Old Ones.

At some point we're going to have to accept that we're human and incapable of absolute perfection. K5 moderation actually works pretty good.

[ Parent ]

Not just an average. Instead, show... (2.60 / 5) (#73)
by marlowe on Sun Sep 24, 2000 at 01:26:30 PM EST

counts for both approvals and disapprovals. That will give an index of how controversial a comment is. It also gives an idea of how many care.

I'm meeting that "rating tribes" proposal halfway here. This will at least show where tribalism exists.


-- The Americans are the Jews of the 21st century. Only we won't go as quietly to the gas chambers. --
[ Parent ]
Rating Tribes (3.60 / 15) (#50)
by Jah-Wren Ryel on Sat Sep 23, 2000 at 11:57:25 PM EST

Consider this alternate rating scheme:

1) Provide no guidelines as to how articles should be rated, let people grade on quality as well as personal opinion, or the phase of the moon if they feel like it.

2) Provide a way for users to see a personalized view of articles depending on how much they like the opinions of the people who have rated the article. In essence, let people give personal weightings to other raters and let those weightings influence the ultimate rating an article gets as seen by each user. So, if I like John Doe's ratings, his ratings get more weight in the final score than someone I've never heard of before.

This personalization could be implemented automatically by comparing scores. If I give an article a 5, then the system automatically bumps up the personal weighting given to everybody else who also gave the article a 5.

Re: Rating Tribes (4.22 / 9) (#54)
by Delirium on Sun Sep 24, 2000 at 12:10:32 AM EST

This seems somewhat interesting, but the implementation would be horrific - a database would have to be kept mapping every user with their agreement with every other user, and then this would be used to generate on-the-fly moderation scores. You'd need a whole server farm just to do score calculation.

You'd also have a problem with users who consistently rate up comments they agree with and rate down comments they disagree with getting a warped view of the site, as the ratings they see would be based on users who do the same thing, so they'd only see the posts they tend to agree with. Of course if they're the type of user who moderates up/down in this manner perhaps that's the result they wanted anyway.

[ Parent ]

Re: Rating Tribes (3.40 / 5) (#65)
by blight on Sun Sep 24, 2000 at 07:11:36 AM EST

There is of course the problem of cpu power required for this. This could be solved by having the readers' computers handle the calculation. Perhaps a java-applet or create a client that people could use to read this site. Maybe a plugin for netscape/ie/other browsers?


[ Parent ]
Re: Rating Tribes (2.16 / 6) (#67)
by GreatUnknown on Sun Sep 24, 2000 at 09:40:19 AM EST

ugh. I for one would not download any extra clent or plugin to be able to view a single site.


[ Parent ]
Re: Rating Tribes (3.00 / 3) (#69)
by blight on Sun Sep 24, 2000 at 12:16:13 PM EST

I'm not suggesting forcing everyone to use the plugin/client. I'm suggesting to create an alternative for people who are willing to download it to enhance their kuro5hin reading experience.


[ Parent ]
Re: Rating Tribes (2.80 / 5) (#71)
by kaatunut on Sun Sep 24, 2000 at 12:22:57 PM EST

If the calculation moves to user's machine, then likely chances of abusing the system with false calculations do too. And I agree with above for single site not being worth external apps. And Java, *pukes*.

-Kaatunut

--
there's hole up in the sky from where the angels fall to sire children that grow up too tall, there's hole down in the ground where all the dead men go down purgatory's highways that gun their souls
[ Parent ]

Re: Rating Tribes (4.33 / 3) (#74)
by blight on Sun Sep 24, 2000 at 01:33:24 PM EST

Why would the clients need to post the results of these calculations back to k5? They could just perform the calculations for just one user allowing that particular user to see the comments filtered the way he/she wants.

I would also like to add that web browser is a rather clumsy way to do message boards. I liked the way mailreaders for BBS:s worked (and still work. I just don't use them anymore).

This discussion should propably be moved to scoop.


[ Parent ]
Re: Rating Tribes (4.50 / 2) (#80)
by Jah-Wren Ryel on Sun Sep 24, 2000 at 04:28:38 PM EST

I do not forsee an exceptional computational load. Most articles are likely to be rated by far less than 1,000 readers, I'd bet closer to a couple of hundred. So, that's a couple of hundred hashed lookups plus a couple of hundred integer ops per message.

With a 700MHz cpu, with 1000 ops for a message with 200 ratings, with 100 cycles per op including memory latency and other overhead, that's a throughput of 7,000 articles per second.

[ Parent ]

Don't moderate down (3.71 / 7) (#68)
by pete on Sun Sep 24, 2000 at 10:46:35 AM EST

Personally, I disagree with the idea of moderating things down. It's almost always used incorrectly and as many others have suggested is based more on disagreement than anything else.

I'd like to see moderation have one thing: a check box. Checking the box means simply that you think others should notice it, whether it's funny, interesting, whatever. Put a cap of 5 or 10 on posts. And have absolutely no permanent effects of moderation; no karma, no mojo, etc.

Also, there's a bug in the code: I'm replying to "HASH(0x87b81e0)" right now. Not that I've ever done that in my own Perl code. ;-)


--pete


There is no such thing as "moderating down&qu (3.75 / 4) (#72)
by cesarb on Sun Sep 24, 2000 at 12:44:05 PM EST

Moderating down means you are reducing the post's score. However, kuro5hin's moderation does not work in the basis of increasing or decreasing a score; everybody says which score they think the post should be at, and the system merges them in a single score.



[ Parent ]
Re: There is no such thing as "moderating dow (2.66 / 3) (#79)
by pete on Sun Sep 24, 2000 at 04:14:49 PM EST

Well, that's just plain wrong, both in the abstract and in reality. In the abstract, '1' is taken to be 'marked down,' because of the nature of the interpretation of a range of 1 to 5.

And in reality, if a post is marked at 5 and you mark it as 1, you have marked it down. Of course I could moderate your 4 down to prove this, but I won't. ;-)


--pete


[ Parent ]
Suggested moderation (3.55 / 9) (#77)
by DragonMagic on Sun Sep 24, 2000 at 03:01:25 PM EST

Would it be possible to allow the current moderation system with the following changes?

1) Start all posts off at a default score (1 for people with low mojo, etc., and 2 for people with a higher mojo), and include a rating of at least 10 votes for that same score. Averaging out the ratings would take a lot of votes to move it up or down, and a small few won't make it dynamic enough to kill or promote it. This would enable the message to either blow away with apathy, die off with extreme scorn, or have its majesticness promoted with a multitude of plusses.

2) Allow people to use a drop down similar to submission moderations, in that they have choices like "0 Message contains no/off-topic discussion" and "0 Did not agree with post", "1 Discussion worthy", "2 Merited discussion" and "2 Agreed with post", then "3 Post is extremely worth reading". This way, you add another point total to the averaging scale. Also, make the system as high as the current points, 5, with the varying reasons for the point moderations. Then we don't have "Insightful" or "Troll" points, which often get abused on that *other* Posting System (no offense to the system itself, just the moderators).

3) If the allowance is for everyone registered to moderate except on stories they have already replied/posted to, and to stories in which they submitted, to a point that doesn't exceed 10% of total posts to that discussion allowed for moderation per person, then perhaps the abuse will be minimalized. It will never be eliminated, but this could be a start.

Anyone else have comments on this system?

1 - Bad; 5 - Good (3.00 / 7) (#78)
by CentrX on Sun Sep 24, 2000 at 03:55:45 PM EST

I think that this might fix some problems. It makes it much clearer that 5 is good and 1 is bad instead of vice versa.
-- "The price of freedom is eternal vigilance." - Thomas Jefferson
The unfortunate Slashdot effect. (3.20 / 5) (#81)
by simmons75 on Sun Sep 24, 2000 at 04:55:00 PM EST

Slashdot has this problem, and it's more or less just ignored.

The quickest way to get modded down is to mention Microsoft or Apple in a Linux/UN*X discussion. Want to get modded down on a KDE story? Put the word GNOME in your post. GNOME? Put the name KDE in your post. Think someone's an idiot? Tell them in a reply. Just plain disagree, you're sure to get modded as flamebait, a troll, get modded down to -1, etc. Would you rather moderate than simply tell people how wrong they are? Lie. Do anything to get modded up. Avoid honesty. THEN you get to vote on the validity of a comment.

Moderation stinks IMHO. I just use my eyes. If I see a "frist ps0t" or "Natalie Portman ate my balls" or whatever, I ignore it. Apparently, many people aren't smart enough to do that. :^)
poot!
So there.

(3.20 / 5) (#86)
by psicE on Sun Sep 24, 2000 at 06:23:00 PM EST

In fact, I'd leave entirely if it wasn't for the wonderful people here (who wouldn't switch.)

kuro5hin metadiscussions from the trenches (3.20 / 5) (#87)
by kraant on Mon Sep 25, 2000 at 12:11:06 AM EST

Anyone else worried about how the metadiscussions seem to be generating more discussion than any other topics?

We may end up with a reputation for constant communal naval gazing (although that perhaps isn't neccesarily a bad thing)

Heh...

Anyway... Thresholds are bad. They create a sense of disenfranchisement among people who for whatever reason are excluded. Furthermore the disenfranchised frequently turn delinquent.

Making mojo more than a way of ensuring that people who can set things to 0 are people who won't abuse it is a good way to ensure a deeply troubled group of people who feel they are being persecuted and either leave or even worse decide to "fight the man" and try to bring down the system.

What a lot of people don't seem to realise about mojo is that it doesn't mean that everyone who doesn't have mojo couldn't be trusted with deciding what gets set to 0 and what doesn't, it only implies that those who do have high mojo can.

As it stands a person with an unpopular opinion can get constantly moderated to 1 but as long as they don't create pure noise they can still be "heard". Their low mojo does not affect their ability to participate in discussions nor does it silence them.

If thresholds were implemented then a large proportion of the population some of who may actualy have something valid to say are silenced.

This is bad enough in itself since it narrows the field of discourse and creates a feedback loop which encourages certain types of speech and punishes others.

But if having mojo automaticaly moderated a persons comment higher then people will start to compete for mojo. Making the feedback that much worse.

Making a system that rates people who agree with you highly isn't much better.

An incestuous field of discourse can only create closed minds and can cause a delusional view of reality.

Exposure to a range of views makes a better person.

Do we realy want to be a bunch of narrow minded fools?
--
"kraant, open source guru" -- tumeric
Never In Our Names...

Some stats... (3.66 / 3) (#89)
by codemonkey_uk on Mon Sep 25, 2000 at 06:44:14 AM EST

Perhaps the rating system could benafit from a bit of statistical analisis, something a big more advanced than a simple average.

I'm no statistion (please excuse my spelling) but what about looking at the standard deviation, of voting, and the voting patterns of those who vote.

Thad
---
Thad
"The most savage controversies are those about matters as to which there is no good evidence either way." - Bertrand Russell
Suggested change to moderation | 93 comments (92 topical, 1 editorial, 0 hidden)
Display: Sort:

kuro5hin.org

[XML]
All trademarks and copyrights on this page are owned by their respective companies. The Rest © 2000 - Present Kuro5hin.org Inc.
See our legalese page for copyright policies. Please also read our Privacy Policy.
Kuro5hin.org is powered by Free Software, including Apache, Perl, and Linux, The Scoop Engine that runs this site is freely available, under the terms of the GPL.
Need some help? Email help@kuro5hin.org.
My heart's the long stairs.

Powered by Scoop create account | help/FAQ | mission | links | search | IRC | YOU choose the stories!