Kuro5hin.org: technology and culture, from the trenches
create account | help/FAQ | contact | links | search | IRC | site news
[ Everything | Diaries | Technology | Science | Culture | Politics | Media | News | Internet | Op-Ed | Fiction | Meta | MLP ]
We need your support: buy an ad | premium membership

[P]
kuro5hin.org's moderation process ineffective?

By dieman in News
Mon Apr 03, 2000 at 09:12:46 AM EST
Tags: Kuro5hin.org (all tags)
Kuro5hin.org

Really? How many things have you seen die from the queue in the last few weeks. I haven't seen much not posted. Do you think that the process of moderating stories is just a nice big rubber stamp process? In paticular, how could it be improved?


Also, is there a way to get to an archive of rejected stories? Is there a way we can make 'profiles' so that if X type of person that loves what you love approves a story, it appears on your news page eariler, even though its still in the submission queue.

And, can we contact slashdot on perhaps getting their submission queue data so we can make our community-filtered site? <grin>

Sponsors

Voxel dot net
o Managed Hosting
o VoxCAST Content Delivery
o Raw Infrastructure

Login

Related Links
o Slashdot
o Also by dieman


Display: Sort:
kuro5hin.org's moderation process ineffective? | 50 comments (50 topical, editorial, 0 hidden)
This has the potential to be a grea... (1.00 / 1) (#14)
by RJ11 on Mon Apr 03, 2000 at 12:07:49 AM EST

RJ11 voted 1 on this story.

This has the potential to be a great site-- become better than what slashdot used to be (like over a year ago). There just needs to be some guidelines to cut down on stupidity.

Okay... [warming up] ... (2.67 / 3) (#11)
by nascent on Mon Apr 03, 2000 at 12:19:22 AM EST

nascent voted 1 on this story.

Okay... [warming up]

  • I can't think of how you could pull this off w/out leaving it open to abuse, but a way to weight a persons vote in relation to how much they contribute to the site. I've said it before and I'll say it again... I sometimes don't post because of certain people that do nothing but bitch. My "Significant Other Unit" was Technology, Culture, and something that would have gotten people talking (what I'd like to come here for), but it got hammered.

  • Since we all read the "Moderate Submissions" anyways, how about the main page being all submitted stories in the order of their votes. Sort of like a search engine, the items that are being discussed heavily (and thus deemed worthy of reading) would float to the top and the duds would still be chatable but not have prime real estate.

  • I can see the point of posting the votes and comments when the story goes "live", but almost nobody says anything. Maybe reword the comments field to encourage them? I can't help but think people default to using this as a page to catch new stories and not really contribute - which is what I think you're trying to do.

  • If stories have a threshold, perhaps that number should gain interest (say, x 1.01/hr) so that it becomes exponentially harder to get to the main page.

  • A freshness date on the stories would be nice. Discard after 2 days? Three? A period of nobody voting?
    nascent
    http://www.intap.net/~j/
  • Re: Okay... [warming up]... (none / 0) (#28)
    by fluffy grue on Mon Apr 03, 2000 at 11:22:13 AM EST

    Hey, I was looking forward to the "signifigant other unit" story as much as you. :) I was surprised when it didn't make it through, but my impression from the moderation comments was that most of the people moderating just didn't understand what you were getting at. Which is too bad, since I thought the way you were going about getting at it was quite funny (and, of course, my comment went to the extreme of actually using your proposed situation).

    I think a lot of users don't read the moderated submissions. However, it does seem that having all those moderation comments there (like "No, don't post this crap") kind of keep people from posting more comments, and since you can't reply to comments when it's currently in moderation, that kills a LOT of discussion. Perhaps a better way to handle moderation comments would be to allow anyone who's voted full access to the story (comment posting and all), since then, even if the discussion does die, some discussion could have taken place.

    I also still disagree with Rusty's statement in the past that people who have viewed the moderation comments are then automatically "tainted" and cannot be relied upon to make for good voting. I feel that a lot of stories get moderated down because people don't realize that they do actually want to see them, and when they realize that it IS something Nifty, it's too late to change the vote. At least allowing the changing of the vote would be nice.
    --
    "Is not a quine" is not a quine.
    I have a master's degree in science!

    [ Hug Your Trikuare ]
    [ Parent ]

    Re: Okay... [warming up]... (none / 0) (#30)
    by rusty on Mon Apr 03, 2000 at 11:35:36 AM EST

    I kind of like the idea that once you've voted, you can discuss like any other story. I had thought that maybe that'd be jumping the gun a little, but hell, why not allow it?

    As for voting before you see others' comments. I still maintain that there is value in voting for yourself, before your opinion has been swayed either way. I think it makes people more conservative and careful in their voting-- at the very least because If I go all gung ho "THIS SUX!!" and vote -1, and then see that every other vote was +1 and had a really intelligent comment, I'm gonna feel like an ass. Again, it's the power of the panoptic system. You know people will see you, but you can't see them yet. :-)

    Now maybe, just maybe, you could be allowed to change your vote. I'd say you can change it once, and perhaps you could only have half a vote the second time. Maybe initial votes would be +/-2, and changed votes only +/-1. I would rather weigh "untainted" votes more heavily, at least.

    Oh, and about moderation comments carrying over. Yes, you're totally right. It's awkward and sometimes harmful, and really really needs fixing. I was gonna do it in an hour this weekend, but then damn "spring forward" and the opportunity was lost. ;-) Seriously, though, it's high on the to-do list.

    ____
    Not the real rusty
    [ Parent ]

    Re: Okay... [warming up]... (none / 0) (#43)
    by Inoshiro on Mon Apr 03, 2000 at 07:31:17 PM EST

    You should move to Saskatchewan. Land of the no-time-change ;-)

    I like the effect of not knowing what will happen before a vote, it does force people to be honest.. Although I think some pick the "safe" +1 or 0 too much. Have you thought of scoring each story on a scale of 0-5 in various catagories, and then letting people pick their own personal "score" settings + topic so they can have a biased view of certain stories? :)



    --
    [ イノシロ ]
    [ Parent ]
    Re: Okay... [warming up]... (1.00 / 1) (#29)
    by Matthew Guenther on Mon Apr 03, 2000 at 11:35:24 AM EST

    ...but a way to weight a persons vote in relation to how much they contribute to the site...

    I think this is a really bad idea. It is impossible to do this fairly and without the possibility of abuse. First, most people think of participation == posting, but on this site that's not necessarily true; with the ability to submit articles, vote on articles, read comments, post comments, and moderate comments. However how do you measure and weigh all the different forms of participation, while remaining fair to all types of users. Second, Slashdot's moderation works on similar principles (you can only moderate if you participate a lot) so you end up with a lot of stupid meaningless comments for the express purpose of getting people's "participation" rating up. I would hate to see the same thing happen here.

    Since we all read the "Moderate Submissions" anyways, how about the main page being all submitted stories in the order of their votes. Sort of like a search engine, the items that are being discussed heavily (and thus deemed worthy of reading) would float to the top and the duds would still be chatable but not have prime real estate.

    Where'd you get that from? Since we have ~600 readers and only about 20-30 ever vote regularly (from what I can see), "we"don't all read the moderate submissions page. Also, the whole point of the voting system is to have the community decide if it's worth discussing, then it gets discussed. If you want to discuss everything that gets submitted, go to usenet. :)

    I can see the point of posting the votes and comments when the story goes "live", but almost nobody says anything. Maybe reword the comments field to encourage them? I can't help but think people default to using this as a page to catch new stories and not really contribute

    I agree, I almost never comment on my votes... if I have something to say I'll wait until it's a story. However by the time the story gets to the front page most of the comments seem to have been made and discussion dropped. Most of the comments that show up during voting usually aren't of the "explain your rationale for voting your way" kind, but are about the topic themselves.

    A freshness date on the stories would be nice. Discard after 2 days? Three? A period of nobody voting?

    This is also an essential feature I think, although I would simply set a negative "deletion" threshold, and then have a negative vote applied at a certain time interval. (As a suggestion the interval should be such that if a story were just below the posting threshold when people stopped voting on it, it should take two days to disappear)

    MBG



    [ Parent ]
    Re: Okay... [warming up]... (none / 0) (#49)
    by nascent on Mon Apr 03, 2000 at 10:31:12 PM EST

    I think this is a really bad idea. It is impossible to do this fairly and without the possibility of abuse.

    Yeah, you're right. While I'll agree, I have to point out that just about anything that Rusty implements is open to abuse tho...

    Where'd you get that from? Since we have ~600 readers and only about 20-30 ever vote regularly (from what I can see), "we"don't all read the moderate submissions page.

    Yeah, but 20-30 people voting doesn't mean 20-30 reading that page. I'd guess there are 150 people or so who visit this site regularly (3+ times/wk), so that # has to be taken with a large grain of salt.


    nascent
    http://www.intap.net/~j/
    [ Parent ]

    Re: Okay... [warming up]... (none / 0) (#41)
    by rongen on Mon Apr 03, 2000 at 04:32:57 PM EST

    how about the main page being all submitted stories in the order of their votes. Sort of like a search engine, the items that are being discussed heavily (and thus deemed worthy of reading) would float to the top and the duds would still be chatable but not have prime real estate.

    I REALLY like this idea... but maybe just have a panel that has links to stories in this order while keeping the main page as is (ordered chronologically, sort-a). I am thinking of a system like the one used by themes.org for the most frequently downloaded themes... So my vote is for: keep the front page as-is, and have a sidebar with a list of stories in order of comments posted...

    This is great, a discussion about the site. Just wanted to say "kudos!" to Rusty for making this possible, and letting it happen...
    read/write http://www.prosebush.com
    [ Parent ]

    Re: Okay... [warming up]... (none / 0) (#50)
    by ramses0 on Tue Apr 04, 2000 at 12:19:03 AM EST

    Here's an idea- keep a list of 10 or so "hot" stories... that is to say: the 10 stories sorted in order of which most people have it on their hotlist.

    So... if 10 people have "kuro5hin moderation" hotlisted, and 9 people have ummm... the MS/DOJ story hotlisted, then the list would sort....

    • Kuro5hin Hotlist
    • kuro5hin moderation (50 comments, hotlisted 10 times)
    • MS/DOJ decision (5 comments, hotlisted 9 times)
    • ...

    This would blend well with a "features" box, listing the most recent features. And when you get user preferences (or slashboxes) working, then you can give users somethign to have preferences about ;^)=

    --Robert
    [ rate all comments , for great justice | sell.com ]
    [ Parent ]

    It seems some things do get dropped... (3.00 / 1) (#7)
    by shepd on Mon Apr 03, 2000 at 12:52:35 AM EST

    shepd voted 0 on this story.

    It seems some things do get dropped from the queue... The "nsync" (or was it backstreet boys) used as torture article disappeared... I like kuro5hin.org, and there aren't enough stories. I don't think we should start dropping a bunch of them just to prove "moderation works" unless we actually start getting deulged with new stuff. Just my 2 cents.

    Good questions all. I've been ponde... (2.00 / 1) (#1)
    by rusty on Mon Apr 03, 2000 at 01:30:57 AM EST

    rusty voted 1 on this story.

    Good questions all. I've been pondering this stuff myself. Actually one story today did get shot down ("A Late April Fool's Joke"), but I agree that there needs to be more "grain" to the process. Let's have your ideas!

    ____
    Not the real rusty

    Ok, I would like to see this posted... (none / 0) (#3)
    by Nyarlathotep on Mon Apr 03, 2000 at 01:31:13 AM EST

    Nyarlathotep voted -1 on this story.

    Ok, I would like to see this posted, but your just asking to have me moderate you down with that comment. Ahh.. moderation humor.. :) I don't think this site is big enough to worry too much about interetes groups, but they would be a good idea eventually.
    Campus Crusade for Cthulhu -- it found me!

    woohoo! here's my rubber stamp ...... (none / 0) (#13)
    by djzoot on Mon Apr 03, 2000 at 01:38:05 AM EST

    djzoot voted 1 on this story.

    woohoo! here's my rubber stamp ... erm

    but seriously, balancing the required percentage of yea votes is pretty tricky ... especially with the time-sensitivity of some stories ... (i want to hear rusty's thoughts ...)

    are stats kept for approval ratings of stories, the number/which stories have died from the queue, etc ... ?

    last, where is the comment moderation going to lead?
    currently, it seems useful only for review of articles during the life of their discussion, but realtively useless for pre-sorting comments on your first read ... *shrug*

    -rob
    --
    There is no K5.

    Well it may be a rubber stamp but i... (none / 0) (#6)
    by kraant on Mon Apr 03, 2000 at 02:16:21 AM EST

    kraant voted 1 on this story.

    Well it may be a rubber stamp but it sure cuts out the _realy_ bad submissions... and anyway how many people vote out of the 500? registered users15? 20?... I think it's a nice little self selecting filtering mechanism :)
    --
    "kraant, open source guru" -- tumeric
    Never In Our Names...

    /. troller! :P... (none / 0) (#9)
    by pulsar on Mon Apr 03, 2000 at 02:35:22 AM EST

    pulsar voted -1 on this story.

    /. troller! :P

    Yes, get this out in the open, and ... (5.00 / 1) (#2)
    by ramses0 on Mon Apr 03, 2000 at 02:57:30 AM EST

    ramses0 voted 1 on this story.

    Yes, get this out in the open, and flame on. I agree that a "dead stories bucket" would be nice, so people don't resubmit the same-old stories. But having the submission queue out in the open like this really makes people think about what stories should be posted.

    Personally, I'm always a little hesitant about voting -1 on a story, perhaps it should be "+1, -1, -2" instead (because apathy, imho, is a good reason to kill a story). Buuut... since Kuro5hin is still a growing site, I'm pretty sure that a +1, -1, -2 would trash at least half of the only stories that we get.

    Think your wishes through before bringing them up in public, and perhaps this might be better discussed on scoop.kuro5hin.org ;^)=

    --Robert
    [ rate all comments , for great justice | sell.com ]

    Re: Yes, get this out in the open, and ... (none / 0) (#21)
    by rusty on Mon Apr 03, 2000 at 10:47:54 AM EST

    You haven't looked at scoop.kuro5hin.org lately, have you? :-)

    ____
    Not the real rusty
    [ Parent ]
    Re: Yes, get this out in the open, and ... (none / 0) (#39)
    by ramses0 on Mon Apr 03, 2000 at 03:38:24 PM EST

    heh. i just did.

    somehow i don't think much discussion will get done on that site ;^)=

    --Robert
    [ rate all comments , for great justice | sell.com ]
    [ Parent ]

    Hey, at least here you get asked to... (none / 0) (#10)
    by pwhysall on Mon Apr 03, 2000 at 03:25:17 AM EST

    pwhysall voted 1 on this story.

    Hey, at least here you get asked to comment on the process.
    --
    Peter
    K5 Editors
    I'm going to wager that the story keeps getting dumped because it is a steaming pile of badly formatted fool-meme.
    CheeseBurgerBrown

    Let me briefly describe what I'm pl... (4.50 / 2) (#8)
    by raph on Mon Apr 03, 2000 at 04:39:28 AM EST

    raph voted 1 on this story.

    Let me briefly describe what I'm planning to do at Advogato. There's going to be a new area set up (probably called the "workshop") for posting drafts. Certified members can go into the workshop and read the current drafts. When they do, they're supposed to comment on it. They'll also get to vote, probably on a 1-10 scale. After a draft has collected some comments, the original poster has three choices: she can post it to the front page, revise it, or delete it. The average score gets posted along with the story too. If Advogato ever develops a problem of too many low-quality posts, I'll just start tightening the posting requirements, i.e. you can only post directly to the front page if you're a Master, otherwise you have to go through the workshop first and get a score of at least 5. This design is geared towards longer posts like essays, not news stories, but I can see how it would work pretty well here, as well. It avoids some of the immediate problems I see here, including:

    • Granularity of voting. -1/0/+1 is probably not fine-grained enough.
    • Speed. The kuro5hin rating system generally takes over a day for a story to work its way through the queue.
    • The integration effect. Basically, the total score is equal to the integral of the incremental score, so roughly equal to the average rate at which the score changes times the amount of time the story has been in the queue. So, in the limit, the only thing that matters is whether there are more +1 votes than -1. By contrast, averaging the votes should actually converge.
    Slashdot moderation suffers from many of the same problems, and some even worse. In particular, the rate of score changing is directly proportional to the visibility of the comment, which often has nothing to do with the quality. In any case, I think it's fabulous that people are playing with this stuff. We need more experimentation so we can figure out what really works.

    I think it is too early to pass any... (5.00 / 1) (#4)
    by Philipp on Mon Apr 03, 2000 at 05:38:10 AM EST

    Philipp voted 1 on this story.

    I think it is too early to pass any judgement: There are still simply too few stories, so people are more lenient. Most stories are quite good anyway. Having the threat of a voting mechanism already forces posters to write meaningful content.

    alias kn 'killall -9 netscape-communicator'

    Stories need to lose a certain (sma... (none / 0) (#5)
    by Imperator on Mon Apr 03, 2000 at 07:10:20 AM EST

    Imperator voted 1 on this story.

    Stories need to lose a certain (small but significant) number of points each day they're in the queue. There should also be a hard timeout on story life in the queue. If a story takes a month to garner the votes for posting, it probably doesn't deserve posting.

    Just how would we measure effective... (none / 0) (#12)
    by marlowe on Mon Apr 03, 2000 at 08:20:58 AM EST

    marlowe voted 1 on this story.

    Just how would we measure effectiveness objectively? I like that archive of rejected stories for a start. Also, how about some running statistics on the moderation engine.
    -- The Americans are the Jews of the 21st century. Only we won't go as quietly to the gas chambers. --

    Re: kuro5hin.org's moderation process ineffective? (none / 0) (#15)
    by krogoth on Mon Apr 03, 2000 at 09:33:40 AM EST

    I think the moderation system is a lot better than other similar sites, but for now, the news is a bit slow and the moderation queue is small because there aren't as many people visiting the site. Maybe a system similar to slashdot, where people get the temporary ability to moderate submissions, would be better.

    Right now, there isn't much trouble with the system, but as more people start coming, there will be people who try to annoy the regular readers, so it would have to based on the score of a user (total of moderation, something like that).
    --
    "If you've never removed your pants and climbed into a tree to swear drunkenly at stuck-up rich kids, I highly recommend it."
    :wq
    Re: kuro5hin.org's moderation process ineffective? (none / 0) (#38)
    by ramses0 on Mon Apr 03, 2000 at 03:36:44 PM EST

    You might be interested in looking at http://www.advogato.org/, which has something called "trust metrics" ... basically, your vote doesn't count as much until people "vouch" for you. Yes, it would be nice to one-day integrate the two methods (user edited, as well as "trust metrics"), but rusty is pretty busy ;^)=

    --Robert
    [ rate all comments , for great justice | sell.com ]
    [ Parent ]

    First Post! (none / 0) (#16)
    by eann on Mon Apr 03, 2000 at 10:07:48 AM EST

    Sorry, I had to say it, since I'm apparently the first person to comment who didn't catch this in the submission queue. :)

    I think that's probably an indicator that the system may be too small to know whether we're getting a good sample of what articles people want to see. I'm sure the population affects the latency between submission and posting--if not enough people go through moderation, even "breaking news" gets lagged by hours. Like most of you, if I catch it in the queue, I'll read the comments there when I've moderated. If it really interests me, I'll go back and look at it again when it's made it to the home page, to see if anyone else said anything. Usually, I'm disappointed.

    I like the idea of more granularity in the modulation. +2 to -2 (or 0 to 4 or 9 or whatever) would likely work a little better--very important items could break the magic ratio (which would likely need a little tweaking) much faster than so-so things.

    But I'm not sure the granularity should be in the same dimension or if we should look for a way to make it orthogonal. There are two variables: whether it's a worthy story, and whether you're interested in discussing it. Then, a logged-in user could get stories s/he'd marked as interesting float to the top of what he sees on the home page, or perhaps linger there longer before they're moved to the archive box. Or, with a little fancy database work, float some other stories marked by people who previously indicated interest in the same things. Some of this will probably be accomplished with sections, but I don't know if that'll be the best dividing line.

    There's a lot of potential for different ways of doing things. It's really just a matter of having enough time and/or patience to think about them, implement them somewhere, and get enough people to play with it to see if they work.

    Damn. Now I'm craving grits (for breakfast, though, not underwear).

    Our scientific power has outrun our spiritual power. We have guided missiles and misguided men. —MLK

    $email =~ s/0/o/; # The K5 cabal is out to get you.


    User-moderated stories (4.00 / 3) (#17)
    by End on Mon Apr 03, 2000 at 10:11:47 AM EST

    Well, after thinking it would be a nice idea to have moderation of stories as well as comments, I found a site (this one) that actually does it, and have been able to see how it works.

    You know what? Moderating stories as they appear in the queue takes the fun out of visiting a news site. You actually see all the news twice, first when you moderate it, second when it actually gets to the front page. It's kind of deflating; the story is old and has been commented on before it even gets to the front page.

    -JD

    Re: User-moderated stories (none / 0) (#18)
    by rusty on Mon Apr 03, 2000 at 10:16:38 AM EST

    But often there are comments you're just itching to reply to, while it's in moderation, and eventually, it goes up, and you're free to talk. I find that part fun. :-)

    Anyway, story moderation is doubtless not for everyone. Some people do it religiously, some never do it at all. As long as 4 or 5 percent of you do it often, then the whole thing works-- the people who care get to have their say about stories, and the effect should be to improve the quality for everyone else. A dud of a story can fool one editor, but can it fool 60 or 70 editors? That's really the goal of a system like this.

    ____
    Not the real rusty
    [ Parent ]

    Re: User-moderated stories (none / 0) (#22)
    by Skippy on Mon Apr 03, 2000 at 10:49:56 AM EST

    I sort of agree with the second part of this post (about seeing things twice). That having been said, I also think that the growth of Kuroshin will alleviate some of this. Right now I moderate stories religiously since I know how important it is with just a few of us right now and I usually post a comment when I do. I usually go back to the story looking for discussion but only find the moderation comments. I think this will get better as Kuroshin grows because more people who don't moderate will post discussion on the stories and I'll have someone to talk to :-) As for the story moderation system, I think it's ok if stories die. Bad stories are killed, mediocre stories die in the queue, and good stories make the front page. I really think that right now is too early to make a judgement on the efficacy of the story moderation system. I think it will work as it is now, we just need to reach a population "event horizon" where there's enough input to make it work. Some of my concerns about how it works (and they are details) are dead accounts and lurkers. A dead account registers and posts once and never comes back. That's dead weight and makes it harder to get pages to the front page. Lurkers come and read, but don't post or vote, once again making it harder to get stories to the front page. The reason I say these are details is that I think that a small adjustment in the vote percentage required to get to the front page wil fix them.
    # I am now finished talking out my ass about things that I am not qualified to discuss. #
    [ Parent ]
    Rate: -5 to +5 (2.00 / 1) (#19)
    by hgayosso on Mon Apr 03, 2000 at 10:21:29 AM EST

    I agree in that there should be a more ample range for rating. like -5 to +5
    support the Free Software!!! http://www.gnu.org
    My thoughts (4.70 / 3) (#20)
    by rusty on Mon Apr 03, 2000 at 10:47:01 AM EST

    I want to reply to a whole bunch of you, so I'm doing it all in one spot here, for easy reading...

    As probably the most hawk-like observer of the system, I have noticed some ways in which it does definitely work:

    • I don't have to spend all my time deciding which articles stay and which go. There have been some that I was very torn on, and would have made the wrong decision about, according to the popular vote. This is a huge relief to me, as an administrator, because I don't have the whole responsibility for making sure the readers are reading interesting content.
    • Stories actually get posted, and most people aren't displeased with the content so far.
    • Users can always see what's been submitted, so reposts and multiple submissions have been just about nonexistent.
    • Frequently, the need to entice readers to vote for a story prompts submissions that go deeper than just "here's a neat link" and provide some thoughts or commentary on why the story is interesting. Not all stories, mind you, but many. The overall tone tends to be less inflammatory. Whether thats a result of moderation or not, I don't really know, though.
    So that's the good. Now, there are definite weaknesses to the system as well. Such as:
    • Speed, or lack thereof. Breaking news or timely stories just don't get posted fast enough. This could be solved by having sections for different types of stories, and setting the thresholds and expire times differently. For example, in the "Discussions in Depth" section, the threshold would be high, so that articles would have to be really good to get posted, and would stick around for a while, while in "News" the threshold would be pretty low, for speed and current info. And quickies, of course, would be miniscule, cause who doesn't like a fresh quickie or two every day. :-)
    • Inconsistency: Right now, there are around 600 active user accounts, and about 650 that factor into the scoring percentage. The number of accounts tends to increase in spurts, though, and new users don't always realize that they're supposed to be voting on stories. So some weekends, we'll see the number of users increase by 20 or 30, and yet the number of active voters remains the same. This is something that would get smoothed out by volume, I think. 20/600 is significant. 20/10,000 isn't, really.
    • Granularity/revision: People need to be able to vote "extra yes!" or "extra-no!" sometimes, and also I'd like there to be a revision system like raph describes in his comment. If enough people vote "revise" then it goes to the "drawing board", where people can comment in terms of what should be fixed, and the author can revise, and later resubmit. There should also be a rejects file, cause someone's always gonna be curious. :-)
    • Time-decay/expiration: There needs to be a way to auto-expire old stories. Right now I do it manually when things languish for too long, but it should be an automated system, to decay points by the day (again, this should be configurable-- news stories might decay 1 point per 12 hrs, while D.I.D features might not decay at all).
    • "Quality ratings"? I'd like to do some kind of quality rating, along with the voting. This could be used to help people filter their own home page, or to provide section-summary info.
    All in all, there's still a lot to do. This is all a big experiment, and it's going pretty well so far. Amusingly enough, the very day this story goes up, not one but two stories got shot down in moderation (a record!). So to answer the original question, yes, things do get voted down sometimes.

    I really appreciate all your feedback. Please tell me what to do. :-)

    ____
    Not the real rusty

    Re: My thoughts (none / 0) (#23)
    by joeyo on Mon Apr 03, 2000 at 11:00:18 AM EST

    Some quick thoughts... class in 30 seconds.... :) * extra yes / extra no is a great idea but i see lots of potential for abuse. * as for expired stories how about having the poster set how long the story can sit in the queue before it is expunged, that is if the story is breaking news or otherwise time related he can set a shorter expire time but this also lets it be posted with less votes, and if he has an editorial or whatever he can set a longer 'languish' time but as a result the story needs more votes to get posted... ack times up... I really think that moderation will start to really gell when there are more people using K5...

    --
    "Give me enough variables to work with, and I can probably do away with the notion of human free will." -- demi
    [ Parent ]

    Re: My thoughts (none / 0) (#31)
    by Matthew Guenther on Mon Apr 03, 2000 at 11:49:32 AM EST

    Granularity/revision: People need to be able to vote "extra yes!" or "extra-no!" sometimes, and also I'd like there to be a revision system like raph describes in his comment. If enough people vote "revise" then it goes to the "drawing board", where people can comment in terms of what should be fixed, and the author can revise, and later resubmit. There should also be a rejects file, cause someone's always gonna be curious. :-)

    I think this should be implemented like a special attack in Street Fighter or other similar games. You can "Power Up" and vote say +2 or -2, but only every certain number of votes, or after a certain time interval.

    "Kuro5hin Shinryuuken!"

    :)

    MBG



    [ Parent ]
    Re: My thoughts (none / 0) (#44)
    by Inoshiro on Mon Apr 03, 2000 at 07:51:12 PM EST

    Breaking news or timely stories just don't get posted fast enough.

    It would be nice to have an indicator on the main page which would notify the user of un-voted upon stories in the submission queue. Kinda like how a certain site makes sure people remember to meta..stuff.

    I really appreciate all your feedback. Please tell me what to do. :-)
    Send an blank cheque to my address! Mwuahahaha :-)



    --
    [ イノシロ ]
    [ Parent ]
    Death Meter (none / 0) (#48)
    by nascent on Mon Apr 03, 2000 at 08:47:52 PM EST

    Also, perhaps if a message gets to a certain level (-10, a percentage of the readership, etc), then it just dies on the spot. =)

    I just worry that this could be abused, but I guess any of this stuff could be.
    nascent
    http://www.intap.net/~j/
    [ Parent ]

    Several comments.... (5.00 / 1) (#24)
    by mattdm on Mon Apr 03, 2000 at 11:05:53 AM EST

    First, I think that the mere existence of a moderation system increases the quality of stories. People are less like to post total crap, because there's little point. Second, there _really_ needs to be a way to post comments with the submission queue and NOT have them carry over to the main article. In fact, this should be the default. This would vastly help with the revision process Rusty plans. And finally -- moderation on comments doesn't seem to be working -- whenever I click on a story, everything is 'none'. Oh, sure, I can make rated comments show up first, but then, anything rated low shows up before any good unrated comments -- not exactly what I want....

    Re: Several comments.... (2.00 / 1) (#25)
    by rusty on Mon Apr 03, 2000 at 11:09:18 AM EST

    About comment ratings-- There hasn't been very much rating going on. I'm pretty sure it works though, technically speaking. Try setting your order to "Unrated, the Highest". This will put unrated comments first, then sort high to low.

    ____
    Not the real rusty
    [ Parent ]
    Re: Several comments.... (none / 0) (#32)
    by fluffy grue on Mon Apr 03, 2000 at 12:41:48 PM EST

    My favorite is newest first, ignore ratings. Then I can actually follow conversational threads in a sane manner, regardless of whether some fool's done random ratings.
    --
    "Is not a quine" is not a quine.
    I have a master's degree in science!

    [ Hug Your Trikuare ]
    [ Parent ]

    Re: Several comments.... (none / 0) (#47)
    by Inoshiro on Mon Apr 03, 2000 at 08:05:59 PM EST

    LOL.. Random ratings ;-)

    I also use newest first/ignore ratings. But when I see ratings, I appreicate them. I don't always remember to rate comments, and most comments don't need ratings.



    --
    [ イノシロ ]
    [ Parent ]
    rating comments (none / 0) (#40)
    by mattdm on Mon Apr 03, 2000 at 03:41:11 PM EST

    Yeah, I'm not too happy with that. The net result is:
    
      -- mediocre comment
      -- silly comment
      -- mediocre comment
      -- mediocre comment
      -- bad comment
      -- good comment 
      -- mediocre comment
      -- mediocre comment
      5: great comment
      1: bad comment
    
    Which isn't any better than no moderation at all. I think maybe unrated
    comments should show up in the middle somewhere.


    [ Parent ]
    Re: rating comments (none / 0) (#42)
    by rusty on Mon Apr 03, 2000 at 06:44:41 PM EST

    I will also be adding a "batch rate" button, so that as you read comments, you can set the rating box to whatever you think it should be, then at the end, hit "Rate!" and rate all of them at once. And remember, only (a minimum of) one person needs to rate a comment -- if everyone else agrees, then the work has been done. Hopefully when I get multi-comment ratings in, a few more people will rate more often, which would make the whole thing more useful.

    ____
    Not the real rusty
    [ Parent ]
    Re: Several comments.... (1.00 / 1) (#26)
    by fluffy grue on Mon Apr 03, 2000 at 11:11:01 AM EST

    Yeah, I just have all comment rating turned off. The only time I ever see any ratings happen it's usually someone setting everything to 1.00 for some reason.
    --
    "Is not a quine" is not a quine.
    I have a master's degree in science!

    [ Hug Your Trikuare ]
    [ Parent ]

    Re: Several comments.... (none / 0) (#27)
    by rusty on Mon Apr 03, 2000 at 11:18:03 AM EST

    Re: Several comments.... (1.00) (#26)

    I couldn't help myself. :-)

    ____
    Not the real rusty
    [ Parent ]

    Re: Several comments.... (none / 0) (#46)
    by Inoshiro on Mon Apr 03, 2000 at 08:04:44 PM EST

    Damnit, I can't -1 your comment ;-P

    --
    [ イノシロ ]
    [ Parent ]
    Re: kuro5hin.org's moderation process ineffective? (2.00 / 1) (#33)
    by henrik on Mon Apr 03, 2000 at 12:59:27 PM EST

    I think a lot of k5's features aren't really needed for a site of this size. Comment rating works, but is largely unnessicary because of the high S/N ratio here. Same thing with comment moderation. The scoop engine should scale well, and i think we'll see more use of moderation as the site grows.

    k5 of today is small enough for a communistic, everybody trusts everybody system to work - and we don't really need much safeguards against bad seeds right now.

    The good thing about story moderation is that Rusty can go away on skii holidays and stuff and the site will still be updated. Also, there isn't any mysterious demigodlike person sitting there deciding what I like. All the stories that i find interesting may not get posted, but i still get to check 'em out.

    ps - i think the "YOU choose the stories!" link looks stupid *grin* :)

    -henrik

    Akademiska Intresseklubben antecknar!

    Re: kuro5hin.org's moderation process ineffective? (none / 0) (#34)
    by rusty on Mon Apr 03, 2000 at 01:16:57 PM EST

    ps - i think the "YOU choose the stories!" link looks stupid *grin* :)

    Yeah, me too, but I had to draw newcomers' attention to it somehow. Got any better suggestions?

    ____
    Not the real rusty
    [ Parent ]

    Re: kuro5hin.org's moderation process ineffective? (5.00 / 1) (#35)
    by henrik on Mon Apr 03, 2000 at 01:23:01 PM EST

    Hmm.. how 'bout this:
    keep track of registered users, if they havent voted on any stories for a couple of days, insert a gentle reminder ("It's been a while since you voted on any stories, please go here to vote and here to see why this is important") at the top of the main page.

    -henrik

    Akademiska Intresseklubben antecknar!
    [ Parent ]

    Re: kuro5hin.org's moderation process ineffective? (none / 0) (#37)
    by ramses0 on Mon Apr 03, 2000 at 03:22:03 PM EST

    great idea
    [ rate all comments , for great justice | sell.com ]
    [ Parent ]
    Re: kuro5hin.org's moderation process ineffective? (none / 0) (#45)
    by Inoshiro on Mon Apr 03, 2000 at 08:04:02 PM EST

    "user selected stories"



    :-)

    --
    [ イノシロ ]
    [ Parent ]
    YAFR (Yet Another Feature Request) (5.00 / 1) (#36)
    by eann on Mon Apr 03, 2000 at 02:01:12 PM EST

    The "Moderate Submissions" link on the home page could indicate to logged-in users how many items are waiting for their vote in the queue. If it's already a slow news day, don't disappoint us more by making us go look to confirm it.

    Our scientific power has outrun our spiritual power. We have guided missiles and misguided men. —MLK

    $email =~ s/0/o/; # The K5 cabal is out to get you.


    kuro5hin.org's moderation process ineffective? | 50 comments (50 topical, 0 editorial, 0 hidden)
    Display: Sort:

    kuro5hin.org

    [XML]
    All trademarks and copyrights on this page are owned by their respective companies. The Rest 2000 - Present Kuro5hin.org Inc.
    See our legalese page for copyright policies. Please also read our Privacy Policy.
    Kuro5hin.org is powered by Free Software, including Apache, Perl, and Linux, The Scoop Engine that runs this site is freely available, under the terms of the GPL.
    Need some help? Email help@kuro5hin.org.
    My heart's the long stairs.

    Powered by Scoop create account | help/FAQ | mission | links | search | IRC | YOU choose the stories!