Kuro5hin.org: technology and culture, from the trenches
create account | help/FAQ | contact | links | search | IRC | site news
[ Everything | Diaries | Technology | Science | Culture | Politics | Media | News | Internet | Op-Ed | Fiction | Meta | MLP ]
We need your support: buy an ad | premium membership

Federating K5

By kostya in Meta
Sat Jan 27, 2001 at 03:21:07 AM EST
Tags: Kuro5hin.org (all tags)

"K5 is dying", "K5 sucks", "K5 is going to suck", etc. No, I'm not posting one of those articles--or at least I hope I'm not.

I'd like to suggest a possible "migration plan" for K5, as its numbers continue to grow. When the article "The Death of Kuro5hin" was posted, I responded with an idea: Break it down?. Well, I've been think about it, and I still think it might have some merit. A brief summary:

K5 isn't dying, but simply experiencing the normal effects of having so many people participating. Slashdot has shown how bad the signal to noise can get when there is such a large number of people participating in the discussion. The signal to noise problem is directly related to the size of the user base; to blame it on anything else is to miss the real cause. Slashdot used to have great discussion. Those of us there in the beginning will remember (if you were there before they started handing out userids, you know what I mean). So if the problem has to do with size, why not meet that problem head on. Instead of one K5, have a federation of K5s--each a smaller K5 community that feeds into the large meta-community. This allows us to keep the user base small while maintaining some of the advantages that come with such large communities

I see several ways of doing this, and they are not mutual exclusive:
  • K5 by country and region. That gives us K5-UK, K5-FR, K5-RU, K5-US-NE, K5-US-WEST, etc.
  • K5 by interest. Technology, OS, whatever. Although, I'm not sure this would work, since K5 is a community that spans multiple interests by default.
  • K5 by group--i.e. you just split them up and make them into groups, not depending on anything. This would give you K5-Alpha, K5-Beta, and God help us if we run out of greek letters. We could use military terms then :-) K5-Bravo, K5-Zeta--neat :-)
  • [insert some nifty way to classify or group]

However it works, all the mini-K5s would be a part of the federated K5. You could even call it a "K5 Network". This means that the stories might have a new threshold limit: one that when they are voted high enough or generate enough quality discussion they get sent to the master distribution queue. Then either it gets posted or we put it in another queue and the sub-K5 decides whether or not it wants to post it. You could even have a section just for meta-K5 stories, allowing you a smaller, more close-knit K5 with a section that keeps you in touch with the other mini-K5s out there.

Basically, we are talking about K5 chapters of a sort. I think regional would work well, but something that is regional, but not limited expressly to region, would be even better. It would also help those who are around the world get at a faster/closer site. It would provide a certain level of redundancy, making K5 as a "thing" more resiliant and tougher to take down. I think it also might help the diversity.

There are problems with this idea. To name a few:

  • Cliques/Balkanization/snobbery: K5 chapters might grow apart very quickly and suddenly turn towards isolationism and such. Perhaps, the national-centric stuff might get worse?
  • Administration: this idea would probably require an administrator for each chapter.
  • Hardware, connection, etc: probably need separate hardware and a collocation site for each chapter. This makes keeping the whole thing solvent tough. You could make each chapter responsible for its own costs and hardware, but that would be a headache in itself.
  • User accounts: tracking, logins, comments, abuse--you name it.

Even though there are problems, I think the idea has some merit. Maybe not a whole lot, but I am sure that some bright people here might be able to take this idea and rework it into something good.

What do you think? A United Federation of K5s? It could be the next generation of weblogs? Any other ideas on how to do it, or some new features it might allow us to do?


Voxel dot net
o Managed Hosting
o VoxCAST Content Delivery
o Raw Infrastructure


I think the idea of federating K5:
o sucks! 52%
o rocks! 4%
o has some merit! 44%
o who cares, K5 sucks no matter how you look at it 0%

Votes: 75
Results | Other Polls

Related Links
o Slashdot
o Kuro5hin
o "The Death of Kuro5hin"
o Break it down?
o Also by kostya

Display: Sort:
Federating K5 | 28 comments (21 topical, 7 editorial, 0 hidden)
This is an interesting idea (+1) (3.60 / 5) (#2)
by Mantrid on Fri Jan 26, 2001 at 01:55:46 PM EST

This sounds like an interesting idea. Perhaps the K5-lets could simply be assigned new users on a random basis? I think regional would be a very bad way to do it; part of K5's appeal is the variety of people you get to meet from different places.

I guess the K5's would have a local mod page and a global mod page sort of thing? Would there be a global mod page for each section? Lots of details but might be neat if things get really big.

Yeah, and I don't like "by interest" eit (none / 0) (#22)
by error 404 on Tue Jan 30, 2001 at 10:56:23 AM EST

I learn most from stories that are a little outside my current interest.

And what's the point of World Wide if you group by region?

Picking K5 republics randomly is probably good, as is the idea of picking by similar voting patterns.

But if we go by voting patterns, let's automate the pattern selection and see what emergent phenomena happen. What I mean is this: let's not decide "OK, people who like stories of this type, over here in K5akistan, people who like celebrity stuff go to K5ifornia". Let's map the data abstractly and see where the clusters are without attempting to determine the meaning. Better yet, let's automate finding the clusters and let it happen without specific human interference.

There is a potential problem in that a Federal District is likely to appear: a K5DC, where those who revel in meta congregate. That republic would be likely to have exessive influence, so membership would have to be opened.
Electrical banana is bound to be the very next phase
- Donovan

[ Parent ]

A bit of an over reaction? (3.83 / 6) (#5)
by GreenCrackBaby on Fri Jan 26, 2001 at 02:01:11 PM EST

K5 isn't dying, but simply experiencing the normal effects of having so many people participating.

Huh? A typical story submission gets at most 400 votes. A typical story gets at most 100 comments. I'd hardly consider this "so many people."

I think K5 can safely evolve without radical changes.

Try Slashdot (none / 0) (#28)
by RandomPeon on Tue Feb 20, 2001 at 03:31:38 AM EST

Having read Kuro5hin for exactly one week, I can say the "massiveness" problem hasn't even begun hear compared to Slashdot.

Posting on Slashdot is like trying to talk at a massive gathering, like a rally or packed discussion. You have to shout to be heard, you have to be brief, and you have to take the floor quickly. My mental picture of Slashdot comes from the National Assembly scenes in Danton - hundreds of people trying to have a discussion all at once amid utter chaos. (Don't recommend the movie, it's in French and boring, but it captures a badly fuctioning legistlature well.) No one wants to read a Slashdot comment that's more than two paragraphs long unless it's truly brilliant - there are just too many comments and not enough time.

Kuro5hin still feels much, much more like a small gathering. My mental picture is more like a party where everybody knows someone else you can carry on a conversation, even though you might have to raise your voice once in a while. You still post a comment with more than one paragraph and expect that people will at least skim it. I think the sky isn't falling yet, although it might.

[ Parent ]
Putting it to practice (3.66 / 6) (#6)
by Flavio on Fri Jan 26, 2001 at 02:05:03 PM EST

If we *had* to do this, I'd suggest breaking down by interest.

An older article (I believe it was never even posted) suggested opening K5's non-sensitive database data to the public. This would allow us to plot statistics, basically.

As we don't have any AI capable of reading user comments and deducing personalities, I have a much simpler yet effective suggestion. Write a script to sort K5 users into groups by using their K5 story voting results.

This way we'd have K5 republics based on personalities. It'd be really fun, I believe. New users would join a global K5 for a test period (2 months or 100 story votes, for example) during which their personalities would be analyzed.

No one would be actually able to submit stories directly to the global K5. The global K5's queue would be composed of story submissions randomly from each republic and would compare voting results to each republic's data.

And to top it off, we wouldn't be torn apart because we'd still be part of global.

What do you think?


Could we also use MBTI types? (none / 0) (#19)
by psicE on Sun Jan 28, 2001 at 03:52:39 PM EST

That's a pretty good indicator of personality, at least if you only use temperaments. :) I'm pretty sure that it has a 75% accuracy rate, and that's for all four factors. Just the temperament (which only depends on 2 factors, depending on which temperament) is probably more accurate than that. Anyway, if it doesn't work, you could always try something else (that's true no matter what factors you use).

[ Parent ]
MBTI types (4.00 / 2) (#20)
by Flavio on Sun Jan 28, 2001 at 07:21:47 PM EST

Sure, but I don't know how well that'd work.

Profiling by story moderation is probably foolproof. I have no idea what the correlation of story moderation to MBTI would be.

MBTI would also require a personality test. I'm not sure people would be willing to take one and it could sound kind of intrusive.

But anyway, overall it's a very good idea.

Perhaps we could use MBTI+story moderation and make a K5 matchmaking service =)

I'm practically sure K5 federation won't be done, but it's fun to think about it.

[ Parent ]
Grouping by MBTI class considered harmfull (none / 0) (#21)
by error 404 on Tue Jan 30, 2001 at 10:30:39 AM EST

You really want to be in a group comprised of people that similar to yourself? I don't. I'm not that fun to be around.
Electrical banana is bound to be the very next phase
- Donovan

[ Parent ]
Remember the topic (none / 0) (#23)
by psicE on Sun Feb 04, 2001 at 06:42:28 PM EST

These comments were in reference to the K5 Federation idea, and I was suggesting a better way to put similar people into groups than by story moderation. We're starting from the premise that we WANT similar people in the same group(s). Whether or not that's a good idea is another story.

[ Parent ]
Similar interests, sure (none / 0) (#24)
by error 404 on Mon Feb 05, 2001 at 11:54:57 AM EST

but similar personality types?

I think there might be some worthwhile grouping information in the voting records. People who like the same kinds of stories as I do, well, like the same kinds of stories as I do. Good basis for discussion.

People with the same personality type as I do don't neccessarily share interests. But we are likely to share posting patterns: either post a lot, or very little. Lots of emotional stuff, or very cold analysis. Grouping by personality type would probably lead to some very high traffic republics and some very low ones, some flame-ridden ones and some cold ones. None with the kind of balance I'd like to see. Even humor would suffer - segregate by sense of humor and some groups have no humor at all while others have too few straight lines to play off of.
Electrical banana is bound to be the very next phase
- Donovan

[ Parent ]

i agree, but story moderation is no better (none / 0) (#25)
by psicE on Mon Feb 05, 2001 at 03:55:31 PM EST

I definitely agree with your points about the MBTI's unhelpfulness in this case, but I can't really see how story moderation works easier. There might be a way to implement it so that it's workable (although we're probably not going to have the federated K5s), but a lot of times, if there's a story that I'd otherwise vote Don't Care on, I'll randomly vote it up or down depending on what mood I'm in. Sometimes I just vote everything up w/o reading it; sometimes I vote stuff to section and front page based on length.

However, I could understand grouping people by their favorite categories and/or hotlisted stories, if and when we implement customized front pages. That is actually an indicator of what kind of stories the person likes, instead of just what mood they're in when they visit K5.

[ Parent ]
Not a big fan of going by moderation myself (none / 0) (#26)
by error 404 on Mon Feb 05, 2001 at 04:14:57 PM EST

In my case, mostly because I get the most value from stories that are a little outside my current interests.

In your case, I guess you would be assigned a republic sort of randomly.

My thought is that someone would put together some kind of set-theory N-dimensional graph, and that most people would end up in clusters in that graph. The clusters would then become republics.

But if we do a federation, I'd rather just pick people at random and see how things develope after importing and exporting stories and comments and giving people the ability to emigrate. Still, I really don't see enough people here to require that.
Electrical banana is bound to be the very next phase
- Donovan

[ Parent ]

fairly easy (4.37 / 8) (#7)
by Arkady on Fri Jan 26, 2001 at 02:08:02 PM EST

This would be pretty easy to set up, and (though I'm not certain it's necessary yet) would obviate the need to deal with database replication across multiple servers as K5 grows further. The technical issues will, I think, drive the need for multiple sites more quickly than the social ones, as I don't think K5 is degenerating nearly as much as some of you seem to.

I think it'd be best to let people _choose_ how they wish to associate. Any group of registered users (of a predermined size, say 1000 or more) who could put up a technically viable machine, connection and admin could declare themselves a K5 site. Their accounts would then be homed at that site, though usable to read and comment (but not vote) at any other K5 site. Trusted user status on one site could be restricted to that site, and used to manage user accounts and administrative details as well.

Add a new option for "Post to Global" to each site's submission queue which would (if it won the vote) scp the story's relevant data to the global server, where a cron job would pick it up and insert it into the db after validating its source.

That would allow the interest groups your talking about to form on any basis they want and let folks change groups to associate with whatever set they want. If the nationalists want a (say) Serbian K5 to be distinct from a Kosovar K5, that's their deal. I'd personally want to join a group with a diverse set of views and interests, but I wouldn't mind losing some of the liberation capitalists. ;-)

Anarchists have long preferred the voluntary federation model for social associations, and this suggestion gives us an opportunity to see how well they could be implemented on a global scale.

I'm up for it.


Turning and turning in the widening gyre
The falcon cannot hear the falconer;
Things fall apart; the centre cannot hold;
Mere Anarchy is loosed upon the world.

Great ideas (3.00 / 1) (#8)
by kostya on Fri Jan 26, 2001 at 02:12:47 PM EST

You expressed several that I had in my head, but forgot to mention or couldn't articulate well. The "global" userids that allow you to participate, but only post locally were something I thought would help the signal to noise ratio. If we keep the commenting limited, we cut down on the "me toos".

Veritas otium parit. --Terence
[ Parent ]
account migration? (3.80 / 5) (#11)
by HiRes on Fri Jan 26, 2001 at 03:05:27 PM EST

A subtle problem with your plan that wasn't mentioned is the matter of migrating accounts. Who decides which accounts go where when K5 undergoes a split? Would it just be random?

Obviously it would depend on the criteria used to make the divisions, but it's something that would have to be considered.
wait! before you rate, read.

Are you nuts?! ;) (3.00 / 6) (#12)
by Seumas on Fri Jan 26, 2001 at 03:22:59 PM EST

Yeah, right -- one day you're forming a federation and the next thing you know, you're being assimilated by the Borg!
I just read K5 for the articles.
Interesting (4.25 / 4) (#13)
by xiox on Fri Jan 26, 2001 at 03:46:24 PM EST

It's an interesting idea, but changing k5 in that way might break what is best about it - bringing regional and topical ideas together. If you want to lower the Signal-to-Noise, then how about getting more sections - regional sections, narrow topic sections (technology, software, education, US-politics, European-politics, global-politics.. etc). Why not keep the whole thing as a single site? That way you will be able to closely interlink everything.

I'd quite like to see something like Mono - a UK internet bulletin board. It has lots of interesting files where the discussion is very good and worthwhile.

Half-Empty idea (5.00 / 2) (#14)
by BonzoESC on Fri Jan 26, 2001 at 05:48:26 PM EST

Nebby had the same idea with Half-Empty and Glasscode, in that each site would host a specific area of discussion, and specialize in that, but would also link to the main site to handle sharing of entire categories across sites, allowing you to choose which site and flavor of .5e you like, but you could see anything on the .5e network though it. (nebby, correct me if I'm wrong about your vision.)


Normally, my sig is an image.

My idea... (5.00 / 6) (#16)
by skim123 on Fri Jan 26, 2001 at 07:28:34 PM EST

I've been kicking around this idea in my head for a while. I think it has its benefits over a regional approach because the Internet kind of removes all geographic boundaries and such. Also, since a community site is only as good as its members, here's my idea:

Every person can create a list of k5 members they count as trusted friends, or people whose comments/posts/stories the find interesting. They can then, optionally, view a story with a filter turned on saying, "Show me posts only by my group of friends," or "Show me posts only by those on my friends list and the friends on my friends list," or "Show me everyone's comments." You could also filter the submission queue via this method.

So, why is this a Good Thing? There are certain k5 posters whose comments I enjoy reading, and I'd like to be able to get to those quickly. There are posters who I've not enjoyed reading, and I'd like to be able to easily skip over that. Granted, I do not have the time to pick through newer people and decide whose new posts I should like, but if someone on my friends list enjoys user x's commentary, chances are I will too. So I can view his posts by setting it so that I see posts by my buddies and by my buddies' buddies.

This would have the effect of partioning k5 into chunks of users who found one another interesting. I guess you could make some prefabricated groups that people could sign up to if they wanted to, special interest groups, like, "People from location x," or "People whose userid is less than 1,000," or "People who have active diaries," or whatever. There could be an optional poll/survey on the user page where people could indicate interests and then sign up to groups of people with similar interests and all.

I know this all may sound a triffle hokey, but it would be a wonderful way to get to know those posting. If, rather than seeing 100 posts, you say a handful of posts from a group of people you hand picked or who shared something in common with you, then you'd get to know those folks better, be more incentivized to participate in a discussion, etc.

Just my two cents...

Money is in some respects like fire; it is a very excellent servant but a terrible master.
PT Barnum

What about newbies? (none / 0) (#27)
by RandomPeon on Tue Feb 20, 2001 at 03:17:35 AM EST

Just like the original story, this sounds like a good solution to the seemingly intractable problem of size growth.

But what about recent immigrants from Slashdot and beyond, such as Yours Truly? I don't know any of you Kuro5hin people - I'm sure you're all nice people, but who are my "buddies"?

You're setting up the same problem that happens when people move - you have to have some friends in order to make friends. I don't recognize any of the posters, except Signal 11, who I do not wish to recognize, and at the moment I am "friendless".

An immediate solution would be to assing newbies a random set of buddies. But they could get people who never post, people who are trolls, and so forth.

[ Parent ]
A less drastic solution (none / 0) (#18)
by vasi on Sat Jan 27, 2001 at 06:11:39 PM EST

I agree that there's a problem with the expansion of K5's user base, but I'd like a deeper analysis of what this problem is, and why exaclty it's happening.

The situation we have is a K5 with more and more users joining. More users post more comments, more stories, and on a wider range of topics. These new contributions aren't uniformly better or worse than the old; there are just more of them. K5 has had two options to deal with this:

  1. Give all the new contributions the same standing as contributions have always had. This means rating stories and comments in the same way K5 has always done, with the consequence that a lot more comments and stories will get through to most readers. Such an option is good in that we get a broader range of viewpoints, but bad in that there's an information overload and more "unwanted" content.
  2. Keep K5 the same size. This means being a more discriminating (not in the prejudicial sense, in a good-vs.-bad sense) when judging comments and stories. By doing this, we're keeping the quality as high as we'd like, but not letting through some of the worthwhile contributions there are out there.

The reaction, in reality, has been some combination of these two, yielding some combination of the benefits and problems of both. We're left with too many comments and stories for any one of us to handle, and a range of topics too wide for any of us to be able to comment on appropriately. So that's the what and why of the situation now.

The proposed solution to break K5 into pieces "solves" the problem by focusing only on the second option. This is a great solution for getting rid of the information overload, but just like the second option it would narrow K5's focus and leave a lot of the good contributions unavailable to those of us who'd want to see them. I think that we still need to balance both the first and second options in any solution for the problems on K5.

We do need to split up K5 in some way, probably along the lines of topics and sections. However, we also have to make sure that there's a cross-over between splits, so they're not isolating. I think the simplest way to do it is to have individual thresholds on submissions. I'd like to be able to decide that I only want to see MLP stories rated at > 4.5/5 on my section page, and > 4.8/5 on my front page, to make sure that I only see the greatest ones.

This solves several problems. First, each reader only gets (and respond to) stories (s)he's at least somewhat interested in. This cuts down on ignorant posts, means that I see less of what I consider "noise", and means that I can concentrate on what I like. Second, there's no longer any need to vote a story down because "I vote all MLP down". If you don't want it, you never see it, and you don't have to enforce your preferences on anyone else. Third, and critically, it does all this without changing K5's organization too much, and without totally eliminating anything I don't list as my sole preference.

I can't pretend this is perfect, of course. How can I polish this idea a bit? (Or why will it cause the world to end? ;-)


Federating K5 | 28 comments (21 topical, 7 editorial, 0 hidden)
Display: Sort:


All trademarks and copyrights on this page are owned by their respective companies. The Rest 2000 - Present Kuro5hin.org Inc.
See our legalese page for copyright policies. Please also read our Privacy Policy.
Kuro5hin.org is powered by Free Software, including Apache, Perl, and Linux, The Scoop Engine that runs this site is freely available, under the terms of the GPL.
Need some help? Email help@kuro5hin.org.
My heart's the long stairs.

Powered by Scoop create account | help/FAQ | mission | links | search | IRC | YOU choose the stories!