Kuro5hin.org: technology and culture, from the trenches
create account | help/FAQ | contact | links | search | IRC | site news
[ Everything | Diaries | Technology | Science | Culture | Politics | Media | News | Internet | Op-Ed | Fiction | Meta | MLP ]
We need your support: buy an ad | premium membership

Yet another file-sharing project?

By Eloquence in News
Sat Jul 08, 2000 at 03:48:31 PM EST
Tags: Freedom (all tags)

Recently I have discovered konspire. It's an interesting Java-based Gnutella-like file sharing project. Contrary to Gnutella, it makes a difference between clients and server. Every user can decide to run only either one or both. The reasoning behind this is that "for instance, a large memory and a fast network connection is needed to run an index server, though any machine with a hard drive and a modem will work as a client". Each server carries a list of all files on the network (while Gnutella doesn't have such a global index), which is supposed to make searches very fast. They also provide a table comparing Gnutella, Napster and konspire. (They are incorrect about "File transfers resumable" though.)

konspire is relatively easy to install if you have the JRE (under Win32, for example, you just do "java -cp konspire.jar konspire.client.Client"). It has a neat Swing GUI and is, of course, open-source and free. Unfortunately, the network is relatively empty right now. Perhaps we can change that with a "kuro5hin-effect"? ;-) Also, it seems to be necessary to share the files in a "files" subdirectory, which makes it difficult to share existing collections.

For the discussion, I think several questions are interesting:

  • Which network architecture is the best for a file sharing project:
    • central like Napster
    • distributed servers sharing one index like konspire
    • distributed servers sharing different indexes and returing the results to the searching IP individually, like OpenNap
    • totally distributed network like Gnutella, without any form of a centralized index
  • Which additional features are required to make these projects more attractive?
  • How do you create a Napster- and Gnutella-like hype around a project?
    • I hate to say it, but it seems like the early existence of a Win32 client is very important. Both Gnutella and Napster had this, and that resulted in the early adoption of the protocols by open-source alternatives. konspire, FreeNet and JungleMonkey don't have this, and there are hardly any alternatives to the official clients.
    • Which role do the media play? Before a certain news site reported about Gnutella, the user base was fairly small. Is it a good idea to make such projects known early on? In the Gnutella-case, it resulted in the end of the project because of AOL's involvement. (I don't think this danger exists for the konspire project, although Cornell might decide to pull the "HardenedCriminal software" [that's what the konspire guys call themselves] site.)

In any case, I believe that konspire is a neat addition to the pool of already existing file-sharing & publishing apps (see this site for a list). The really interesting question is whether governments and corporations will try to outlaw and combat this technology and what impact it will have on laws against copyright-infringements, pornography and other controversial information.


Voxel dot net
o Managed Hosting
o VoxCAST Content Delivery
o Raw Infrastructure


Related Links
o Kuro5hin
o konspire
o "for instance, a large memory and a fast network connection is needed to run an index server, though any machine with a hard drive and a modem will work as a client"
o table
o JungleMonk ey
o this site
o Also by Eloquence

Display: Sort:
Yet another file-sharing project? | 7 comments (7 topical, editorial, 0 hidden)
naming conventions... (3.00 / 2) (#1)
by warpeightbot on Sat Jul 08, 2000 at 03:52:07 PM EST

"Konspire," eh? I love it. If there's one thing the hacker community (dmr and spaf and linus and esr, not script kiddies) has had for thirty years and more, it is the ability to laugh at itself by what it names things. less(1), GNU and all its puns (bison for yacc, to name an egregious one), Spam, KDE, konqueror, and now konspire. As if we were a bunch of trenchcoat-wearing shady dudes out of the cold war, making dead-drops of virtual CDs full of data stolen from the Evil Empire.... I just LOVE the image :)

Brave new world, that has such people in it!

(4.00 / 1) (#2)
by karjala on Sat Jul 08, 2000 at 06:39:29 PM EST

Which network architecture is the best for a file sharing project:

I think that among the two middle models, the distributed servers sharing different indexes (OpenNap) is the most appropriate one. It's the least bandwidth-consuming model and gives accurate results, like Napster, whereas the results given by Konspire might not be up-to-date.

I did the following calculation to find out which one consumes less bandwidth: The number of MP3 files owned by a user is on average a lot greater than the number of searches he'll do in a single session. Therefore if the data that has to be exchanged between the servers is the file list of every user logging on the network, it would respectively consume a lot more bandwidth than if the data that had to be exchanged between servers were the users' queries for files. Hence the OpenNap is a lot more efficient.

Which additional features are required to make these projects more attractive?

The feature which I would like to see most is the ability to broadcast streaming media through the network - if the servers would play the role of anonymizers, that would enable us to watch all the TV channel that exist.

Things I'd like to see... (none / 0) (#3)
by Anonymous Hero on Sat Jul 08, 2000 at 10:17:48 PM EST

I'd like to see a totally distributed file-system, but with the ability to generate complete file lists (or rather, damn-large file lists.... more than is on a single server) to make searching faster.

And having the distinction between running a client and server is nice, but both should be included in the software, and both should run as default. You need to maintain a *lot* of servers, or a distributed file-system can be closed down.

Of course, it should be freeware/OSS, or it'll get sued out of existence.

I'd also like to see version control by author/publisher. If you publish something you can't ever take it back, This is sometimes a problem, if you have updated information - or you wanted to change your contact address on information. Or you just changed your mind about something. This wouldn't stop people from keeping copies of old information, but it should make such information post as retracted by the author/original publisher. You would *have* to maintain anonymity of the publisher, or they'd be tracked down and forced to relinquish control of files that people didn't want published.

-- Ender, Duke_of_URL (damn tired on the weekends)

Filesharing through jabber ? (3.00 / 1) (#4)
by Anonymous Hero on Sun Jul 09, 2000 at 10:34:12 AM EST

What I'd like to see is filesharing through a jabber transport (www.jabber.org)
That way not only could you manage all your communication through the same client. You could manage all your file transfers through it too. And it offers the same distributed advantage.

Decentralized storage is great but... (4.00 / 1) (#5)
by tzanger on Mon Jul 10, 2000 at 02:55:04 AM EST

... I belive you need some kind of scatter/gather network in order to keep anonymity of actual transfers.

How I had described this on /. is like this:

The server providing the file does not send directly to the client. Instead the file is broken up into 'x' chunks, encrypted and md5'd (or something similar) and sent to 'x' random server/clients which check their chunk for validation, break it up and send it off again. This is done about a half dozen times or so and then the pieces get to the client which puts the pieces back together after validating each one. Missed fragments can be asked for again (with a less scattered effect).

Compromised/poisoned scatter nodes would be eliminated from the pool of available nodes by some kind of combination of heuristic and blacklist. i.e. if x% of the packets from node y are corrupt/don't get to where they should be, eliminate them from the pool of scatter nodes. Since a particular scatter node has no idea if its target is the final destination or not there isn't really any way to track who asked for the file, unless you capture a large percentage of the available scatter nodes and log them, hoping that enough of them are sending to the same client, whom you can then identify as the target node.

Problems with this? Node processor use is higher as each fragment needs to retain some kind of fingerprint of the original to check against, while at the same time be unable to be altered without being noticeable. something along the line of some kind of fingerprint which can be split up and still be valid, but not be alterable. Bandwidth useage amongst the scatter nodes is higher and overall transferred bytes are higher as well.

I don't have the answer to these problems, all I have is the idea. :-) Obviously nodes should be able to opt out of the scatter network if their available bandwidth is too low or they need all they have. I would think that if you're a server you should be willing to scatter for at least a couple people though.

Freenet! (none / 0) (#6)
by Anonymous Hero on Mon Jul 10, 2000 at 10:21:58 AM EST

As soon as Freenet becomes searchable (they´re working on it), it´ll be the ideal file sharing program.

Win32 Client for Freenet et. al. (none / 0) (#7)
by hardburn on Mon Jul 10, 2000 at 10:58:19 AM EST

Why the importance on a Win32 client? Both Freenet and this new thing is written in Java. There are other clients (for Freenet, anyway) that go sometimes go to specific archatectures, yes, but the referance implementation is in Java.

Also, I don't see why we need konspire. It seems to do everything Freenet does, except it adds centrialized searching. Freenet developers are currently hacking out a system for decentrailized search (which is very difficult but it will be great when the pull it off).

while($story = K5::Story->new()) { $story->vote(-1) if($story->section() == $POLITICS); }

Yet another file-sharing project? | 7 comments (7 topical, 0 editorial, 0 hidden)
Display: Sort:


All trademarks and copyrights on this page are owned by their respective companies. The Rest © 2000 - Present Kuro5hin.org Inc.
See our legalese page for copyright policies. Please also read our Privacy Policy.
Kuro5hin.org is powered by Free Software, including Apache, Perl, and Linux, The Scoop Engine that runs this site is freely available, under the terms of the GPL.
Need some help? Email help@kuro5hin.org.
My heart's the long stairs.

Powered by Scoop create account | help/FAQ | mission | links | search | IRC | YOU choose the stories!