Kuro5hin.org: technology and culture, from the trenches
create account | help/FAQ | contact | links | search | IRC | site news
[ Everything | Diaries | Technology | Science | Culture | Politics | Media | News | Internet | Op-Ed | Fiction | Meta | MLP ]
We need your support: buy an ad | premium membership

[P]
Morpheus and the Gnutella explosion

By scanman in Internet
Mon Mar 04, 2002 at 03:38:15 PM EST
Tags: Software (all tags)
Software

With the switching of Morpheus to the Gnutella network, the size of the Gnutella network has grown tenfold, overnight. This has had the effect of making the network almost unusable for dial-up users.


The Gnutella network is designed to be a purely peer-to-peer network. Unfortunately, not all would-be file sharers are created equal - some have fast connections, some do not. The Gnutella network was not designed to handle such huge inequalities gracefully.

When a fast node connects to a slow node, the slow node will not be capable of accepting packets fast enough, and most clients will simply drop it. This has caused an "undernetwork" to form - people without enough bandwidth to share, trying to share with other people without enough bandwidth to share.

This is obviously not an ideal situation. People with less bandwidth should not be expected to bear the full load of a large gnutella node. The obvious solution is, of course, creating an extension to the protocol that allows searches to be "injected" into the network, without the originating node needing to be fully connected to the network.

Without an official channel to discuss and implement protocol changes, the problem is made more difficult to solve. Altering the protocol inevitably fragments the network even farther, because of the several different groups who supply clients, all working seperately. A respectable authority figure needs to step in and make a decisive statement in order to cause a universal change in the protocol.

For example, the use of the new version 0.6 protocol has not become widespread until recently, although its specification has existed for years. BearShare originally adopted the protocol, and created a network of almost exclusively BearShare users.

The mass exodus of Morpheus users has worsened all of these problems in various ways, by increasing the number of dial-up users, and by adding one more client to the already disorganized group.

Sponsors

Voxel dot net
o Managed Hosting
o VoxCAST Content Delivery
o Raw Infrastructure

Login

Poll
Can the gnutella network be fixed?
o It's fine the way it is. 4%
o Yes, but it would be very difficult/unlikely. 12%
o Yes, it will sort itself out. 32%
o No, the protocol is too simplistic. 18%
o No, the lack of coordination will be its downfall. 12%
o Only if Inoshiro takes over. 19%

Votes: 109
Results | Other Polls

Related Links
o Also by scanman


Display: Sort:
Morpheus and the Gnutella explosion | 45 comments (37 topical, 8 editorial, 0 hidden)
Why bother? (4.00 / 9) (#6)
by schrotie on Sun Mar 03, 2002 at 08:40:38 PM EST

Why should the gnutella protocol be further developed? There are too many essential features, that gnutella lags:
  • swarm (downloading 1 file from many users, as kaza/morpheus and other modern p2p protocols supported).
    This is essential for good performance for all users without assigning too much load to anybody.
  • Crypting contents like freenet and mojonation (BTW does anybody know what has become of mojonation?) allow. Only networks that allow encryption imho stand a chance to survive the coorporate assault (I know of one case where some Movie company "shared" shrek to morpheus or kazaa and then identified the users who further spread that copy of shrek then urging the german provider to do something against this - with success).
  • I think mojonation's idea of using mojo/karma/whatever to better spread the load is very good, but it looks like they sold out or something...
Why not use and improve one of the more advanced protocols rather than trying to keep gnutella artificially alive?

Mojo Nation .. (4.33 / 3) (#8)
by Eloquence on Sun Mar 03, 2002 at 08:59:18 PM EST

.. is being developed into MNet.
--
Copyright law is bad: infoAnarchy Pleasure is good: Origins of Violence
spread the word!
[ Parent ]
Swarm (5.00 / 3) (#9)
by LukeyBoy on Sun Mar 03, 2002 at 09:28:20 PM EST

Gnucleus (for Windows only so far) has swarming built-in and it works flawlessly, downloading from sometimes 10 hosts at once with no data corruption. You should check it out.

[ Parent ]
Because. (4.33 / 3) (#11)
by mofospork on Sun Mar 03, 2002 at 09:35:38 PM EST

Most of the other protocols are either closed protocols or reliant on central severs. That's reason enought to continue to develop gnutella. Napster is dead, the kazaa network has been fucked and is being retooled into a delivery mechanism for spyware. Despite its flaws gnutella has been relatively stable, and I suspect that it will eventually take on aspects of the other networks instead of being replaced.

[ Parent ]
freenet (none / 0) (#32)
by speek on Mon Mar 04, 2002 at 01:43:41 PM EST

not closed nor centralized.

--
al queda is kicking themsleves for not knowing about the levees
[ Parent ]

Despite that... (none / 0) (#42)
by Canar on Tue Mar 05, 2002 at 02:03:22 AM EST

Nor useful.

[ Parent ]
giFT could be the answer... (5.00 / 1) (#34)
by jonathan_ingram on Mon Mar 04, 2002 at 03:22:07 PM EST

...or at least the underlying protocol it uses. giFT was originally based from observations of the FastTrack network traffic, but quickly developed into its own system once FastTrack closed themself off from the world. It's running quite nicely on UNIX, and a Windows client is in development.

It's not reliant on central servers like Napster (or, indeed, the new model FastTrack), but it's still more network efficient than Gnutella (which is an exceptionally badly designed protocol). Documentation is lacking, but will be coming as the developers work toward a 1.0 release. Swarming (multi-source downloading) and resuming is already in place, and working well (although multi-source downloading may be disabled in the most recent CVS, as they are removing some kinks in the current design).

All in all, it's shaping up to be quite an exciting project. Get the source from the Sourceforge giFT page, or come and talk to the developers at #gift on IRC openprojects.net
-- Jon
[ Parent ]

That sounds pretty cool. (5.00 / 1) (#46)
by mofospork on Sun Apr 21, 2002 at 04:06:24 AM EST

I'll have to try it out sometime.

[ Parent ]
barf (4.33 / 3) (#14)
by scanman on Sun Mar 03, 2002 at 10:31:00 PM EST

  • Swarm is actually a client issue, rather than a protocol issue. Several clients now implement this feature.
  • Crypto and "mojo" could easily be added to the next verson of the protocol, but first someone needs to design it. The problem is, nobody is really working on this.

"[You are] a narrow-minded moron [and] a complete loser." - David Quartz
"scanman: The moron." - ucblockhead
"I prefer the term 'lifeskills impaired'" - Inoshiro

[ Parent ]

swarm? (none / 0) (#29)
by boomi on Mon Mar 04, 2002 at 12:13:43 PM EST

* Swarm is actually a client issue, rather than a protocol issue.

huh?
If the only information to compare files is the name and the size, you will not get much.
eDonkey2000 uses a CRC for this matter.

[ Parent ]
Nope. (none / 0) (#33)
by scanman on Mon Mar 04, 2002 at 02:35:43 PM EST

Gnutella uses SHA1 keys, at least the newer clients do.

"[You are] a narrow-minded moron [and] a complete loser." - David Quartz
"scanman: The moron." - ucblockhead
"I prefer the term 'lifeskills impaired'" - Inoshiro

[ Parent ]

My Samizdat protocol is in development (5.00 / 1) (#44)
by kiwipeso on Tue Mar 05, 2002 at 02:33:44 PM EST

*   swarm (downloading 1 file from many users, as kaza/morpheus and other modern p2p protocols supported). This is essential for good performance for all users without assigning too much load to anybody.
Phex supports multiple candidates for files, it just switches candidates if a download stops or stalls. Phex is a gnutella based app.
*   Crypting contents like freenet and mojonation (BTW does anybody know what has become of mojonation?) allow. Only networks that allow encryption imho stand a chance to survive the coorporate assault (I know of one case where some Movie company "shared" shrek to morpheus or kazaa and then identified the users who further spread that copy of shrek then urging the german provider to do something against this - with success).

My Samizdat network is strong encrypted, it shares slices of files over different routes to users. This is an old trick I came up with 5 years ago and is a reliable way of delivering encrypted info.
*   I think mojonation's idea of using mojo/karma/whatever to better spread the load is very good, but it looks like they sold out or something... Why not use and improve one of the more advanced protocols rather than trying to keep gnutella artificially alive?

What my system has is hives, cells and nodes. Hives are grid servers, they handle large loads with high speed connections. Hives deal with cells, they are like DNS roots.
Cells are local servers, they take a list of local nodes and what files they have to offer. Cells can be run from 128k connections, no less.
Nodes are the basic user connections, they just send the database of their files to their local cell.

The Samizdat system runs on a democratic vote basis, hostile hives, cells or nodes can be voted off the grid network.
Any node with 128k or more traffic and some space could be offered a turn as a cell at random. Users vote on if a cell is hostile or friendly, if the cell has enough traffic and friendly votes it may be offered a turn as a hive.
Hives can be voted out like cells or nodes, hives handle the cross cell results of searches. users will get results from their cell first, then other cells on the local hive, then from other hives.
I will post more on this to my website today.
Kaos operating system creator.
[ Parent ]
So drop gnutella (4.00 / 5) (#10)
by CaptainSuperBoy on Sun Mar 03, 2002 at 09:30:01 PM EST

the use of the new version 0.6 protocol has not become widespread until recently, although its specification has existed for years.

But Gnutella was only released 2 years ago.. I'm not at all familiar with Gnutella development, but I don't see how it's possible to have a new version of the protocol before the old version is even released.

Aside from that nitpick, isn't this just another argument against gnutella? In my wholly unscientific testing, I've found that I can't download much with gnutella. Anything that does go, goes slow. Why not just move to a better protocol?

--
jimmysquid.com - I take pictures.

The Gnutella network protocol is really awful (3.66 / 3) (#12)
by Ken Pompadour on Sun Mar 03, 2002 at 10:08:32 PM EST

... but Gnutella is the only protocol that has a good file sharing client, called 'Bearshare.'

If Bearshare didn't exist, I'd agree with your sentiment, but this client is really compared to every other client for every other protocol available.

...The target is countrymen, friends and family... they have to die too. - candid trhurler
[ Parent ]
From the status information of the 0.6 handshake (none / 0) (#38)
by murklamannen on Mon Mar 04, 2002 at 03:34:22 PM EST

Status : Finalized
Proposed By : CHRISTOPHER ROHRS (LimeWire, LLC.)
GREG BILDSON (LimeWire, LLC.)
<info@limewire.com>
*Proposed On : August 3, 2001*
Last Updated : September 1, 2001

Notes:

This proposal is finalized. See "Development" folder for more details.

Released less than a year ago. Bad fact-checking.

[ Parent ]
What song (1.33 / 12) (#16)
by medham on Mon Mar 04, 2002 at 03:38:45 AM EST

Is it that goes "trying to walk a straight line/on sour mash and cheap wine?"

And is there somewhere I can download it without paying?

The real 'medham' has userid 6831.

Do a google search (4.50 / 2) (#20)
by Secret Coward on Mon Mar 04, 2002 at 05:21:08 AM EST

It's AC/DC, Have a drink on me. It's from Back in Black. It was a very popular album. I'm sure you can buy it at your local music store.

[ Parent ]
For those of you intrested in numbers (4.25 / 4) (#17)
by kingosric on Mon Mar 04, 2002 at 04:30:45 AM EST

For those of you intrested in real numbers, LimeWire maintain a graph of the number of hosts on the Gnutella network against time. See limewire.com. Note the spike!

Kazaa Lives (3.33 / 3) (#18)
by n8f8 on Mon Mar 04, 2002 at 04:40:46 AM EST

And has also grown greatly. A week ago the average users logged on was about 1/2 million. Now it has spiked to appx 3/4 million average. This is probably because now that Morpheus is no longer a part of the network ,and GnuTella with so many users doesn't suck as mutch, usrs are now running both simultaneously.

Sig: (This will get posted after your comments)
Morpheus == Kazaa (3.00 / 1) (#22)
by kaitian on Mon Mar 04, 2002 at 10:04:22 AM EST

And Morpheus had the greater number of users. Kazaa won't live for very much longer, I've heard stories that it was aquired by some company notorious for planting spyware on people's computers, and that they killed off Morpheus because it wouldn't distribute the spyware.

[ Parent ]
Kazaa coming apart at the seams (5.00 / 1) (#39)
by BelDion on Mon Mar 04, 2002 at 05:48:40 PM EST

I've also noticed this increase of users, but have you noticed that content has decreased?

I'm not really sure why it's happening, but since Morpheus seceded from the FastTrack network, although the user number on Kazaa have jumped up (I've seen it at over 1.2 million), the actual amount of content is going down. At this very moment (quicky check), almost 1M users, 1'600K files shares in 26'000GB.

This alone is one heck of a decrease, but on top of that, it seems that a whole of of people are claiming to be behind firewalls. Most of the time, I hit somewhere between 1/4 and 1/2 of sources claiming to be behind firewalls (and thus useless).

While I hate to make predictions, I have to believe that this isn't a good sign. I'm starting to think that the FT network is doomed to implosion in the next few months.

Oh well; life didn't end when Napster died either.

[ Parent ]
Re: Kazaa coming apart at the seems (none / 0) (#45)
by xtyger on Wed Mar 06, 2002 at 10:54:09 AM EST

Didn't Kazaa have access to all the files that Morpheus users were sharing because they were both on the FastTrack Network?

So since most of the users of Morpheus have moved on the their new Preview Version, they are no longer sharing their files on FastTrack Network. That may be what is causing the decrease in file shares.

Or am I missing something?

[ Parent ]
I switched... (3.75 / 4) (#19)
by Betcour on Mon Mar 04, 2002 at 04:58:39 AM EST

to Grokster (same network as Kaaza and ex-Morpheus). This is exactly the same application with the same ease of use and easy downloads. I had to neuter the spyware that came with it though...

I tried gnutella or napster networks, and frankly the system Morpheus used is the best by a long shot : being able to download the same file from multiple sources, restart from alternative source and automatic detection of firewalled sources are must-have features.

How to remove the spyware (none / 0) (#31)
by p0ppe on Mon Mar 04, 2002 at 01:17:23 PM EST

http://www.project-insomnia.com/grokster.html


"Democracy is three wolves and a sheep voting on what to have for dinner."
[ Parent ]
Got this, MacPhex uses gnutella (none / 0) (#43)
by kiwipeso on Tue Mar 05, 2002 at 02:02:20 PM EST

MacPhex does allow multiple candidates. if the host of a download leaves or stalls, you just get it from another in the list.
It doesn't restart from alternative sources, it continues from where it left a candidate.
Auto firewall detection is no big thing, it's already there in Phex.
Kaos operating system creator.
[ Parent ]
Let Darwin do his thing.. (2.20 / 5) (#23)
by jabber on Mon Mar 04, 2002 at 10:22:14 AM EST

So the slow and weak are not keeping up with the herd.. What's your point?

[TINK5C] |"Is K5 my kapusta intellectual teddy bear?"| "Yes"

Guess what? (4.00 / 6) (#25)
by YesNoCancel on Mon Mar 04, 2002 at 11:21:49 AM EST

Not everyone lives in an urban area and has access to broadband internet. Also, broadband is expensive (I pay around EUR 110/month for 64 kbit up/512 kbit down with a traffic limit of 3,8 GB per month), so there are plenty of people who don't want to pay that much. Should they be excluded from file-sharing services? I don't think so.

[ Parent ]
Who says 'excluded'? (2.50 / 2) (#26)
by jabber on Mon Mar 04, 2002 at 11:35:32 AM EST

The article seemed to say that slower users were at some sort of 'disadvantage' due to an influx of fast users. They're not.. There's just more users.

If these new, faster users can hop online, share their files and drop offline like everyone else, all this does is increase the number of places slow users can get their files, no?

The addition of broadband Morpheus users is a benefit to the dial-up Gnutella people.. The faster certainly don't slow down the slower, do they?

This sounds like veiled jealousy and resentment of pipe width, and nothing else.

Anyway, back to the point, someone elses access to a faster connection does not exclude a slower connection from carrying on the way it did before. There is no exclusion here.

[TINK5C] |"Is K5 my kapusta intellectual teddy bear?"| "Yes"
[ Parent ]

Read again (4.50 / 2) (#28)
by boomi on Mon Mar 04, 2002 at 12:02:54 PM EST

The problem is the overall network size. As the gnutellaNET gets bigger, more traffic has to be routed by each node. Because of the broadcasting sheme of gnutella, every user incrases the load on the network, however big his pipe. The small ones can't cope with this traffic.

[ Parent ]
Thank Gods it's open source then, huh? (2.00 / 2) (#30)
by jabber on Mon Mar 04, 2002 at 12:32:45 PM EST

So the Amish can tweak it to cap bandwidth, limit the number of connections, etc.

Really, I'm very sorry, but I have no pity for Gnutella. If it doesn't scale, it dies. If it doesn't adapt and evolve, it dies.. It's really a good thing that it can evolve, if people care enough to make it do so. And if it doesn't, then it's just another failed mutation.

[TINK5C] |"Is K5 my kapusta intellectual teddy bear?"| "Yes"
[ Parent ]

Already solved (none / 0) (#37)
by murklamannen on Mon Mar 04, 2002 at 03:31:31 PM EST

This has already been solved, see Ultrapeers. This shields slow users from the traffic but still allows them to share and search files.

[ Parent ]
so what? (4.33 / 3) (#27)
by boomi on Mon Mar 04, 2002 at 11:42:20 AM EST

The slow and weak will die off. That means the herd won't grow :)
Eventually, if another herd grows stronger, it will be crushed.

The gnutella protocol is so inept and unaware, it will be replaced by much better protocols.

Darwinistic, huh?

[ Parent ]
darwinistic (4.00 / 1) (#40)
by nodsmasher on Mon Mar 04, 2002 at 05:53:01 PM EST

not really darwinistic, not very much random mutation here
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Most people don't realise just how funny cannibalism can actually be.
-Tatarigami
[ Parent ]
Unrelated comment (3.00 / 5) (#24)
by CrazyJub on Mon Mar 04, 2002 at 10:51:54 AM EST

People should be taking note, they were warned: Kill Napster and three more will take it's place. Sure enough, here we are. Now they want to take these away?

He he he, you ain't seen nothin' yet.

Errors (4.25 / 4) (#35)
by murklamannen on Mon Mar 04, 2002 at 03:24:51 PM EST

>This is obviously not an ideal situation. People
> with less bandwidth should not be expected to
>bear the full load of a large gnutella node. The
> obvious solution is, of course, creating an
>extension to the protocol that allows searches
> to be "injected" into the network, without the
>originating node needing to be fully connected
>to the network.

Limewire already implements this, it is called Ultrapeers. BearShare and the other servents are working on support for it.
Unfortunatly Gnucleus, which morpheus is a fork of, doesn't have it yet.
Swarming is already implemented by a number of servents.
Metadata searching will soon come to the Gnet too.


>Without an official channel to discuss and
>implement protocol changes, the problem is made
>more difficult to solve.

There is an official channel for discussions about the gnutella protocol which is used by many major gnutella servent developers (both limewire and bearshare are there). It is called the_gdf and is a group on Yahoo groups(www.egroups.com). Their mail-list is very active.
Unfortuantly Morpheus have failed to join, even though they have been invited (except for a comidian who impersonated Steve Griffin (Morhpeus' CEO) on the list =).


>For example, the use of the new version 0.6
>protocol has not become widespread until
>recently, although its specification has existed
> for years. BearShare originally adopted the
>protocol, and created a network of almost
>exclusively BearShare users.

The 0.6 protocol isn't really a protocol. It is just a new handshaking method that allows for negoiating capabilities. This, btw, helps work against fragmentation since servents can implement their own extensions at will and negotiate their use at connection time (instead of creating a seperate network with an incompatible protocol).


Protocol extensions/development (4.25 / 4) (#36)
by murklamannen on Mon Mar 04, 2002 at 03:28:56 PM EST

Btw, check out http://rfc-gnutella.sourceforge.net/ for the new protocol extensions proposals (it is only slightly out-of-date).

There you can also see that the 0.6 handshake was proposed on August 3, 2001, that is, less that a year ago.



Why gnutella ? (4.00 / 2) (#41)
by SoGo on Mon Mar 04, 2002 at 06:21:52 PM EST

I really wonder why they choose the gnutella protocol, after the breakdown of the gnutella net, following the end of napster...
Aren't they other not-patented, which are more advanced ?
Why not mojonation/freenet (OK, the later is harder to implement, still lacks some features) ? What about the edonkey protocol ?

Morpheus and the Gnutella explosion | 45 comments (37 topical, 8 editorial, 0 hidden)
Display: Sort:

kuro5hin.org

[XML]
All trademarks and copyrights on this page are owned by their respective companies. The Rest 2000 - Present Kuro5hin.org Inc.
See our legalese page for copyright policies. Please also read our Privacy Policy.
Kuro5hin.org is powered by Free Software, including Apache, Perl, and Linux, The Scoop Engine that runs this site is freely available, under the terms of the GPL.
Need some help? Email help@kuro5hin.org.
My heart's the long stairs.

Powered by Scoop create account | help/FAQ | mission | links | search | IRC | YOU choose the stories!