Kuro5hin.org: technology and culture, from the trenches
create account | help/FAQ | contact | links | search | IRC | site news
[ Everything | Diaries | Technology | Science | Culture | Politics | Media | News | Internet | Op-Ed | Fiction | Meta | MLP ]
We need your support: buy an ad | premium membership

[P]
The New Internet?

By flamingcow in Culture
Sat Apr 29, 2000 at 04:54:26 PM EST
Tags: Culture (all tags)
Culture

With tools like Gnutella and Freenet appearing, are we facing a revolution in the way information on the Internet is stored and moved? Prompted by attempts to throttle data on the Internet (DeCSS!), people are searching for ways to keep control away from anyone, and keep the Internet as an open community. [editor's note, by rusty] In response to some comments, flamingcow sent me un updated rewrite-- that's why this is much longer than it used to be. :-)


These tools differ from classic Internet data-transmission methods in that there is no centralization. There are no servers that legal bodies can shut down. All data comes from other peers on the network, and the peers IP is reasonably hidden (totally so in freenet). Freenet even optimizes traffic patterns and caches the most used data in the most used areas.

(the word "key" below refers to the hash meaning, i.e. a URL is a key)

The gnutella protocol uses a mass-relaying system (every host relays every packet it receives) along with GUID (globally unique identifier, essentially a random number on every message) to determine whether it has seen the message before. A TTL determines the distance that the message propagates. There is no effort at key-uniqueness or caching on any node (data transfers are still direct source-destination, not going through the network). All clients on gnutella are servers; there is no distinction.

The freenet protocol is completely key-based, and makes sure that a key on the network is always unique. Keys are handed out first-come, first-served. Servers use a stack-based implementation to cache the most used information locally, so retrieval is faster where the information is access the most (making very efficient use of bandwidth at the cost of some space on servers). Freenet is peer-to-peer among its servers, then clients connect to the servers. There is still no centralization, but there is a distinction between clients and servers.

My question is this: Can these services replace the current data systems (FTP, HTTP)? It would certainly be possible to put HTML documents on these sites. Linking is more difficult, but still possible.

(pronoun "he" used below for simplicity)

In order to work with linking on these networks, two types of linking targets (data linked *to*) must be examined: static and dynamic. Static content is something that the author of the linking document knows completely about when he links to it, and it will not change without his knowledge. Dynamic content is a data that will change regularly, but is consistent in some manner (author, format, etc).

Static content is fairly easy to deal with on both services:

On freenet, the key-uniqueness guarantee makes it possible to see this content as static. However, a key may expire out of all caches from disuse, and someone might be able to insert another trojan key in its place. To stop this, a couple of minor additions to HTML would be useful. The A tag could have two attributes added: FILESIZE, a count of the number of bytes in the target file, and MD5, a standard md5 hash of the target file. This would guarantee that trojan files would not be inserted in place of old link targets (a user would not magically get the right file due to this, they would just get a "not found" error when the file was not available or another file had been put in its place.

On gnutella, the two HTML additions described above would also be useful. A search would be done for the filename in the URL. Search results returned from gnutella include the size of the target file; this could be used to screen out obviously wrong files, but would leave the possibility of trojans. Files would have to be downloaded, then MD5'ed, to determine if they were the correct file. If not, a download attempt to another host would be attempted, and so on until the correct file was obtained, the client ran out of hosts, or the user cancelled. Assuming that the network will always have more cooperating hosts than evil trouble-makers (I *am* an optimist), the user should get to the correct file, statistically, on the first try most of the time.

Dynamic content is *much* harder. Neither gnutella nor freenet has any author verification (that would imply a central authority), and without verification there is no way to allow selective replace of old keys (freenet) or determining if the correct person has authored the new content (gnutella). I have trouble seeing a logical way around this, and this is where I would throw this out to kuro5hin.

Sponsors

Voxel dot net
o Managed Hosting
o VoxCAST Content Delivery
o Raw Infrastructure

Login

Related Links
o Kuro5hin
o Gnutella
o Freenet
o Also by flamingcow


Display: Sort:
The New Internet? | 33 comments (33 topical, editorial, 0 hidden)
This is the way Akamai already work... (none / 0) (#1)
by rusty on Sat Apr 29, 2000 at 02:48:20 PM EST

rusty voted -1 on this story.

This is the way Akamai already works-- you insert a key in, say, an image URL, and their servers decide what to send you (and which server to get it from) from a highly distributed set of sources.

Interesting thoughts. I'd much prefer this be written up a little longer, to explore the new protocols and talk about how they actually work (we know they're decentralized. But How?), and perhaps lay out a sketch of what the "New Internet" would really be like. Just to be perfectly clear, I like it, but there's not enough original thought here. Please rewrite, and perhaps submit to editors@kuro5hin.org. And yes, I'm going to fix the voting stuff. :-)

____
Not the real rusty

Hm. So would it be possible to set ... (none / 0) (#6)
by Field Marshall Stack on Sat Apr 29, 2000 at 02:53:56 PM EST

Field Marshall Stack voted 1 on this story.

Hm. So would it be possible to set up an entire "shadow WWW", on freenet, with at least a reasonable amount of the functionality of the original? That would so rock...until every last country in the world made it illegal.... :(
--
Ben Allen, hiway@speakeasy.org
"Nobody ever lends money to a man with a sense of humor"
-Peter Tork

Re: Hm. So would it be possible to set ... (none / 0) (#11)
by skim123 on Sat Apr 29, 2000 at 08:19:02 PM EST

What would be the advantage of doing this? What's wrong with the Internet? While it may seem like the Internet is getting censored every now and then (i.e., the AP photos in the WHAZZUP Elian spoof), but even in those instances, the video made its way to other servers on the Internet.

So why would we want such a system? It seems like it would just be a slower, smaller, and less reliable alternative to the Internet.

Money is in some respects like fire; it is a very excellent servant but a terrible master.
PT Barnum


[ Parent ]
Re: Hm. So would it be possible to set ... (none / 0) (#17)
by Tr3534 on Sat Apr 29, 2000 at 10:54:57 PM EST

It seems like it would just be a slower, smaller, and less reliable alternative to the Internet.

actually, with freenet, the content is dynamically cached throughout all the nodes. say people on one corner of the network keep requesting a file on the other side, the nearby node caches the file. instead of everyone dragging it across the whole net, it suddenly is moved to right where they are, improving performance.

as for reliabilty, take down a node and the others simply route around it. the users not using that node won't even notice.
Sigmentation Fault: Post Dumped.
[ Parent ]
Re: Hm. So would it be possible to set ... (none / 0) (#19)
by skim123 on Sun Apr 30, 2000 at 01:59:00 AM EST

actually, with freenet, the content is dynamically cached throughout all the nodes. say people on one corner of the network keep requesting a file on the other side, the nearby node caches the file. instead of everyone dragging it across the whole net, it suddenly is moved to right where they are, improving performance.

as for reliabilty, take down a node and the others simply route around it. the users not using that node won't

Good points, failed to remember that fact about FreeNet. The disadvantage that seems to still remain, though, is the inability to do anything too terribly useful. Removing the dynamic nature of the Web and it becomes pretty useless, IMO. A place to share static information, yes; a place to buy things, to book flights? No.

Money is in some respects like fire; it is a very excellent servant but a terrible master.
PT Barnum


[ Parent ]
Re: Hm. So would it be possible to set ... (none / 0) (#32)
by your_desired_username on Sun Apr 30, 2000 at 10:02:20 PM EST

A place to share static information, yes; a place to buy things, to book flights? No.

This is possibly as much an advantage as a disadvantage.

It may mean that large repressive corpations will be much slower to take intrest in it, and therefor slower to realize its potential. Slower to try to regulate it, possibly.

Freenet will be less popular, and less likely to be dominated by comericial interests. Less likely to become a successor to TV like the www has become.

Higher S/N ratio. Less spam. Less advertising.

[ Parent ]
This, along with standards for doub... (none / 0) (#3)
by Inoshiro on Sat Apr 29, 2000 at 03:13:52 PM EST

Inoshiro voted 1 on this story.

This, along with standards for double-linking via anchor tags and HTTP 1.2 (:)), a way to charge people not on the white-list a fee for sending email (effectively stopping all forms of spam), etc, would make the Intenet a much better place.

Although I doubt these protocols will replace, they will likely suplement the existing standards



--
[ イノシロ ]
wow, this is a big topic. I can't ... (none / 0) (#2)
by ramses0 on Sat Apr 29, 2000 at 03:15:15 PM EST

ramses0 voted -1 on this story.

wow, this is a big topic. I can't vote for it because it's not specific enough. to summarize (i think):

Distributed, anonymous networks are cool. They have nifty side effects which add useful benefits to document request and retreival performance. Would it be good to ~hack~ your standard HTTP clients (browsers) and ftp clients to support navigating documents in a freenet? Discuss below.
[ rate all comments , for great justice | sell.com ]

I don't think that's ever going to ... (none / 0) (#5)
by evro on Sat Apr 29, 2000 at 03:17:56 PM EST

evro voted 1 on this story.

I don't think that's ever going to happen. The problem is that there are hundreds of millions of computers using "the old system," and with an installed userbase like that, change is pretty much impossible. That's why I don't think even upgrading to IPv6 will go well; if anybody decides not to do it, the whole plan fails. There was a story on I believe C|Net a couple of days ago about a "new internet" that had censorship workarounds built into it and some other cool features, but I think the same problem applies to them.
---
"Asking me who to follow -- don't ask me, I don't know!"

Re: I don't think that's ever going to ... (none / 0) (#18)
by Anonymous Hero on Sun Apr 30, 2000 at 12:21:35 AM EST

Posting as a lurker, your comment has moved me to finally post. Censorship? "Cool feature"? Censorship has, historically, led to the retardation of technological innovation and cultural development. It's also a way to keep the population at large docile. Why would you *want* this? Bomb information on the web doesn't kill people, people kill people. Usenet porn doesn't rape/molest anybody, rapists/molesters rape/molest people. Since when has information itself been harmful, in general? As for the ipv6 section of your comment, why wouldn't ipv4<->ipv6 bridges work? They already exist today, IIRC.

[ Parent ]
censorship _workarounds_ (none / 0) (#29)
by joeyo on Sun Apr 30, 2000 at 02:28:02 PM EST

"...a 'new internet' that had censorship workarounds built into it..."

I agree that it's vague but I think he is saying that the censorship workarounds are the cool part :)

--
"Give me enough variables to work with, and I can probably do away with the notion of human free will." -- demi
[ Parent ]

However, if information systems are... (none / 0) (#7)
by enthalpyX on Sat Apr 29, 2000 at 03:50:59 PM EST

enthalpyX voted 1 on this story.

However, if information systems are so decentralized, will it be possible to find anything of value?

Re: However, if information systems are... (none / 0) (#13)
by skim123 on Sat Apr 29, 2000 at 08:22:33 PM EST

However, if information systems are so decentralized, will it be possible to find anything of value?

Good point. I think one way you could do it is keep track of how often something was accessed and add "value" to that. Of course this would likely result in showing porn as containing the most value on the Net. Hmmmm...

Money is in some respects like fire; it is a very excellent servant but a terrible master.
PT Barnum


[ Parent ]
Re: However, if information systems are... (none / 0) (#23)
by slycer on Sun Apr 30, 2000 at 02:59:33 AM EST

Right, but gets rid of SPAM. If it's not requested, it fizzles out. Now that is
a nifty feature of freenet :-)


[ Parent ]
Re: However, if information systems are... (none / 0) (#25)
by skim123 on Sun Apr 30, 2000 at 03:27:28 AM EST

I agree that it would get rid of spam (unless spammers wrote scripts to frequently access their spam, so it seems as though everyone is looking at the spam) but what makes you think I like what everybody else likes? What if I have some interest that very few others share, then my content fizzles out, while mainstream stuff explodes.

Money is in some respects like fire; it is a very excellent servant but a terrible master.
PT Barnum


[ Parent ]
Re: However, if information systems are... (none / 0) (#30)
by joeyo on Sun Apr 30, 2000 at 02:31:36 PM EST

I don't know if it is written into the spec, but you could presumeably flag data on your own node to reside there even if it becomes wildly unpopular. It just wouldn't be fruitfull and multiply like the mainstream stuff. :)

--
"Give me enough variables to work with, and I can probably do away with the notion of human free will." -- demi
[ Parent ]

Isn't this already a problem with the current web? (none / 0) (#33)
by marlowe on Mon May 01, 2000 at 05:58:15 PM EST

Search engines are overloaded, with backlogs measured in months.  Plus, they're
full of keyword spam.  So, we try to cope with word-of-mouth mechanisms.  But
all that is a separate issue.

--- I will insist on my right to question ---

-- The Americans are the Jews of the 21st century. Only we won't go as quietly to the gas chambers. --
[ Parent ]
Good idea, and some juicy problems.... (none / 0) (#8)
by warpeightbot on Sat Apr 29, 2000 at 04:47:48 PM EST

warpeightbot voted 1 on this story.

Good idea, and some juicy problems. I'm looking forward to some good ideas from bigger brains than mine... The virtual network idea is the way we can keep Big Brother from robbing us our freedom completely. We just have to get it right.

I understand where the authors of s... (none / 0) (#4)
by starlitz on Sat Apr 29, 2000 at 04:54:26 PM EST

starlitz voted 1 on this story.

I understand where the authors of such services are coming from. Censorship of controverisal ideas is far too easy with the current internet. Not to mention corporate legal bullying.

FreeNet takes care of the unjust censorship problem, but introduces many more. What happens if someone posts a libelous document about me? That document will be there until it expires from all nodes. There is no way I can have it removed, and no way I can easily identify the original poster the offending article.

Unfortunately, there is no easy way to resolve these complex issues. It seems to me the easiest way to handle this is by reforming the justice system, rather than making it impossible to track down people who some might find offensive, as there are a lot of valid complaints against people on the internet. Of course, politics would most likely make reformation ineffective as well.

I think that's enough ranting.

Re: I understand where the authors of s... (none / 0) (#10)
by Anonymous Hero on Sat Apr 29, 2000 at 06:16:25 PM EST

What happens if someone posts a libelous document about me? That document will be there until it expires from all nodes.

If people want free speech, they need to accept the problems that come along with it. People should already know not to believe everything they read on the internet. If someone says something untrue about you, post a document telling your side of the story.

[ Parent ]

Re: I understand where the authors of s... (none / 0) (#16)
by starlitz on Sat Apr 29, 2000 at 10:30:58 PM EST

If people want free speech, they need to accept the problems that come along with it. People should already know not to believe everything they read on the internet. If someone says something untrue about you, post a document telling your side of the story.

While that is a nice idea in theory, it is unlikely to adequately address the problem.

If some child says I abused them, when I did not, I would be labeled "scum" and judged by the public based on the allegations. I would then be treated differently than if the allegations had not been posted, thus restricting MY freedom. If I say that nothing happened, who would believe me?

Free speech does not grant the right to deny rights to someone else. Don't get me wrong. I am not pro-censorship, but I think that the structure of the FreeNet would be far too easy to abuse.

[ Parent ]

Re: I understand where the authors of s... (none / 0) (#22)
by Anonymous Hero on Sun Apr 30, 2000 at 02:50:03 AM EST

How is this different from the current system? What does it matter if the information is taken from the net, if many people have already seen it and are propagating the information by other means, such as telling their friends etc.

So it makes no difference whether the offending document is or is not taken off the net. If you get labeled as something, then you simply are labeled as something, unless you express your point of view which convinces the rest of the people to change their minds.

Your child-abuse example is not any different than in the current system. Suppose I put out a web site claiming such a thing. What could you do? You could try getting it taken down. But since you don't know who and where I am, I could just put up another site and so on.

[ Parent ]
Re: I understand where the authors of s... (none / 0) (#26)
by skim123 on Sun Apr 30, 2000 at 04:14:18 AM EST

Suppose I put out a web site claiming such a thing. What could you do? You could try getting it taken down. But since you don't know who and where I am, I could just put up another site and so on

What do you mean? He would know exactly where the site was being run from. If you were using a GeoCities type page, he could contact GeoCities and threaten a lawsuit if they didn't take it down.

Money is in some respects like fire; it is a very excellent servant but a terrible master.
PT Barnum


[ Parent ]
Re: I understand where the authors of s... (none / 0) (#27)
by PerlGeek on Sun Apr 30, 2000 at 09:01:26 AM EST

There are already ways to post information anonymously, in such a way that it
gets heard. I believe that was the point of what he said, and I agree with him.
Anonymous websites are just the beginning.

Already, anyone can be labeled as anything by anybody. No matter how many "fire
in a theater" laws and libel laws we make, this will not change.

It's only fair to let you know that my best friend's dad was a high-ranking
public official, and he was slandered by the local newspapers. He then lost his
job. He was considering sueing the newspapers for libel, but decided not to,
and I respect him greatly for that. AFAIK, his reasons were, one, it's not
worth it, two, God has a plan for him in some other line of work, and that plan
doesn't include revenge.

If we want free speech, we're going to have to accept the risk of libel. If we
sacrifice free speech to stop libel, free speech will be gone and libel will
still exist, as we have just seen.

I'm behind Freenet and Gnutella 100 percent, no reservations.


[ Parent ]
Re: I understand where the authors of s... (none / 0) (#28)
by Anonymous Hero on Sun Apr 30, 2000 at 01:48:39 PM EST

If somebody anonymously libels you on Freenet, people wouldn't be as likely to
take trust the honesty of the libeler.	However, if somebody includes their
identity, you would stil be able to sue them and vindicate your reputation,
even if you couldn't force a removal of the document posted about you.

I would imagine that if libel becomes a problem on Freenet, people will develop
sound strategies of determining the truth of one person's testimony about
another.  Anonymous testimonies will be less likely to be trusted, unless of
course they're in accordance with many other testimonies. 


[ Parent ]
Re: I understand where the authors of s... (none / 0) (#31)
by Notromda on Sun Apr 30, 2000 at 08:44:54 PM EST

Problem is, people are just dumb: they will believe an anonymous posting, and there's just nothing that can convince people otherwise. I had a public forum on the home page for our dialup customers - one or two people, posting anonymously, were making it sound like our service was awful. A few more people jumped on, making it sound like we were about to lose all of our customers. The only thing I could do was shut down the whole forum. The problem went away overnight, and I have never gotten a complaint since then. The anonymitity just made people go beserk!

Censorship? yeah, I guess it was, but it was our own corporate page, and they were posting flat out lies and trash. There were plenty of people that could have easily refuted the troublemakers, but they tend not to speak up. Posting replies that point out the truth did no good - the troublemakers just keep posting. Whack a mole. Anonymous troublemakers just simply have far too much power, as too many people believe anything they read on the Internet.

[ Parent ]

Re: I understand where the authors of s... (none / 0) (#21)
by Anonymous Hero on Sun Apr 30, 2000 at 02:43:16 AM EST

Suppose someone does post a "libelous document" about you. Does it hurt your ego that much? You can publish a correction, and be done with it.

It's simply too big a job to change the Old System (justice system, corporation, goverments, ...), so instead we should just make something new!

[ Parent ]
Re: The New Internet? (none / 0) (#9)
by Camelot on Sat Apr 29, 2000 at 05:23:32 PM EST

Why bother with HTML ? Sure, it would with the existing browsers, but surely for anything like this ("new Internet", as you say), XML would be the way to go.

Re: The New Internet? (none / 0) (#12)
by bse on Sat Apr 29, 2000 at 08:21:54 PM EST

ok, im reading this, and all the comments, and i agree with most of it. but i
think youre missing one thing.. "new internet". but the internet has always
been a network of decentralised peers sharing off each others information
(basically)..  shouldnt we be mearly thinking "new WWW" or "WWW replacement",
especially with the *ML related comments.

FreeNet looks like a very cool project. gnutella, well i dont know. ahvent
really looked into that ;) and might just spread over the world and take over
where WWW missed out.. if done correctly of course, could turn into a WWW
killer. freenet that is, or an evolution of it, at least. "homepages", chat
groups, corporate sites, all co-locating the same place. i know we sort of have
this with the likes of dmoz and yahoo and all the other many directories out
there. but all this is in one relative place.. with a neat stack based dynamic
network thing, keeping it fast and reliable.

and no, i have no affiliation with freenet.. just my thoughts on the system.
very "Matrix" if you ask me.. matrix as in the neuromancer series, not as in
the movie (even though they do share their similarities)  ;)

---
"Please sir, tell me why, my life's so pitiful, but the future's so bright? When I look ahead, it burns my retinas." -- Pitchshifter - Please Sir

Re: The New Internet? (none / 0) (#14)
by ejf on Sat Apr 29, 2000 at 09:01:12 PM EST

Distributed, anonymous, robust networks are a plus these days. I personally
would like to see freenet accepted on a basis more broad; It practically
prevents censorship. Of course it opens the way for illegal activity, too, but
that is the price one has to pay for freedom. The U.S. has always proclaimed to
have paid a huge price for freedom -- is this still true today ? We shall see
...

The issue of updating a document already in place could be sorted out with
public key cryptography I think. Just sign the original document, and if you
want to post an update, youŽll have to sign it with the same key. This does not
disclose your identity; you could even build an anonymous identity using this
(just sign all of the files pertaining to that identity with the same key).

The only problem I see is to get enough hosts to make the net truely robust;
not many people are willing to pay the price for freedom. I sincerely hope this
takes off to complement the rest of the net.

--- men are reasoning, not reasonable animals.
Re: The New Internet? (none / 0) (#15)
by geekpress on Sat Apr 29, 2000 at 09:41:11 PM EST

I can see the problem of getting sysadmins et al to adopt new standards; for many of them, the cost of the upgrade will not be worth the benefit of privacy. But that's not such a serious issue, for if end users want the upgrades, they will eventually happen.

However, the real hurdle, in my opinion, lies in convincing enough people to use a system like FreeNet for legitimate, legal activity. If that does not happen, governments will easily be able to shut it down -- after all it's only used by nuclear terrorists, child pornographers, drug dealers, and money launderers (Tim May's Four Horsemen).

FreeNet could do spectacularly well if, for example, it allowed online businesses to conduct sales more securely than on the web. Having dissidents from other contries use it to share information about human rights abuses would probably also be a boon. What other types of activity could help legitimize it in the eyes of the populace? Any ideas?

-- Diana Hsieh
GeekPress: Today's Tech News, Sifted and Summarized

Re: The New Internet? (none / 0) (#20)
by Radagast on Sun Apr 30, 2000 at 02:13:59 AM EST

What it's used for isn't that important, given that it's impossible to shut down (without shutting down the whole Internet). But, I see the point, having some good, strong free speech issues would make it even more untouchable.

One idea that springs to mind which is both worthwhile in itself and an excellent way to make the system politically completely untouchable is to gather holocaust survivor testimonials and upload them wholesale. Given a few hundred, or better, a few thousand of these in the network, it's going to be awfully hard for any politician or law enforcement agency to crack down on it. Which I have to say is a shame, because I'm principally opposed to any subject being untouchable like this, but as long as this is the situation, we might as well take advantage of it.

[ Parent ]

Re: The New Internet? (none / 0) (#24)
by Anonymous Hero on Sun Apr 30, 2000 at 03:04:11 AM EST

I don't know about the rest of you, but I'd rather keep the corporations off Freenet (whenever it is that it is ready). They can have the web for all I care, they seem to act like they own it anyway.

I think strong cryptography (modular plugin/plugout architechture so you could change ciphers) should be transparently built-in. And as for the number of people running the software, I hope it would be like running a distributed.net client. I've been running it for over a year now, and sometimes I forget it's there. It just runs happily, and starts automatically in the rare occasions when I have to boot the machine and so on. I'd like to see such a fire-and-forget approach with Freenet.

I think the basic services what Freenet needs to have built in are:
  • person-to-person communication: e-mail, IRC like protocol
  • computer-to-computer communication: file transfers, FTP-like protocol

The rest, like news and www-like thing could be built on top of these. Of course the architechture needs to be such that people can easily create new protocols and services, like in the Old Internet.

[ Parent ]
The New Internet? | 33 comments (33 topical, 0 editorial, 0 hidden)
Display: Sort:

kuro5hin.org

[XML]
All trademarks and copyrights on this page are owned by their respective companies. The Rest © 2000 - Present Kuro5hin.org Inc.
See our legalese page for copyright policies. Please also read our Privacy Policy.
Kuro5hin.org is powered by Free Software, including Apache, Perl, and Linux, The Scoop Engine that runs this site is freely available, under the terms of the GPL.
Need some help? Email help@kuro5hin.org.
My heart's the long stairs.

Powered by Scoop create account | help/FAQ | mission | links | search | IRC | YOU choose the stories!