Kuro5hin.org: technology and culture, from the trenches
create account | help/FAQ | contact | links | search | IRC | site news
[ Everything | Diaries | Technology | Science | Culture | Politics | Media | News | Internet | Op-Ed | Fiction | Meta | MLP ]
We need your support: buy an ad | premium membership

Finding the good stuff on the Net

By Eloquence in News
Wed Jul 12, 2000 at 09:03:47 PM EST
Tags: Internet (all tags)

Often when dealing with "hot-button issues" like gun-rights, abortion-rights or evolution vs. creationism I have a hard time to locate sites that have a good selection of information on the topic. For example, on the topic of evolution, this site is an excellent repository of information--but I only found when some guy linked to it in a guestbook. The problem is that especially with such issues, there is such a vast number of sites that it's extremely hard to find the relevant ones if you don't have a starting point.

You come across a lot of sites just saying stuff like "Hey, my name's Joe Average, and I like guns! Every American should have one ..". Even the official organizations dealing with the issues often have crappy sites. Then, on the other side, there are often huge, excellent, but hard-to-find info-collections. Here are some thoughts on how to possibly overcome this problem, and some questions.

I have written an article on ShouldExist, a Scoop-powered site for collecting ideas, that suggests the creation of a decentralized database of ratings for products, websites, services etc. You could basically connect to a Gnutella-like network and query for all ratings on a certain product, by a certain person, on a certain category etc. (also see this article by yours truly on how such a database could be built). This would allow users to select the highest-rated websites from a list on a topic. Perhaps people could create profiles for themselves so that I could look for ratings by people who are pro-choice, anti-gun etc.

Another solution might be to include such a rating system in dmoz, an IMHO terrible looking, but well working, open website directory similar to Yahoo.

Basically, I don't care much how it's done, as long as I get independent ratings and reviews on stuff that interests me. My questions are:

  • Do you know of websites, especially non-commercial ones, which do this kind of stuff? Deja comes to mind, but they only rate products and are obviously commercial (therefore they will a) collect all private data they can gather, b) delete ratings if it's in their commercial interest).
  • Who would be willing to build something like that? I believe it should be done soon before a corporation builds a monopoly on the need.
And, of course, feel free to throw in any good links on the topics I mentioned ;-).


Voxel dot net
o Managed Hosting
o VoxCAST Content Delivery
o Raw Infrastructure


Related Links
o Scoop
o Yahoo
o this site
o article
o ShouldExis t
o this article
o dmoz
o Also by Eloquence

Display: Sort:
Finding the good stuff on the Net | 29 comments (27 topical, 2 editorial, 0 hidden)
You're right (none / 0) (#1)
by scriptkiddie on Wed Jul 12, 2000 at 07:38:58 PM EST

On the Web, it seems like startlingly little content ever actually gets through.

It sounds to me like you're advocating a site to trade opinions, about products, politics and everything else. I think that bulletin-board format sites are not all that great for this kind of sharing, since opinions are made in the heat of the moment - some kind of system where users could post carefully meditated opinions - and later change them! - could be a valuable resource.

Re: You're right (3.00 / 2) (#12)
by fluffy grue on Thu Jul 13, 2000 at 12:59:34 AM EST

"Is not a quine" is not a quine.
I have a master's degree in science!

[ Hug Your Trikuare ]
[ Parent ]

WikiWiki (none / 0) (#17)
by SpiderBoris on Thu Jul 13, 2000 at 06:31:27 AM EST

A WikiWiki web would seem to do most of what you're asking for - it provides a place where anyone can post their own opinions on a subject, and because all of the pages are world writable, you can change anything you've (or anyone else) has written at a later date.

Check out: this intro page

Of course, this relies heavily on the good behaviour of the users, but it seems to work quite well...

-- Cut off my head to email me...
[ Parent ]

Epinions (none / 0) (#2)
by eann on Wed Jul 12, 2000 at 08:21:19 PM EST

Check out Epinions. It's not exactly what you described (especially in its lack of total openness), but it may give you better ideas for a starting point to see what has come before.

Our scientific power has outrun our spiritual power. We have guided missiles and misguided men. —MLK

$email =~ s/0/o/; # The K5 cabal is out to get you.

Epinions style search engine (none / 0) (#25)
by birchcap on Thu Jul 13, 2000 at 02:08:57 PM EST

One aspect I like about epinions is it's external link rating system. It puts a little bar at the top of the screen and lets you rate the site once you've looked at it.

A search engine that included a simple site rating frame could quickly become very useful. With the web of trust system that they use, it could even be somewhat spam resistant.

Unfortunately, epinions' structure is heavily product centered.

[ Parent ]

Re: Epinions style search engine (none / 0) (#29)
by eann on Mon Jul 17, 2000 at 09:09:54 AM EST

I don't think of Epinions as product-centered. I had sort of ignored them for a long time, never bothering to check the place out, and came across it when I was looking for some info about the Painted Desert in Arizona. A trip to a national park isn't exactly a "product", but the model fit anyway.

And it reminded me that at least one out of four people is a complete moron.

Our scientific power has outrun our spiritual power. We have guided missiles and misguided men. —MLK

$email =~ s/0/o/; # The K5 cabal is out to get you.

[ Parent ]
what you're looking for (sort of, it's commercial) (none / 0) (#4)
by madams on Wed Jul 12, 2000 at 08:32:14 PM EST

Apple Computer has an excellent site called iReview, which bills itself as a guide to "what's good on the web and what should be better". iReview is a collection of categorized, rated reviews for websites. While not as extensive as one might hope, the reviews are surprising well written, and the site doesn't avoid giving negative reviews. While it isn't the user created space you are looking for, iReview does allow you to suggest sites you want the editors to review, submit your own review, or rate already reviewed sites (you have to be an "iTools member", however, which just includes registering with the site, as far as I can tell, since I haven't done so).

Perhaps an open site like this is what you're looking for. Adding ratings and user comments to a site like dmoz.org (since it is "open") might be a good way to get started.

Mark Adams
"But pay no attention to anonymous charges, for they are a bad precedent and are not worthy of our age." - Trajan's reply to Pliny the Younger, 112 A.D.

We need the software for it (none / 0) (#6)
by Yzorderex on Wed Jul 12, 2000 at 08:41:17 PM EST

I've been looking for something like this and it ain't easy. Wanted specifically a CGI for rating Pattaya (Thailand) hotels. Ratings with comments on something that has never changing data (location), slow changing data (prices).
Nothing really satisfies. Admin time is too extensive for this unless its commercial, besides moderation to keep out flamewars and idiots.
may have to write it :-))

distributed ratings for websites (none / 0) (#7)
by dash2 on Wed Jul 12, 2000 at 09:19:55 PM EST

for websites, but not for other products, check out Opencola who are developing a distributed search engine which relies on users to provide data to each other. I'm not clear exactly how it works - top secret? - but it might be suggestive.
If I speak with the tongues of men and of angels, but have not love, I am become sounding brass, or a clanging cymbal.
dmoz getting ratings (none / 0) (#8)
by noek on Wed Jul 12, 2000 at 09:45:48 PM EST

I was gonna rant here about that you call dmoz teribbly looking. I won't do it. Taste is personal.

If you would put ratings on links added to the directory it would become more spam sensitive. Already some categories like Adult and Business are subject to spam.

At dmoz sites are rated by the editors. This means if some site is above average in one category it will be "cooled" and will show up on top. For more info see ODP guidelines (i don't know if this link works for non-editors)

Re: dmoz getting ratings (2.00 / 2) (#11)
by bgdarnel on Thu Jul 13, 2000 at 12:09:39 AM EST

I agree with noek - dmoz looks fine as it is; there's no need to try and spice things up. I like the idea of dmoz simply being the data repository; remember that it's open, so other sites can put their own pretty face on the data. Of particular relevance to this topic is directory.google.com which applies Google's page-ranking algorithm to ODP data.

[ Parent ]
Re: dmoz getting ratings (none / 0) (#13)
by vagary on Thu Jul 13, 2000 at 01:03:34 AM EST

As a Dmoz editor, I must express my feeling that coolness is inadequate. I can cool up to two sites, but that's all I can do to effect sort order. What we need is a bunch of different flags to apply. Some which can only be used once per category, some to specify what kind of site it is (eg: commercial/link/etc.).

As for the Dmoz project, I feel it's coming along really great -- it's often my first stop for information. I just wish that the backend code was as free as the content so that I could implement all the improvements I've got in mind. :)

[ Parent ]
Re: dmoz getting ratings (none / 0) (#18)
by noek on Thu Jul 13, 2000 at 08:00:40 AM EST

"I just wish that the backend code was as free as the content so that I could implement all the improvements I've got in mind."

I don't know if you had a look on these tools. Maybe you could adjust some of the Perl tools to add ranking. Also an other editor is working on a project to use the RDF data for a new search engine.

[ Parent ]

Try Using Google (3.70 / 7) (#9)
by matta on Wed Jul 12, 2000 at 09:54:42 PM EST

Your example of a link that was very hard to find -- an evolution FAQ on www.talkorigin.com -- was only the 3rd link listed on a Google search of the word "evolution."

Google rates web pages higher if other web pages link to them, giving more weight to links coming from sites that are already highly rated. The truly popular sites tend to bubble towards the top.

Take a look at this idea... (none / 0) (#10)
by eries on Wed Jul 12, 2000 at 10:20:39 PM EST

This is at the heart of an <a href="http://www.kuro5hin.org/?op=displaystory;sid=2000/6/19/19337/5166">idea I posted a while back</a>. I think the concept of feedback (in the strict sense used in systems analysis) is critical to the future of the web. But that's for another article...
Promoting open-source OO code reuse on the web: the Enzyme open-source project
cutting out the crap (3.00 / 2) (#14)
by Digambaranath on Thu Jul 13, 2000 at 04:56:10 AM EST

Well, I checked out dmoz, and my site came out number 8 after searching on "philosophy linguistics", so they can't be that bad!

I don't think Google's link-counting is terribly reliable because:

  1. Some good sites don't attract links because of their nature (e.g. academic papers);
  2. Similarly, some types of site attract links like flies (e.g. teenage homepages);
  3. Commercial sites are savvy to link-counting and generate spurious links just for this purpose.
What I would really like is a search engine that lets me control the fitering. You can do some things by using negative keywords, but I would really like something that prioritises ceretain domains or website types and downgrades others.

Just the FAQs mam (5.00 / 1) (#15)
by pharm on Thu Jul 13, 2000 at 05:50:41 AM EST

I find the FAQs from a relevant usenet newgroup can be a good resource for this kind of thing. Not necessarily on every topic, but most go out of their way to be even-handed and give pointers to all sides of the 'hot-button' debates that characterise that particular newsgroup. If they don't then there's usually a 'couter-FAQ' to fill in the gaps :)

Apart from that, yahoo and dmoz are useful, and google is fantastic. nb. To the poster who said that google can be subverted by sites 'linking to themselves': it ain't necessarily so...google's algorithms are supposed to be naturally resistant to that kind of self regarding linking. A closed set of links from pages which aren't themselves rated very highly will not do much for the rating of a given page.

have a look at the Web Searchlores (1.00 / 1) (#16)
by Anonymous Hero on Thu Jul 13, 2000 at 06:06:26 AM EST

you can find it here or here.

DirectHit.com sort of does what you're talking abo (none / 0) (#19)
by Anonymous Hero on Thu Jul 13, 2000 at 08:10:07 AM EST

Its search engine returns hits in order based on ratings of how many people actually clicked on them before, or some other such metric. So when you search for say Picts, you get some pretty useful sites right at the top.

DirectHit ain't that great (none / 0) (#28)
by ToastyKen on Sat Jul 15, 2000 at 10:56:34 PM EST

I find Directhit not to be that useful. Their system seems to be a little too self-reinforcing, for one thing.

[ Parent ]
The Freedom Forum (none / 0) (#20)
by ronfar on Thu Jul 13, 2000 at 08:31:05 AM EST

Located at http://www.freedomforum.org, does a good job of chasing stories on First Amendment and other Bill of Rights issues.

I often find out about things going on with the legislature and the courts that I wouldn't know otherwise.

They have sections devoted to Freedom of Religion, Speech, the Press, Technology and Assembly.

Jon Katz haters beware, he also writes the occaisional column for this site.

Simpli.com (3.00 / 1) (#21)
by madams on Thu Jul 13, 2000 at 10:03:52 AM EST

A great search engine that hasn't gotten a lot of press is Simpli.com.

Simpli.com searches for pages based on the meanings of words. For example, if you type "Latin" into SimpliFind, it comes back with:

Latin Music
Other... (add your own!)

SimpliFind also knows that Joan Fontaine is an actress and that Richard Stallman is a hacker, for instance.

Simpli.com was founded by cognitive scientists, so the search engine is backed by a lot of linguistiv theory.

But SimpliFind is still a search engine, and as such, it isn't perfect. For example, it doesn't know who Frank Gehry or Rem Koolhaas are, but the restults it returns on those two names are still pretty good.

Mark Adams
"But pay no attention to anonymous charges, for they are a bad precedent and are not worthy of our age." - Trajan's reply to Pliny the Younger, 112 A.D.

Re: Simpli.com (none / 0) (#26)
by mezzo on Thu Jul 13, 2000 at 04:03:44 PM EST

hrm.. i tried it out for a few minutes and it still seems rather rough.
it doesn't say how many results a search returned.
you need to click the search button twice to get the results.
foreign words tend to result in '(none)' in that simplifind box.


"The avalanche has started, it is too late for the pebbles to vote."-- Kosh
[ Parent ]
Re: Simpli.com (none / 0) (#27)
by Anonymous Hero on Sat Jul 15, 2000 at 07:39:05 AM EST

I can't get it to respond at all. Looks like it's down.

[ Parent ]
Popularity and Community ranking Search engines. (none / 0) (#22)
by ctm on Thu Jul 13, 2000 at 10:20:10 AM EST

A lot of search engines now try to base its metrics on the popularity of a site (or in some cases how much they paid). Unfortunately information on websites can be very sporadic. Even bad design can prevent someone from finding information that they need or prevent the popularity of a site from growing. This doesn't invalidate the data contained, but if it has the one crucial piece you need it could be frustrating. I've found that community based search engines aren't necessarily the answer. Just as people here at kuro5hin complain about stories being hung in the queue its the same in the search engines. Some people like the site, some people hate the site. Most people are indifferent on what they are ranking. I recently had a notice that some website had added me to be evaluated by their "many" reviewers. Essentially they pay people to rate webpages. I later recieved a message that one(1) person reviewed it and they said it was fast, and well designed(not exactly brilliant and you could say that about 25% of the webpages out there). Now I know they have more than one person doing it, but unless you're paying people in the form of a salary I've found that community ranking isn't exactly the best answer(obviouly if they do get a salary then you don't get as many opinions and it isn't the best answer either).

I don't have an answer to be honest. I think that if one is to review a website it needs to have more than one metric. I can think of at least 3 catagories. Design, Content, Speed. The problem with this is that this has its own problems. This is a lot more time consuming, and I normally don't go back to the search engine once I've found the information I need. When I'm searching for information, I rarely take time to stop and smell the roses and think that this would probably translate into some skewed ratings of sites.
Which is worse ignorance or apathy?

Who knows? Who cares?

Recent Paper on the Subject (none / 0) (#23)
by cthulhu on Thu Jul 13, 2000 at 11:54:18 AM EST

There was recently (within 3 months) a paper on the subject of autheticated searches published at a major recent AI conference. I didn't attend the conference, nor did I read the paper, but I did hear a fairly in-depth description of the methodology from a colleague.

Essentially the system used a probability model that used known, trusted sources of 'authenticity' and then examined the hyperlinks from that site to external (and internal) pages. A probability model was developed such that a page (or site) with many external links from authenticated sources was itself considered a highly probable authentic source.

Thus, the set of trusted sources is ranked by probability and expanded. Of course there were many sources of noise introduced into the probability tables, such as advertising et. al. If I remember right, it was mentioned to me that the author addressed several of these issues.

If anyone is really interested I'm sure I can dig up a reference to the paper.

The art of web searching... (4.00 / 1) (#24)
by skeezix on Thu Jul 13, 2000 at 01:01:52 PM EST

Web searching is more of an art than a science, I think. One of the most valuble tools I've found is Google. If you learn what keywords will give you the best information, it is unbelievably powerful. For example, I typed (and this was on the first try), "evolution creation controversy" and hit the "I feel lucky" button. Guess where it took me. Yep, talkorigins.org.

To give another example, a friend asked me the other day what whether a higher or lower dot pitch was better when looking to buy a computer monitor. Then he wondered what exactly dot pitch was. So I went to google, typed, "dot pitch explained" and hit the "I feel lucky" button. It took me to this page which explained dot pitch in great detail. :)

Finding the good stuff on the Net | 29 comments (27 topical, 2 editorial, 0 hidden)
Display: Sort:


All trademarks and copyrights on this page are owned by their respective companies. The Rest 2000 - Present Kuro5hin.org Inc.
See our legalese page for copyright policies. Please also read our Privacy Policy.
Kuro5hin.org is powered by Free Software, including Apache, Perl, and Linux, The Scoop Engine that runs this site is freely available, under the terms of the GPL.
Need some help? Email help@kuro5hin.org.
My heart's the long stairs.

Powered by Scoop create account | help/FAQ | mission | links | search | IRC | YOU choose the stories!