Kuro5hin.org: technology and culture, from the trenches
create account | help/FAQ | contact | links | search | IRC | site news
[ Everything | Diaries | Technology | Science | Culture | Politics | Media | News | Internet | Op-Ed | Fiction | Meta | MLP ]
We need your support: buy an ad | premium membership

Google Everflux

By kpaul in Technology
Wed Mar 12, 2003 at 03:43:49 PM EST
Tags: Internet (all tags)

In a previous story regarding the Google Dance, I briefly mentioned something called everflux. As promised (or implied), here is a closer look at the process wherein Google constantly scours the web looking for minty fresh content, changing their index accordingly. Basically, everflux refers to the constant change in Google's Search Engine Results Pages (SERPs).

Even if you don't run your own website, it's interesting to see how as the web grows, new ways to tame all the information are put into place. If you are a webmaster and run your own site, Cre8asite forums suggests, "don't go hacking your pages to bits on account of Google's everflux." In other words, it's not something to freak out about, but it's still something a well rounded webmaster should understand.

According to Marcia, one of the first discussions at Webmasterworld regarding the phenomena took place in July 2002. Later that year, Brett Tabke offered up this information on everflux, "Lastly, they could be working on the index, rolling indexes back, switching parts of the index, backing up parts of the index, rewriting some offending part of the index, deleting parts of an index - or a multitude of other actions or problems that only Google could know about."

GoogleGuy, a Google employee who posts incognito at WebmasterWorld, offers this explanation of freshbot:
As we do a full crawl of the web, we find most of the sites from our fresh crawl and put them in our regular index. My advice on our fresh crawl is to view it as a nice "bonus" on top of Google's deep index. Users can always search our full index, but sometimes we can serve up even fresher pages as an extra nicety.

What does this mean for the average webmaster? In the word of the great Hitchhiker's Guide, "Don't Panic." Just do the normal things you should do:

1. Create a great site.
2. Submit your site to Google on our "add url" form.
3. Get a link from the Open Directory Project or other directories (Yahoo, etc.).
4. Don't panic if your site takes a little while to show up in Google. Be patient, and start to look around the web--there's lots of great advice about improving your site for users and search engines.
Good advice. A good web makes for a good index, I imagine.

Everflux is another evolutionary step in the process of offering the most recent and relevant snapshot of the web to the public. Google is adding to their value as a search tool by giving their index some of the same qualities as what is being indexed. That is, the more fluid and adaptable an index of the web is, the more accurately it will be able to reflect the fluid and adaptable nature of the web.

If you watch your log files at all, you can begin to see traffic surges for certain search terms on certain days - everflux in action. For example, say you create a page on the web (or as the kids put it these days, make a blog entry) about a movie which is just coming out on DVD and freshbot crawls your site and makes note of it.

Because of its relevance in time (overly simplified: sort results by pagerank and date), your page climbs to the top of the SERPs for a few days. Eventually, though, the story slips off your homepage and is replaced by another story about another movie which is soon gobbled by freshbot. Meanwhile, the long-standing sites regarding that particular movie regain their dominant positions in the SERPs.

While some may term it the Google Everflux Cauldron, I like to think of it as just another way to make research on the web easier and more productive. In fact, taking a cue from GoogleGuy's mention of Douglas Adams (may he rest in peace), when it comes to Everflux I think the entry in the HHGTTG would read, "Mostly Harmless."


Voxel dot net
o Managed Hosting
o VoxCAST Content Delivery
o Raw Infrastructure


o Gesundheit. 25%
o Saddam is behind it. 1%
o Osama is behind it. 0%
o The Office of Total Information Awareness is behind it. 4%
o Isn't that the game all my geeky d&d friends talk about? 16%
o It still confuses me. 7%
o Yet another cool thing about Google. 26%
o Doesn't concern me. 18%

Votes: 71
Results | Other Polls

Related Links
o Yahoo
o Google
o Google Dance
o everflux
o minty fresh
o everflux [2]
o Cre8asite forums suggests
o Marcia
o first discussions
o informatio n on everflux
o GoogleGuy
o explanatio n of freshbot
o Google Everflux Cauldron
o Douglas Adams
o Also by kpaul

Display: Sort:
Google Everflux | 16 comments (10 topical, 6 editorial, 0 hidden)
no offense (1.42 / 14) (#4)
by postindustrialist on Wed Mar 12, 2003 at 09:30:55 AM EST

but fuck google..

it's a web monster.. it's probably is as bad for us now as AOL was a while ago..

use kartoo

ANYYTHING is better than google
oooh.. looks likes somebody has anger problems.
question everything.
this sig is only one hundred and fifty characters long and it's still not eno

A couple of years ago this guy said (4.00 / 3) (#5)
by Idioteque on Wed Mar 12, 2003 at 10:07:06 AM EST

but fuck alta vista/lycos/yahoo...

it's a web monster..

use google

I have seen too much; I haven't seen enough - Radiohead
[ Parent ]
gogole is worthless (1.16 / 6) (#8)
by postindustrialist on Wed Mar 12, 2003 at 11:57:47 AM EST

it pops up seven or either links to the same site, or even worse, it has seven or eight links to each page on the site, renderin its, literally, millions of responses utterly pointless to sift through and make sense of.
oooh.. looks likes somebody has anger problems.
question everything.
this sig is only one hundred and fifty characters long and it's still not eno
[ Parent ]
multiples (4.66 / 3) (#9)
by kpaul on Wed Mar 12, 2003 at 12:08:24 PM EST

The most I've seen (in my experience) is a listing for two pages on a single site.

However, there sometimes still is a problem of webmasters buying up a lot of domains and redirecting them to the same spammy site. This is a lot less of a problem than it used to be, though.

Getting good results also depends a lot on how you use your search terms - how you phrase things.

One thing I wish Google had was 'related terms' suggestions when you search for something. AllTheWeb had it for a while, but they now hide it on the bottom of their page which makes it less useful.

I do still use other search engines because all of them have their strengths and weaknesses. In general, though, I think Google is one of the strongest out there at this time.

Out of curiosity, what search engine do you primarily use now and why?

2014 Halloween Costumes
[ Parent ]
the one i use now (none / 0) (#15)
by postindustrialist on Sun Mar 16, 2003 at 03:32:54 PM EST

is the one i listed above. i agree, they all have their weaknesses and their strong points and as one of the comments above stated, they use flash (although there is a flashless version.. )

my favorite thing about kartoo is that although it does pop up only a limited number of pages per "map" the mapping shows results in degrees of likeness to each other and combines it with grade school level webbing to see similar search terms that relate between similar sites. you can easily reformulate your seach by simply clicking on the + or - next to each key word to either include it or make sure it doesn't come up in your seach results. it does the work for you. there's also a varing degree of size of the "dot" that each site has varying in degree of popularity. (if it's huge, EVERYONE goes there...). it doesn't list popular and corproate sites first, but tries to list them in relativity to the search terms. a simple scroll over the "dot" each site has reveals on the left hand of the screen a brief summary of the site. my roommate hates it. it can't use a map or do webbing worth shit but i found it highly visual and extremely intuitive.
oooh.. looks likes somebody has anger problems.
question everything.
this sig is only one hundred and fifty characters long and it's still not eno
[ Parent ]

Kartoo's interface (none / 0) (#12)
by AtADeadRun on Wed Mar 12, 2003 at 02:31:11 PM EST

isn't as flexible as Google's, and Flash is a pain in the tuckus, particularly when I'm at work, where the sysadmins have our boxes locked down and Flashless.

It's also a lot easier for an advanced user to get Google to puke up good results than Kartoo, since advanced searching is a lot better-developed and more usable on Google.

Pain heals. Glory is forever. Keep running.

We would not die in that man's company
That fears his fellowship to die with us.
[ Parent ]
they have a flash free version i've heard... (none / 0) (#16)
by postindustrialist on Sun Mar 16, 2003 at 03:38:08 PM EST

and i've never had a problem with using their flash.... beyond that, you say it's easy for advanced google users, but i bet its still twenty to thirty minutes weeding through google to find what you want..

mind you i naswered the question below first, and i REALLY am regreting it considering it sounds too much like an advert, but i hate google. i would suggest using anything over using them.even yahoo is all right compared to google's mess...

oooh.. looks likes somebody has anger problems.
question everything.
this sig is only one hundred and fifty characters long and it's still not eno
[ Parent ]

Works fine for me. (none / 0) (#14)
by bjlhct on Thu Mar 13, 2003 at 12:10:53 AM EST

What drank your vodka?

[kur0(or)5hin http://www.kuro5hin.org/intelligence] - drowning your sorrows in intellectualism
[ Parent ]
Huh, not much content (4.25 / 4) (#11)
by khallow on Wed Mar 12, 2003 at 01:21:10 PM EST

So Google does whatever it does, continuously now rather than once a month? The algorithm causes spikes on Google traffic to your site? And you needed seven paragraphs and the quote to say that?

Stating the obvious since 1969.

Probably... (none / 0) (#13)
by kpaul on Wed Mar 12, 2003 at 07:00:10 PM EST

...better sectioned as an MLP, as the seven grafs were needed to spread the groovy links out a bit.

I was kinda afraid to try to go too in depth about the subject.

A more interesting article might be on Google's recent forays into the content realm - e.g. the Pyra purchase and the just announced deal with Knight Ridder to serve textads on newspaper story pages.

2014 Halloween Costumes
[ Parent ]
Google Everflux | 16 comments (10 topical, 6 editorial, 0 hidden)
Display: Sort:


All trademarks and copyrights on this page are owned by their respective companies. The Rest 2000 - Present Kuro5hin.org Inc.
See our legalese page for copyright policies. Please also read our Privacy Policy.
Kuro5hin.org is powered by Free Software, including Apache, Perl, and Linux, The Scoop Engine that runs this site is freely available, under the terms of the GPL.
Need some help? Email help@kuro5hin.org.
My heart's the long stairs.

Powered by Scoop create account | help/FAQ | mission | links | search | IRC | YOU choose the stories!