Kuro5hin.org: technology and culture, from the trenches
create account | help/FAQ | contact | links | search | IRC | site news
[ Everything | Diaries | Technology | Science | Culture | Politics | Media | News | Internet | Op-Ed | Fiction | Meta | MLP ]
We need your support: buy an ad | premium membership

[P]
Oh My God it's Slow!

By rusty in Site News
Fri Jan 10, 2003 at 01:26:13 PM EST
Tags: Kuro5hin.org (all tags)
Kuro5hin.org

DesiredUsername reminds me via email that K5 is slow like continental drift lately, and I haven't said anything about it publically. Of course I know it's slow, and yes I am working on it. Below is a short description of the basic confluence of problems, and what's being done to address them.


ADVERTISEMENT
Sponsor: rusty
This space intentionally left blank
...because it's waiting for your ad. So why are you still reading this? Come on, get going. Read the story, and then get an ad. Alright stop it. I'm not going to say anything else. Now you're just being silly. STOP LOOKING AT ME! I'm done!
comments (24)
active | buy ad
ADVERTISEMENT
First, as it's the beginning of the month (a period of time that's getting longer and longer, it seems) we're being Googled. That usually takes them at least a week now, and last month it generated 1,281,994 hits. Googlebot was our largest single user agent in December. I'm not kidding.

Without Googlebot's attention, the site is already running just about redline in terms of possible performance, especially at peak times of day like 9AM, noon, and 5PM eastern. With Google on top of that, the three daily peaks have basically merged into one continuous eight hour long peak every weekday. This is what you're seeing performance wise.

As for dealing with it, I've got two things underway. First, I bought a new database server, which is currently at the colo and awaiting an OS install, which should be done Friday night. Early next week I'll attempt to get mysql up on it, and switch over to that for the DB. It's got three times the memory of the old box (3 Gb), and the database will be on its own hard disk as well, both of which ought to help a lot. For the curious, the box is a dual 1.2 Ghz P3, and in addition to the memory mentioned above it will have a 36Gb SCSI drive (for the database) and an 18Gb SCSI (for the rest of the OS).

Second, when that's running to my satisfaction, I will be upgrading Scoop to the latest version. It's got some speedups in place from what we're running now, and I will also apply the alpha archiving patches, if they're deemed good enough to try. Archiving stuff will get a lot of extra data out of the main database and should make a world of difference for performance. The way the archive system works is to define a second database for archived material, which holds a slightly simpler version of the normal "stories" and "comments" tables. When a story is archived, it is copied into the archive DB and the original version removed from the main DB. The ability to add new comments is turned off, but all the usual formatting options are still available. This will let us just keep newer stuff in the main database, and keep it down to a size where the indexes and whatnot will stay manageable. The new Scoop's got a bunch of other cool stuff as well, most notably including theme support.

Update [2003-1-10 15:55:44 by rusty]: I have also altered robots.txt to tell Google not to index /comments/ and /users/. This should make it only index actual story pages and some miscellaneous things like special pages (which are fast anyway) and ought to cut down enormously on Googlebot traffic as soon as they re-fetch robots.txt.

So that's what's going on, and I beg your continued indulgence for a few more days while I attempt to fix it. Thanks!

Sponsors

Voxel dot net
o Managed Hosting
o VoxCAST Content Delivery
o Raw Infrastructure

Login

Poll
While waiting for pages to load, I:
o Practice holding my breath till I pass out 7%
o Carve large stone monoliths 22%
o Build ships in bottles 2%
o Run in circles chanting "Gooble, gobble, we accept you!" 8%
o Progress from unpolished stone tools through bronze and into the early stages of cast iron 39%
o Cry 18%

Votes: 448
Results | Other Polls

Related Links
o Scoop
o Google
o DesiredUse rname
o Also by rusty


Display: Sort:
Oh My God it's Slow! | 89 comments (89 topical, editorial, 0 hidden)
Question (4.20 / 5) (#1)
by coryking on Fri Jan 10, 2003 at 01:26:11 PM EST

Will you be including all the new javascript based comment moderation stuff? That would be very cool!

Well we are at it, will you also be incorporating that patch that sends a copy of all comments and stories to TIPS? You'll probably want to consider this at some point.

Yes and no, respectively. :-) (4.00 / 3) (#5)
by rusty on Fri Jan 10, 2003 at 01:59:03 PM EST



____
Not the real rusty
[ Parent ]
As long as you're putting in javascript (3.25 / 4) (#14)
by wiredog on Fri Jan 10, 2003 at 02:16:18 PM EST

you might as well put in the blink tags. Nested tables and "img" tags in comments would be equally useful.

The greatest contribution of the internet to society is that it makes it possible for anyone of any age to become a grumpy old fart.
Parent ]
Dude (5.00 / 3) (#20)
by coryking on Fri Jan 10, 2003 at 03:02:49 PM EST

K5 already uses javascript. And it's not for blinking crap, and flashy junk. If you can belive (and it *is* hard given the amount of javascript based crap out there), javascript sometimes *is* actually useful.

Case in point? What do you think dynamic comments use? This comment moderation stuff in the newer builds is really cool, and lets you rate a comment without reloading the entire page! Can't do that without javascript!

Sometimes, you really do need javascript to make a user friendly, easy to navigate website. It's a such a shame it gets abused the way it does.

[ Parent ]

Nooooooo (3.33 / 3) (#7)
by anon868 on Fri Jan 10, 2003 at 02:03:55 PM EST

Please, keep Kuro5hin clean! We don't need Javascript anything. What's next, support for the <img src=""> tag? I really hope you were joking.
Open a window. No, not that one! One made from actual glass, set in an acual wall, you dork.
[ Parent ]
We already have javascript (5.00 / 6) (#12)
by rusty on Fri Jan 10, 2003 at 02:11:56 PM EST

Dynamic comment mode uses javascript to open and close comments. The changes cory was talking about are to also use javascript in that mode to submit ratings without reloading the whole page. It's pretty slick. Of course if you don't want to use javascript, you don't have to -- the other comment modes still work the same as always.

____
Not the real rusty
[ Parent ]
javascript rules... (none / 0) (#23)
by /dev/trash on Fri Jan 10, 2003 at 03:16:25 PM EST

I have this bug filed for Mozilla:

<http://bugzilla.mozilla.org/show_bug.cgi?id=186064>

It appears to be fixed in the latest nightly though.

---
Updated 02/20/2004
New Site
[ Parent ]

Huh. (none / 0) (#28)
by Canthros on Fri Jan 10, 2003 at 03:36:54 PM EST

I've been using Mozilla for months around here with dynamic threading on, and have never had that problem.

--
It's now obvious you are either A) Gay or B) Female, or possibly both.
RyoCokey
[ Parent ]
yeah (none / 0) (#45)
by /dev/trash on Fri Jan 10, 2003 at 08:04:08 PM EST

A nightly build or one of the stable
versions?  It was only broke for me
on like two different builds
but seems okay now.

---
Updated 02/20/2004
New Site
[ Parent ]
Stable. Ish. (none / 0) (#47)
by Canthros on Fri Jan 10, 2003 at 08:59:55 PM EST

I have 1.1 here, I think, and 1.2.1 at work. I think, anyway. So I probably just missed it, if it's only been in the nightlies.

--
It's now obvious you are either A) Gay or B) Female, or possibly both.
RyoCokey
[ Parent ]
it was strange... (none / 0) (#59)
by /dev/trash on Sat Jan 11, 2003 at 12:23:54 PM EST

I think it was a fluke in someone's code that got changed between nightlies, which makes it next to impossible to track down.

---
Updated 02/20/2004
New Site
[ Parent ]
Suggestions + offers for comment rating (none / 0) (#68)
by gusnz on Mon Jan 13, 2003 at 05:39:52 AM EST

I'm using DHTML Threaded view and love it. However, I've got a couple of suggestions:
  1. When you click 2 links to download some comments, only the second is loaded. The script needs to have a stack-type arrangement where is queues a list of comments to retrieve via the IFRAME and insert into the document.
  2. An "expand all in this thread" option would be great and save my poor mouse.
However, unlike 99% of the other "it's teh borken rusty fix it plz" posts I'm actually in a position to do something about it. Click my shameless self-promotion sig or homepage link -- I do JS/DHTML design as a part time job and have written a very similar iframe loading script for my site. I'd be glad to help out with tweaking the client-side code if you decide to make a go of it -- email me if you're interested.

Cheers - Angus Turnbull.


[ JavaScript / DHTML menu, popup tooltip, scrollbar scripts... ]

[ Parent ]

The second bit (none / 0) (#69)
by rusty on Mon Jan 13, 2003 at 12:10:35 PM EST

The "expand thread" thing is in the update already. As for making it a stack, I don't know -- the javascript stuff is basically outside my realm of expertise. If you're interested in fixing it, grab a page and the javascript and see if you can make it go locally. If it's just a matter of the client-side javascript then you should be able to test it without installing all of Scoop and so forth, right?

____
Not the real rusty
[ Parent ]
javascript (2.00 / 2) (#18)
by eudas on Fri Jan 10, 2003 at 02:46:52 PM EST

what is this evil javascript thing of which you speak?

eudas
"Nothing is on fire, but the day is still young" -- Phil the Canuck
[ Parent ]

No! (5.00 / 1) (#22)
by aonifer on Fri Jan 10, 2003 at 03:12:07 PM EST

Use positive reinforcement on the hamsters.  Give them a raise.  Give them hamster porn and other hamster entertainment.  Give them little hamster cots.  Remember:  A happy hamster is a productive hamster.

[ Parent ]
slow like continental drift (4.83 / 18) (#2)
by wiredog on Fri Jan 10, 2003 at 01:28:39 PM EST

One speedup would, apparently, be to block google from indexing the site until you've got (at least some of) the fixes done. It's nice that new people can find us through google, but not at the cost of driving away the people who are already here!

Another speedup would be to stand over the hamsters powering Scoop with a whip. Whip them indices into shape! Mush! Mush! Onward! Faster! faster!

The greatest contribution of the internet to society is that it makes it possible for anyone of any age to become a grumpy old fart.

About Google (4.80 / 5) (#6)
by rusty on Fri Jan 10, 2003 at 02:00:04 PM EST

I just added lines to robots.txt to tell Google not to index comments and user. This should help quite a bit in cutting down the number of queries from them, as soon as it re-fetches robos.txt.

____
Not the real rusty
[ Parent ]
Flat files (none / 0) (#25)
by izogi on Fri Jan 10, 2003 at 03:24:15 PM EST

Perhaps that's enough, but have you also considered storing the stories separately in flat files? Whenever googlebot comes in you could have the web server simply return the file instead of going to all the trouble of interrogating the database.

Maybe place a note at the end of the file pointing out that it's a google-specific page. That way anyone looking at the cache will know they're not getting the whole thing.


- izogi


[ Parent ]
Better yet, archive them in Nested mode (5.00 / 1) (#46)
by pin0cchio on Fri Jan 10, 2003 at 08:58:57 PM EST

but have you also considered storing the stories separately in flat files?

I'd say make one static page for guest users and one static page for the common robots. Give guest users the default view settings, but give robots Nested. This way, the search engines will still be able to see the comments.


lj65
[ Parent ]
Maybe (none / 0) (#54)
by izogi on Sat Jan 11, 2003 at 06:09:02 AM EST

The reason I was thinking of just archiving the stories is that it should be quite easy to simply dump out a file as soon as the story's posted. If it's based on comments, the file has to be re-written at various intervals. (I haven't thought about how much extra stress this might be on the system.. possibly not much.) There's also the issue of comments heavily inflating the size of a story, which is a potential issue if bandwidth's a problem... I'm not sure if it is.

I suppose the down side is that comments wouldn't be indexed.


- izogi


[ Parent ]
Only have to send static information once (none / 0) (#71)
by pin0cchio on Tue Jan 14, 2003 at 11:17:26 AM EST

There's also the issue of comments heavily inflating the size of a story, which is a potential issue if bandwidth's a problem... I'm not sure if it is.

It probably isn't. If you send the Nested static file to robots, you only have to send the top, right side, and bottom of the page once per story. And given that there exist a limited number of stories on Kuro5hin...


lj65
[ Parent ]
um, no... (5.00 / 2) (#8)
by pb on Fri Jan 10, 2003 at 02:05:31 PM EST

I wouldn't advise that, seeing as how Google provides Kuro5hin's search capabilities!

...or will that be fixed in the upgrade too?  The three hamsters powering kuro5hin have gotten pretty tired lately, it seems.  However, they can type very quickly!
---
"See what the drooling, ravening, flesh-eating hordes^W^W^W^WKuro5hin.org readers have to say."
-- pwhysall
[ Parent ]

Yep (4.66 / 3) (#15)
by duxup on Fri Jan 10, 2003 at 02:17:50 PM EST

I'm with wiredog.  Block google from indexing for a bit.  No need to direct people to a site that's hozed anyway.

[ Parent ]
K5 vs. Google. (3.00 / 1) (#44)
by DrEvil on Fri Jan 10, 2003 at 08:03:59 PM EST

Another speedup would be to stand over the hamsters powering Scoop with a whip. Whip them indices into shape! Mush! Mush! Onward! Faster! faster!

Why not just sick the Hampster's on Google's pigeons?  That'll solve the problem.

[ Parent ]

It seems your in over your head (4.19 / 26) (#3)
by Jack Wagner on Fri Jan 10, 2003 at 01:47:17 PM EST

That's not too uncommon these days as the technology Zeitgeist has surpassed its early stages and matured into a difficult arena that requires a schooled and learned expert to troubleshoot and maintain TCP/IP based client-server environments.

At J. Wagner Consulting LLC we can do just that for you, and more. We are experts at bottleneck performance anlaysis with speed improvements averaging Olog(n) for most Fortune 500 companies, without the need for any additional and often times expensive hardware. No more need for your willy nilly trial and error attempts to fix a technical problem.

As a sign of good faith I'd like to offer up a couple of suggestions for you at no charge and of course when you sign on as a client of J. Wagner Consulting LLC we will come on-site and perform a detailed analysis of the entire K5 site, then offer you a plethora of high value paradigm shift performance fixes.

1) The Apache webserver needs to be recompiled with the SET_DOUBLEFAST_CACHE set to 1. This is an undocumented feature, by the way, one which we at J. Wagner Consulting LLC know and that our lame drunken ass-clown crack smoking competition doesn't.
2) In the computer BIOS you need to set the memory bus settings to activate plutionium core memory emulation. (It's recommended that you unplug your server before doing this) This feature is only available in modern computers and often times will singly handledly take you to the next level.

This is just the tip of the proverbial iceberg but yours at no charge, my good sir. Email me without delay at jwagner@usa.com for quotes and payment terms and I can be on the first flight Monday morning to your colo.

Looking forward to a long term and properous relationship.

Warmest regards,
--Jack

Wagner LLC Consulting - Getting it right the first time

Comments (4.68 / 16) (#10)
by PhysicsGenius on Fri Jan 10, 2003 at 02:06:57 PM EST

I don't have much to say about #1 since I don't know anything about Java.

#2 is kind of a bad idea. You should only activate the plutonium core memory emulation when your computer case has sufficient shielding to keep the increased radiation emulation from escaping into the atmosphere.

I would like to tender another plan, though. We don't seem to have any trouble sending information to the server, just getting it back. That would seem to indicate that the server is informationally (and therefore entropically) "downhill" from us. Perhaps the colo has the temperature in the server room turned too low and machine isn't high-powered enough to shove data up against the temperature gradient?

I would be happy to model this situation for you to test out my ideas, as I recently unpacked my Legos Mindstorms.

--
The human race likes to give itself airs. One good volcano can produce more greenhouse gases in a year than the human race has in its entire history. -- Ray Bradbury.
[ Parent ]

Free consulting is worth every cent (5.00 / 1) (#26)
by CaptainZapp on Fri Jan 10, 2003 at 03:26:46 PM EST

I would like to tender another plan, though. We don't seem to have any trouble sending information to the server, just getting it back.

Since this observation is spot on may I suggest one of my eagerly plotted architectural solutions for sites where retrieving information is troublesome, while sending it is as easy as obtaining a parking ticket in downtown Los Angeles:

The write-only database(TM)

We could also take the chance to change to a database engine, which doesn't only have industry strength transaction support, but also cures the world from bad breath.

There is absolutely no need to thank me...

[ Parent ]

no trade-mark for you (none / 0) (#40)
by speek on Fri Jan 10, 2003 at 07:30:46 PM EST

Sorry, but the write-only database has been done before.

--
al queda is kicking themsleves for not knowing about the levees
[ Parent ]

but does it use write-only memory? [nt] (none / 0) (#48)
by pb on Fri Jan 10, 2003 at 10:12:52 PM EST


---
"See what the drooling, ravening, flesh-eating hordes^W^W^W^WKuro5hin.org readers have to say."
-- pwhysall
[ Parent ]
but is it pure? (5.00 / 1) (#55)
by CaptainZapp on Sat Jan 11, 2003 at 06:16:05 AM EST

No, of course it's not, since the data eventually is replicated.

My concept is: Once written, never retrieved

Also, you don't have to scatter around a badly assorted collection of files on preferably multiple disks (like in Oracle) and you don't have to bother with creating devices (like in Sybase).

You just assign everything to /dev/null.

Just think of the incredible throughput you can achieve. Ah, Larry Ellison: Shiver in your boots.

You're right though, regarding the trasdemark issue. Sorry, while I'm off, registering GoneGoneGone(TM)

[ Parent ]

that's a far better trademark anyway (nt) (none / 0) (#58)
by speek on Sat Jan 11, 2003 at 09:38:37 AM EST


--
al queda is kicking themsleves for not knowing about the levees
[ Parent ]

Dear Kuro5hin Shareholders, (4.20 / 5) (#38)
by Dr Martino Cortez PhD on Fri Jan 10, 2003 at 07:24:17 PM EST

I can vouch for Prof. Wagners consulting firm. A while ago, I needed a top notch expert to bring our corporate extranet into XML compliance. Wagner, and his team of performance analysts recommended we go with AOL, which uses a store and forward method of IP token delivery. We were impressed by their competence as they installed our fiber optic sonnet network between us and the global TCP/IP backbone. After the backbone was installed, we were able to rully gain XML as well as DHTML compliance.

More recently, we had to use him to install an 802.11b wireless network in our underground nuclear waste silo. This network was used so we could monitor the status of our valuable nuclear waste from the surface.

On an asside, is it our corporation, or has AOL been mighty slow in serving this page?

May your days be bright, and my the KURO stock right well into the blue sky.


--
Dr Martino Cortez PhD
CEO - Martin-Cortez Financial Corporation
Copyright 2001, 2002 and 2003, Martino Cortez and Associates. All rights reserved.
[ Parent ]

I second this (5.00 / 1) (#41)
by CaptainSuperBoy on Fri Jan 10, 2003 at 07:32:20 PM EST

I can also vouch for Herr Doctor Wagner, having read his excellent treatise on remote object invocation through intranet push-technology. Truly a great paper.

--
jimmysquid.com - I take pictures.
[ Parent ]
Well, yes and no. (none / 0) (#50)
by QuickFox on Sat Jan 11, 2003 at 01:55:10 AM EST

Truly a great paper.

It was good, but great? He did use some very clever algorithms for increasing system patience while waiting for the push, but here at QuickFox Solutions we get the system to wait almost ten times faster.

Give a man a fish and he eats for one day. Teach him how to fish, and though he'll eat for a lifetime, he'll call you a miser for not giving him your fi
[ Parent ]

Dude (3.50 / 8) (#4)
by TheOnlyCoolTim on Fri Jan 10, 2003 at 01:54:27 PM EST

Put in some hardcore robots.txt, I guess.

Tim
"We are trapped in the belly of this horrible machine, and the machine is bleeding to death."

+1, FP (4.25 / 4) (#9)
by fullcity on Fri Jan 10, 2003 at 02:06:19 PM EST

...for the ancient Zippy the Pinhead reference in the poll.

Or maybe you're referring directly to Freaks (1932).  (Can a full grown woman truly love a MIDGET?)

..  Now I think I just reached the state of HYPERTENSION that comes JUST BEFORE you see the TOTAL at the SAFEWAY CHECKOUT COUNTER!

There's one fly in the ointment that we've swept under the rug.

Freaks (5.00 / 1) (#16)
by rusty on Fri Jan 10, 2003 at 02:19:45 PM EST

The reference was to Freaks. "Gooble Gobble! One of us! One of us!"

Though Zippy has affected me in other ways. Whenever I see a vacuum commercial now, I'm unable to keep from repeating "Bagless upright cyclonic! Bagless upright cyclonic! Bagless upright cyclonic!" Similar phrases often provoke the same reaction.

____
Not the real rusty
[ Parent ]

Well (4.76 / 13) (#11)
by Joh3n on Fri Jan 10, 2003 at 02:08:09 PM EST

As a subscriber, I guess I'm asking myself the question:  would I rather have k5 respond to me now, or have google give me old k5 posts later?

Sure, it's great that k5 is heavily googled, but I would much rather have the site itself.

I vote to nuke the googling first, then patch/switch DB servers.  The users should get first crack, not a search engine.
---------------------------------
You can learn a lot about someone by popping in their un-rewound pr0n tape and seeing where exactly they came.
-terpy

to a faster k5! (4.00 / 3) (#13)
by eudas on Fri Jan 10, 2003 at 02:14:35 PM EST

i love the poll. it took approx 1:15 for it to accept my vote, though. ;)

Keep up the work, and don't let it turn into a snafu. Here's to a faster K5! *toasts*

Cheers!

eudas
"Nothing is on fire, but the day is still young" -- Phil the Canuck

drunk (nt) (none / 0) (#21)
by tps12 on Fri Jan 10, 2003 at 03:05:48 PM EST



[ Parent ]
Archive Database (4.80 / 5) (#17)
by Mister Proper on Fri Jan 10, 2003 at 02:30:33 PM EST

Will the archive database be hosted on the same machine as the main database? I was thinking that perhaps moving it to the old (i.e. current) database machine could help things.

About the Google problem, Google's Webmaster FAQ invites a reply when their bot is crawling your site too fast.

Google - gone (3.75 / 4) (#19)
by X-Nc on Fri Jan 10, 2003 at 02:49:23 PM EST

I hate to sound like a "me too" but I agree with the others that it would be good to block google for a while. At least till the new updates are done.

--
Aaahhhh!!!! My K5 subscription expired. Now I can't spell anymore.
I Blame Desired Username! (4.40 / 5) (#24)
by egg troll on Fri Jan 10, 2003 at 03:18:58 PM EST

Rusty, I've noticed the same problem and I've spent some time researching the problem.

I seem to notice a sluggishness on K5 only around the times that Desired Username - K5's very own Pons and Fleischmann - seems to be here. I'm certain that the slowness issues you're experiencing are no doubt related to one of his whacky physics experiments (I understand his last experiment involved vivisecting a clown, for some reason.)

I also know how reluctant you are to ban someone, but I think in this case its justified. Furthermore I think this situtation can be resolved without you having to take any action, Rusty. You see, I recently sent him a "prize" for winning the Steel Cage Troll Match. He should be receiving it in a day or two. Once he gets it, he'll be too preoccupied with looking for the antivenin to post to K5.

Then we should see a substantial speed increase!

He's a bondage fan, a gastronome, a sensualist
Unparalleled for sinister lasciviousness.

When venins attack (none / 0) (#53)
by carbon on Sat Jan 11, 2003 at 04:01:07 AM EST

Once he gets it, he'll be too preoccupied with looking for the antivenin to post to K5.

Wow... hiding a venin inside the trophy is a devious, evil, and altogether effective maneuver. Bravo!

Last time I got attacked by a venin, it took me weeks to herd all those damned pistachio nuts, unlabelled floppy disks, chewing candies, and Latin professors out of my house, and I never did find out where it hid my liver..


Wasn't Dr. Claus the bad guy on Inspector Gadget? - dirvish
[ Parent ]
-1 (4.42 / 14) (#27)
by VoxLobster on Fri Jan 10, 2003 at 03:31:25 PM EST

too k5-centric.

VoxLobster
I was raised by a cup of coffee! -- Homsar

Question: (4.33 / 6) (#29)
by ti dave on Fri Jan 10, 2003 at 04:06:46 PM EST

When a story is archived...

How long will 'stories' stay in the non-archived, commentable state?

I'd do LotR by Stephen Donaldson, but I can't decide who Frodo would rape.~Badtz Maru

Currently (4.50 / 2) (#32)
by rusty on Fri Jan 10, 2003 at 05:24:20 PM EST

I'm planning to archive after one year. I think we can comfortably hold a year of data in the main DB for the forseeable future, and that should be plenty of time for discussion. :-)

____
Not the real rusty
[ Parent ]
but but.. (5.00 / 2) (#39)
by infinitera on Fri Jan 10, 2003 at 07:26:19 PM EST

I like finding old stories and commenting/rating. *whine*

[ Parent ]
me too (none / 0) (#63)
by turmeric on Sun Jan 12, 2003 at 02:07:58 PM EST

me too

[ Parent ]
Old stories (none / 0) (#73)
by Mitheral on Tue Jan 14, 2003 at 06:37:17 PM EST

Looks like I better post that http://www.kuro5hin.org/story/2001/3/8/18386/36208 men's pill comment I've been working on :)

[ Parent ]
Bleh (none / 0) (#67)
by autopr0n on Mon Jan 13, 2003 at 02:16:07 AM EST

Why don't you just stop using MySQL and start using a real RDBMS?


[autopr0n] got pr0n?
autopr0n.com is a categorically searchable database of porn links, updated every day (or so). no popups!
[ Parent ]
Oh no.... (3.75 / 4) (#30)
by idiot boy on Fri Jan 10, 2003 at 04:18:42 PM EST

it will have a 36Gb SCSI drive (for the database) and an 18Gb SCSI (for the rest of the OS).
What? No RAID? You're a braver man than I. Or maybe just poorer! Maybe I should just get that sub. to help out.

--
Science is a way of trying not to fool yourself
Yes (5.00 / 1) (#34)
by CaptainSuperBoy on Fri Jan 10, 2003 at 06:00:39 PM EST

I agree, if the money's there RAID-5 would be a wise investment. Much faster and somewhat more reliable. If the bottleneck is data access, as opposed to CPU or network, RAID will speed things up.

--
jimmysquid.com - I take pictures.
[ Parent ]
Memory (4.33 / 3) (#36)
by rusty on Fri Jan 10, 2003 at 06:49:42 PM EST

I believe right now the bottleneck is memory. I don't think RAID would make a noticeable difference. If the DB was working right, there shouldn't be a huge amount of disk activity anyway.

____
Not the real rusty
[ Parent ]
Would you be opposed to it if... (2.00 / 2) (#49)
by Pinkerton Floyd on Sat Jan 11, 2003 at 01:35:21 AM EST

... people donated new hardware to the peoject?  I cannot right now, but if I could, I'd be happy to toss a drive your way.  If for example K5 readers could get you say 3 more of those, and a RAID controller, where should they send them?

Remember when you were young? You shone like the sun.
[ Parent ]

DB Performance and RAID (5.00 / 2) (#57)
by idiot boy on Sat Jan 11, 2003 at 07:28:19 AM EST

'scuse me if I'm teaching you to suck eggs :).

Mirroring (R1) is best for database applications in my experience. Although there is a small write penalty (since all transactions have to be written twice), there is a significant performance benefit to be gained on read operations since two ops. can happen simultaneously. The write penalty can be overcome and you can even gain a bonus if you use a hardware RAID controller since writes can be  cached in controller RAM, thereby allowing a write to commit even before it's actually written to disk (need a battery backed cache if you're gonna do this). You'll also get such lovelies as write reordering, allowing the controller to build up batches of writes and then flush them out to disk in the most efficient order possible (assuming SCSI).

Mirroring is preferable to R5 (which also has a read benefit) as the write penalty is significantly lower. An R5 write has to perform many more IOs than do write operations on a mirrored drive due to having to deal with parity.

I wasn't actually referring to performance in the comment. Rather I was suggesting that I wouldn't sleep all that easy at night if I didn't have some sort of data protection.

Mirroring is ideal again for this since the loss of one drive in the pair does not adversely affect performance (unlike R5 where the performance hit is very significant - especially during a rebuild operation). In other words, you get a chance to fix the box before anyone else even notices that anything is wrong.

You are right to seperate out the DB from the system drive(s) though. It's handy if you lose the system to be able to simply migrate the DBAppwhatever drive to another box.

Sorry again if I sound preachy but "Oi Loiks me storage oi does" (South West of UK vernacular :)).

I'll sort out that sub. at some point soon (when my credit cards stop sweating from Christmas :)).

--
Science is a way of trying not to fool yourself
[ Parent ]

RAID 5 considered harmful. (none / 0) (#61)
by rodgerd on Sun Jan 12, 2003 at 02:52:52 AM EST

RAID 5 is a dumb choice for a database system; while you get optimal use of disc, the write performance is lousy, ditto seek times.

A better bet if you've only got two spindles is a mirror (which will improve seek and read times), and if you've got four, a strip and mirror will give you excellent performance.

[ Parent ]

See my post on this subject but... (5.00 / 1) (#62)
by idiot boy on Sun Jan 12, 2003 at 05:56:00 AM EST

The situation gets even worse when you're in a rebuild situation. Regenerating data on the fly from remaining data and parity is hell.

In the medium to long term R5 is a dead duck because rebuild times are getting so long as hard disks get bigger and bigger. You rely absolutely on failure prediction so that you can do a block copy to an online spare as otherwise your performance and data protection go to shit (1-2 days rebuild - no protection anyone?).

--
Science is a way of trying not to fool yourself
[ Parent ]

Not that silvering mirrors is immune... (none / 0) (#80)
by rodgerd on Thu Jan 16, 2003 at 01:36:48 AM EST

It can take Some Time, as well.  However, one can at least have n-way mirror designs to compensate.

[ Parent ]
Nearly always guarenteed a block copy (none / 0) (#81)
by idiot boy on Fri Jan 17, 2003 at 03:06:49 AM EST

It's pretty rare that you're gonna lose both simultaneously. You're absolutely right though that there is a move toward n-way mirrors. It's all EMC's fault of course. They've been banging on about mirroring being the only way for donkey's years.

I don't think that they really knew that R5 would come under the pressure that it has but one way or another, the Symmetrix' approach seems to have more merit by the day

--
Science is a way of trying not to fool yourself
[ Parent ]

rusty please IP block google temporarily (3.14 / 7) (#31)
by jjayson on Fri Jan 10, 2003 at 05:16:02 PM EST

It would fix much of the problem immediately, as everybody else has pointed out. Some people are complaining about searching though. However, I don't really see a big problem:

(1) what use is it to search, when it takes 2 minutes to load the page you were looking for.

(2) K5 search works for material in the last month. Google already has the stories and comments before that from previous searching.

So, please please, won't you think of the CHILDRED?
_______
Smile =)
* bt krav magas kitten THE FUCK UP
<bt> Eat Kung Jew, bitch.

mod_throttle (4.75 / 12) (#33)
by ocelotbob on Fri Jan 10, 2003 at 05:27:39 PM EST

What about installing something like mod_throttle so that when google, or anyone else, tries mirroring, it's slowed down to a much more sane pace.

Why... in my day, the idea wasn't to have a comfortable sub[missive]...
--soylentdas

Since no one else is asking, (2.00 / 17) (#35)
by roam on Fri Jan 10, 2003 at 06:13:55 PM EST

will it be difficult to do some actual work after these many months of not doing any?

We're geniunely concered about your psychological well-being.

___
Are they like hamsters?
Specifically, can I tape up a chinchilla, slather him in axle grease, and shove him up my ass? - Patrick Bateman


Missing Poll Option (4.50 / 8) (#37)
by dirvish on Fri Jan 10, 2003 at 07:24:01 PM EST

Read slashdot!

Technical Certification Blog, Anti Spam Blog
That (5.00 / 2) (#42)
by CaptainSuperBoy on Fri Jan 10, 2003 at 07:35:16 PM EST

is never an option.

--
jimmysquid.com - I take pictures.
[ Parent ]
Indeed... (none / 0) (#43)
by Ranieri on Fri Jan 10, 2003 at 07:51:36 PM EST

But it's gotten a lot less fun since the continous K5 slowness forced me to reach the Karma cap...
--
Taste cold steel, feeble cannon restraint rope!
[ Parent ]
It's ironic (none / 0) (#70)
by Cro Magnon on Mon Jan 13, 2003 at 03:30:17 PM EST

Because I discovered K5 because Slashdot had slowed to a crawl. These days, /. is reasonably fast, and K5 is often unusable. :(
Information wants to be beer.
[ Parent ]
Google!? (4.91 / 12) (#51)
by duxup on Sat Jan 11, 2003 at 03:03:10 AM EST

On behalf of rusty I did some research.  
I believe I've found the problem.
Thanks to theboz for inspiration.


I'm sorry to hear the news. (3.00 / 1) (#52)
by MessiahWWKD on Sat Jan 11, 2003 at 03:37:20 AM EST

It's a shame to hear that Google is now using its power to congest the Internet, rather than enhance it.
Sent from my iPad
Big ups to Rusty for all his hard work (3.00 / 1) (#56)
by Insaa on Sat Jan 11, 2003 at 06:17:48 AM EST

Yo, brothers! Shout it out to rusty and maybe we can make this new box as slow as the old.

Thanks Rusty

One can only hope [NT] (none / 0) (#60)
by Hectibus on Sun Jan 12, 2003 at 12:33:48 AM EST



[ Parent ]
quit whining (3.33 / 6) (#64)
by turmeric on Sun Jan 12, 2003 at 02:14:54 PM EST

look, google was built by very smart very dedicated people. if you dont like it START YOUR OWN INTERNET. its OPEN SOURCE after all, all the protocls are easily available from the IETF archives. nobody is FORCING you to put k5 on the same internet as google. GO SOMEWHERE ELSE AND QUIT WHINING.

Yes, start a no-turmeric internet! /nt (5.00 / 1) (#65)
by Mister Proper on Sun Jan 12, 2003 at 02:51:57 PM EST



[ Parent ]
Whining vs. Explaining, and What it Says About You (none / 0) (#74)
by RadiantMatrix on Tue Jan 14, 2003 at 06:39:03 PM EST

look, google was built by very smart very dedicated people. if you dont like it START YOUR OWN INTERNET. its OPEN SOURCE after all, all the protocls are easily available from the IETF archives. nobody is FORCING you to put k5 on the same internet as google. GO SOMEWHERE ELSE AND QUIT WHINING.
I didn't see Rusty complaining a bit. What I saw was an explanation including verifiable data as to the reasons for the slow site performace. To recap, Rusty explained that the server is/was already at near peak, and that the additional 1.2 million hits generated over a week of Googlebot access was pushing the server over the top.

If Rusty was upset about being indexed, he wouldn't whine, he'd just update robots.txt to disallow Googlebot.

More to the point, the assumption that this clear, level-headed explanation was Rusty's whining says more about you, sir, than about Rusty. Mostly, it says that you are presumptuous and feel you must be always on the moral highground -- even if you must fabricate said highground to stand on. Besides, I must point out that this is still Rusty's site, and if he really wanted to whine, he's entitled; you, on the other hand, probably use the site for free.

It makes me wonder how you intend to justify your ranting...

--
$w="q\$x";for($w){s/q/\:/;s/\$/-/;s/x/\)\n/;}print($w)
I don't like Spam -- remove it from my address when mailing.
[ Parent ]

We've found it! (none / 0) (#76)
by rusty on Wed Jan 15, 2003 at 01:03:18 AM EST

Long considered a myth, researchers on the internet today announced the exciting discovery of the Last Remaining Person Who Tries to Respond Seriously to turmeric...

;-)

____
Not the real rusty
[ Parent ]

It's the plethora of free time :) (none / 0) (#78)
by RadiantMatrix on Wed Jan 15, 2003 at 08:47:33 PM EST

Once my mastery of Perl is a few notches higher, I'll spend said free time working on Scoop instead. I think I've figured out the long-awaited spell-checking feature, but I keep breaking other things when I implement it.... :/

--
$w="q\$x";for($w){s/q/\:/;s/\$/-/;s/x/\)\n/;}print($w)
I don't like Spam -- remove it from my address when mailing.
[ Parent ]
Errrr.... (none / 0) (#79)
by DesiredUsername on Wed Jan 15, 2003 at 10:00:37 PM EST

we already have a spellchecking feature.

Play 囲碁
[ Parent ]
Yes, yes, I had forgotten (none / 0) (#88)
by RadiantMatrix on Sat Jan 18, 2003 at 02:25:14 AM EST

I had forgotten that, since I don't subscribe. Well, I guess I must find some new nifty feature to add...

--
$w="q\$x";for($w){s/q/\:/;s/\$/-/;s/x/\)\n/;}print($w)
I don't like Spam -- remove it from my address when mailing.
[ Parent ]
Right on Rusty (3.00 / 1) (#66)
by vile on Sun Jan 12, 2003 at 10:44:32 PM EST

I complained about the speed (or the lack thereof, rather) in an article about what occurs at certain time periods/days (which was, of course, mod'd down.. and fwd'd to you in an e-mail).. most times, the site is fast as hell.. but was curious for a public response.. and now I have one.. ty. ;)

~
The money is in the treatment, not the cure.
Forward Google requests to a site mirror (none / 0) (#72)
by freality on Tue Jan 14, 2003 at 12:42:40 PM EST

Not sure on the details of this, but if you still have that old machine around, you could duplicate the k5 setup on it, and periodically (pick your period) copy over the database to that machine. Then forward all requests from googlebot.com (or any spider really) to it. You could further optimize the setup for read-only performance. That would take care of your main load hog. I think it's called "source forwarding". It's an IP routing topic.

Good Value! (5.00 / 1) (#75)
by holdfast on Tue Jan 14, 2003 at 08:18:15 PM EST

How much money are the complainers paying? I don't complain about what I get for free.
I use hotmail. It is slow and spam ridden. It is not costing me a penny. I don't complain. If I paid them a membership fee, things might be different. If everyone here paid serious money, there would be no problem with hardware.
You are doing good things with very little. Well done...

"Holy war is an oxymoron."
Lazarus Long
re: Google (none / 0) (#77)
by kpaul on Wed Jan 15, 2003 at 11:27:28 AM EST

I wish I could get to the point where I could say, 'Googlebot go away I don't really need the extra traffic!'

(small smile)

anyways, keep up the good work.


2014 Halloween Costumes

How're we doing? (none / 0) (#82)
by DesiredUsername on Fri Jan 17, 2003 at 09:30:59 AM EST

First, I bought a new database server, which is currently at the colo and awaiting an OS install, which should be done Friday night.

Presumably there were no problems there.

Early next week I'll attempt to get mysql up on it...

This might have been delayed due to illness. ?

...and switch over to that for the DB.

Either this didn't fix it or it hasn't happened. How's the schedule look?

Play 囲碁

Gah (5.00 / 1) (#83)
by rusty on Fri Jan 17, 2003 at 01:04:25 PM EST

Presumably there were no problems there.

You would presume wrong. The box arrived with a flaky SCSI controller, which is now being replaced. I tell you, I have had more hardware problems than is reasonable in the last several years.

____
Not the real rusty
[ Parent ]

Darn, but OK (none / 0) (#84)
by DesiredUsername on Fri Jan 17, 2003 at 01:21:53 PM EST

Just don't forget to keep us updated. Sheesh, you'd think a Mediii Mogul like yourself would know that obsessive reloading is the opiate of the masses. You gotta keep us opiated.

Play 囲碁
[ Parent ]
But... (5.00 / 1) (#85)
by rusty on Fri Jan 17, 2003 at 02:50:05 PM EST

I thought about posting another site news, but at what point does it become annoying? Maybe I should post a diary.

I think I'm going to go ahead and try to upgrade Scoop tonight. I was going to do it after the new DB is up, but since that's taking forever... I suppose I could post a general update and warning that there will be a lot of downtime later tonight.

____
Not the real rusty
[ Parent ]

Please let me know... (none / 0) (#86)
by blixco on Fri Jan 17, 2003 at 05:45:56 PM EST

....(via the usual locations) if you need any hardware. RAID controllers, SCSI controllers, RAM, disks, etc. I'd be happy to ship you what I can get, all for free.

Like, I have several 512MB ECC DIMMs. Two four channel RAID controllers, two 2 channel RAID controllers, some processors, etc.

I'm serious about this. Let me know if you need any parts, even just for spares.
-------------------------------------------
"No. Seriously. Mace the fuckers with prose." - Parent ]

Annoying site news (none / 0) (#87)
by DesiredUsername on Fri Jan 17, 2003 at 06:53:18 PM EST

You could do a diary. Or maybe a special account could do diaries. Or you could just make a special page and update it every 2 or 3 days and we'd know to check. Personally, it would take a long time for me to get annoyed at getting TOO MUCH information about this.

Play 囲碁
[ Parent ]
Delaying Googlebot. (none / 0) (#89)
by vectro on Sun Jan 19, 2003 at 08:01:14 PM EST

Not sure how much spare CPU you have on your router, but you could decrease the influence of Googlebot without any other consequences by setting your router to limit the amout of traffic passed to/from Googlebot's IP. Then Google would probably end up crawling the site over the course of the month, instead of all at once.

“The problem with that definition is just that it's bullshit.” -- localroger
Oh My God it's Slow! | 89 comments (89 topical, 0 editorial, 0 hidden)
Display: Sort:

kuro5hin.org

[XML]
All trademarks and copyrights on this page are owned by their respective companies. The Rest 2000 - Present Kuro5hin.org Inc.
See our legalese page for copyright policies. Please also read our Privacy Policy.
Kuro5hin.org is powered by Free Software, including Apache, Perl, and Linux, The Scoop Engine that runs this site is freely available, under the terms of the GPL.
Need some help? Email help@kuro5hin.org.
My heart's the long stairs.

Powered by Scoop create account | help/FAQ | mission | links | search | IRC | YOU choose the stories!