Kuro5hin.org: technology and culture, from the trenches
create account | help/FAQ | contact | links | search | IRC | site news
[ Everything | Diaries | Technology | Science | Culture | Politics | Media | News | Internet | Op-Ed | Fiction | Meta | MLP ]
We need your support: buy an ad | premium membership

[P]
I've (almost) got a beowulf cluster. What do I do now?

By in News
Thu Jun 08, 2000 at 11:49:55 AM EST
Tags: Help! (Ask Kuro5hin) (all tags)
Help! (Ask Kuro5hin)

The university where I work has just picked up a brand spanking new beowulf cluster (made from 40 Sun Microsystems E420R's no less!) - this inspired me to look around for a cheap beowulf solution that some of the students here in the computer science department could muck around with. It turns out the solution lay right under our noses!


We have around 50 JavaStations that are about to be retired because their performance is so poor. However, they can be made to run Linux, and run it well - with some cheap network switches and a PC as a front end, we'd have a nifty little cluster for the students to use. We just have to build the thing :)

Now, what would be something interesting to do with this set up? The performance isn't going to be stellar (it's primaray purpose will be teaching/research rather than pure number crunching)

The usual suggestions of RC-5/DES cracking, seti@home, etc. have popped up, but I'm curious to see what other people think. What would be cool to try out on a JavaStation cluster?

Sponsors

Voxel dot net
o Managed Hosting
o VoxCAST Content Delivery
o Raw Infrastructure

Login

Related Links
o E420R's
o JavaStatio ns
o Linux
o Also by


Display: Sort:
I've (almost) got a beowulf cluster. What do I do now? | 36 comments (36 topical, editorial, 0 hidden)
I would use Freebsd it has much bet... (3.00 / 2) (#5)
by Commienst on Thu Jun 08, 2000 at 08:52:37 AM EST

Commienst voted 1 on this story.

I would use Freebsd it has much better network performance than linux and better performance in general. Remember operating systems are tools not a damn religion, use the best tool for the job.

Re: I would use Freebsd it has much bet... uhhh (none / 0) (#9)
by tidepool on Thu Jun 08, 2000 at 12:02:55 PM EST

True. However, the entire point of this article was to find a job for the hardware / software combination to do. While networking may be important for the (future) use, it doesn't have to be.

For example: Each node could just have to submit a few numbers back to the main computer - after processing a large amount of data. True, this large amount of data needs to be transferred over the network - but how many times? If the data takes 20 hours to 'be processed', then the data needs to be sent every 20 hours. Stagger the inital ramp-up of the data, and you'll just have a constant flow of data going back and fourth, that does not require much network 'skillz'.

Also, how important is the networking in this case, if the OS / hardware combination doesn't 'put out' (heh) as well as the alternative?

In short, one should not dis linux because it has poor networking support (in relation to BSD's, etc). On the flip side, one should not praise linux just becasuse it's linux. Once the job is decided, that's where the OS choice should come into play

tidepool@suspicious.org
http://www.suspicious.org/~tidepool/

[ Parent ]

I gotta ask (2.50 / 2) (#10)
by Buck Satan on Thu Jun 08, 2000 at 12:07:44 PM EST

I saw a similar comment to this the other day on Slashdot... there was some story about a $2500 cluster someone made. It was pretty cool. Anyway...

They are running FreeBSD on it also and they made the comment about it being better than Linux because of performance and stability. Now, my questions - is this really true? Is there really this much of a difference?

I don't mean this to sound like a flame/troll etc. but I just wanna know. I have been running Linux for a year and a half now and have had tremendous luck with it. The only times the machine goes down are things like when my HD crashed, or I update the kernel/add hardware or something like that.

Where might I find some stats on this?



[ Parent ]
BSD/Linux performance (3.00 / 1) (#30)
by Dacta on Fri Jun 09, 2000 at 01:03:00 AM EST

I've looked and looked for figures to back this kind of thing up.

I've found very little evidence to support this "fact", and I've looked pretty hard.

A while back I posted something about this, with a few links here.

Basically, as far as I can see it is simply anti-Linux FUD to claim BSD has faster network performace.

Until I see some kind of benchmark comparing a recent Linux version with BSD I will continue to believe that the supposed HUGE performance advantage of BSD over Linux is simply an urban legend.

[ Parent ]

Re: BSD/Linux performance (none / 0) (#34)
by billyoblivion on Fri Jun 09, 2000 at 06:38:14 PM EST

I've looked and looked for figures to back this kind of thing up.
I've found very little evidence to support this "fact", and I've looked pretty hard.
A while back I posted something about this, with a few links here.
Basically, as far as I can see it is simply anti-Linux FUD to claim BSD has faster network performace.

A lot of the "BSD is better at networking" comes from the earliest days of the Linux/BSD jihads. Back in the day there was a problem with the copyright on the BSD networking code, and rather than wait for the copyright issues to be resolved the linux team just sort of rolled their own. This got the code out there and pretty much working, but it wasn't as tested or as debugged as the BSD stuff. They the copyright issues surrounding the BSD tcp/ip stuff were resolved, and BSD *did* have faster more robust networking code--for a while. Linux eventually caught up.

Now I seriously doubt there is a significant difference in terms of real world performance--if there is, it's in the drivers and the servers, not the tcp/ip implementation.

To look for any benchmarks, you are going to have to go *way* back. I think this whole thing was pretty much resolved by somthing like 1993/4.

At least that's how I remember it.

-- billy oblivion, living in the damaged world since 1992.
-- billy oblivion, living in the damaged worlds since 1992
[ Parent ]
Re: I would use Freebsd it has much bet... (3.00 / 1) (#12)
by Anonymous Hero on Thu Jun 08, 2000 at 12:18:03 PM EST

FreeBSD doesn't have a working SPARC port yet. I can hope, though...

[ Parent ]
Re: I would use Freebsd it has much bet... (1.00 / 1) (#22)
by mattc on Thu Jun 08, 2000 at 04:25:16 PM EST

This is false. FreeBSD does not have better network performance than any modern version of Linux. FreeBSD does not have "better performance in general" (whatever that means) than Linux either. I run both Linux and FreeBSD servers at work, so I know what I am saying is true. Trying to create artificial rivalries between free Operating Systems is stupid, IMO. Just use what you like and don't bash others!

[ Parent ]
Re: I would use Freebsd it has much bet... (none / 0) (#33)
by billyoblivion on Fri Jun 09, 2000 at 06:23:32 PM EST

Trying to create artificial rivalries between free Operating Systems is stupid, IMO. Just use what you like and don't bash others!
s/like/need/

An OS is a tool like any other, and you don't drive nails with screw drivers.

(if you think about it a bit, squint at it real hard and drink about 2 pints of cheap whiskey, this is close to being on topic)


-- billy oblivion, living in the damaged worlds since 1992
[ Parent ]
Do ECC cracking, not DES/RC-5 borin... (none / 0) (#2)
by Fish on Thu Jun 08, 2000 at 09:21:20 AM EST

Fish voted 0 on this story.

Do ECC cracking, not DES/RC-5 boring old stuff :-)

Oh, and now I know how "Bruce Perens" felt about "Bruce Perens."

-- the original fish

If you don't have a purpose, why bu... (none / 0) (#7)
by dgay on Thu Jun 08, 2000 at 09:29:51 AM EST

dgay voted -1 on this story.

If you don't have a purpose, why build something? Odd - don't try to create demand out of supply.

A case of 'use it or lose it' (none / 0) (#14)
by Pelorat on Thu Jun 08, 2000 at 12:27:34 PM EST

Sounds like they're trying to find a use for 50 workstations before said workstations get pitched. There's no reason *not* to use them... it's not like they're buying new equipment, setting it up, and saying 'Ok, what can we do with it?'

And they're sure to learn something by setting it up, never mind what they actually use it for. That's worth it right there.

[ Parent ]
Re: If you don't have a purpose, why bu... (4.00 / 3) (#16)
by blackwolf on Thu Jun 08, 2000 at 01:41:36 PM EST

From The Jargon File:

hack value n.
Often adduced as the reason or motivation for expending effort toward a seemingly useless goal, the point being that the accomplished goal is a hack. ... As Louis Armstrong once said when asked to explain jazz: "Man, if you gotta ask you'll never know."

"What's it going to be used for?" isn't important - it's a chance to build your very own beowulf cluster. Who needs a reason? That's what hacking's all about.

[ Parent ]

Which one is the duplicate?... (none / 0) (#6)
by Rasputin on Thu Jun 08, 2000 at 09:39:58 AM EST

Rasputin voted -1 on this story.

Which one is the duplicate?
Even if you win the rat race, you're still a rat.

I'm in the last days with my curren... (4.00 / 2) (#8)
by maynard on Thu Jun 08, 2000 at 10:17:05 AM EST

maynard voted 1 on this story.

I'm in the last days with my current employer (BBN) and am about to switch jobs to MIT. However, we currently have a 300+ Linux/Solaris/IRIX cluster which handles batch jobs for training our speech recognition engine.

Given this I'd like to point out that you're putting the cluster before the application, which is bassackwards (so to speak). The first question you should ask is "what kind of computational resources does your department need?" instead of "how can I use this cluster?" We can't help you with that simply because it's different from site to site.

Chances are you may find running a batch manager much easier at first than a traditional Beowulf cluster, simply because programs need to be tailored to use either their MPI or PVM message passing libraries. These are somewhat like a userspace threading library such as pthreads only instead of spawning threads within a process, an application calls one of these libraries to request CPU from another node in the cluster. A batch system is much easier to implement if you already have your application written and don't want to port to one of these libraries... with the caveat that you lose some fine grained load balancing.

You might also want to consider checking out the MOSIX kernel patches which allow for dynamic load balancing and a shared process table space across a cluster. It's quite amazing... I set a six node mosix cluster up for a login server at work and it's run like a charm ever since. The system will dynamically checkpoints a process and move it to the least active node without a user ever knowing... it forwards file descriptors and network sockets to the home node, has a cute memory system which moves dirty pages first and "leaks" inactive pages slowly so it doesn't saturate the network... all sorts of cools stuff.

I'm running out of time so I gotta go...

cheers!
--Maynard

Read The Proxies, a short crime thriller.

Re: I'm in the last days with my curren... (5.00 / 1) (#13)
by Pseudonymous Coward on Thu Jun 08, 2000 at 12:20:02 PM EST

Given this I'd like to point out that you're putting the cluster before the application, which is bassackwards (so to speak).

I think in an educational setting, where hithertofore undiscovered hardware has appeared for use, he's asking the right questions. Sure, in a nearly unlimited funding environment, like the one supporting your large configurations at BBN, it's the right thing to do to look at the application first.

I think, however, that he's asking the right question: "What's out there that I can use to build a cluster from these spare machines?" Getting that kind of information, and then choosing the solution that most fits the school's needs, is the correct thing to do. Tying scarce hardware to the configuration needs of a single application would be needlessly restrictive, especially with this stated purpose:

it's primaray purpose will be teaching/research rather than pure number crunching [sic]

Given that, asking around to find out what solutions exist is what I would consider to be the best possible approach.

[ Parent ]

what's with all the duplicate submi... (none / 0) (#3)
by Pelorat on Thu Jun 08, 2000 at 10:55:24 AM EST

Pelorat voted 0 on this story.

what's with all the duplicate submissions recently? yer preview button stuck or something?

Re: what's with all the duplicate submi... (none / 0) (#21)
by feline on Thu Jun 08, 2000 at 03:33:02 PM EST

when you're trying to write something insightful, I think it makes sense for clicking reload three times a minute to take a back seat.
------------------------------------------

'Hello sir, you don't look like someone who satisfies his wife.'
[ Parent ]

Interesting idea to use old hardwar... (none / 0) (#1)
by hattig on Thu Jun 08, 2000 at 11:27:33 AM EST

hattig voted 1 on this story.

Interesting idea to use old hardware. A 100MHz MicroSPARC CPU can't be that bad really - sounds quite usable anyway! Are they JavaStation 1's or JavaStation NCs?

Reading about them I can't believe that they were ever released, or that any university or person was stupid enough to actually buy them at all!

Go ask over in Chemistry, Physics, ... (2.50 / 2) (#4)
by warpeightbot on Thu Jun 08, 2000 at 11:49:54 AM EST

warpeightbot voted 1 on this story.

Go ask over in Chemistry, Physics, Atmospheric Sciences, and such departments. They're probably scrounging for cycles... just don't let the subatomic boys at it, they'll never let you have your computer back.

Confusion of Identities (none / 0) (#11)
by FFFish on Thu Jun 08, 2000 at 12:13:29 PM EST

I'd just like to point out that the newly-arrived "Fish" who posted this story isn't me. I'm "FFFish", which is short for "Five Fresh Fish", and have been for going on fourteen years.


Re: Confusion of Identities (none / 0) (#23)
by hattig on Thu Jun 08, 2000 at 04:43:24 PM EST

What is it with Fish around here?

First there was the original Fish, now there is a Fish., and now a FFFish.

Not to mention ParanoidFish...



[ Parent ]

Re: Confusion of Identities (none / 0) (#25)
by Anonymous Hero on Thu Jun 08, 2000 at 05:34:30 PM EST

Oi, I'd be surprised if the "original" Fish has been using the name since '86...

Signed FFFish, 'cause I've forgot my password. Bloody Opera not wanting to deal with K5 cookies...

[ Parent ]
Bloody Opera indeed (none / 0) (#27)
by rusty on Thu Jun 08, 2000 at 07:22:37 PM EST

Cookies (at least here) are known-broken with Opera. I wish someone could figure out conclusively if it's a K5 problem, or an Opera problem.

____
Not the real rusty
[ Parent ]
Re: Bloody Opera indeed (none / 0) (#31)
by jgaiser on Fri Jun 09, 2000 at 06:12:01 PM EST

Opera 4.0 Beta 4 works just fine. Jerry

[ Parent ]
Re: Bloody Opera indeed (none / 0) (#32)
by jgaiser on Fri Jun 09, 2000 at 06:19:24 PM EST

Or not!
Damn. Sorry about that.
It handles CSS1 nicely though.
Jerry

[ Parent ]
Re: Bloody Opera indeed (none / 0) (#35)
by rusty on Sat Jun 10, 2000 at 03:49:25 PM EST

This has been my experience too. It works for a little while (which seems to indicate to me that it's not my cookies that are broken), and then it starts to lose stuff, or forget there's a cookie there, or whatever. It seemed to happen when you start opening multiple panes in the browser, AFAICT. This behavior is what leads me to suspect it's Opera that's broken, and not K5. Well, that and the fact that the cookies do work with every other known browser. Still don't know for sure though.

____
Not the real rusty
[ Parent ]
Re: Confusion of Identities (none / 0) (#28)
by TheOtherFish on Thu Jun 08, 2000 at 08:53:50 PM EST

My apologies for the confusion! Who would have though there would be so many fish? I think I picked up that nick in 96 or 97, so I can't claim any historical precedent. I just like fish :) ... and cats! Is it possible to change nickname without making a new account? Seeing the number of fishes out there (aaargh! they're breeding!!) I should probably try and differentiate myself :)
Don't read this. It is irrelevant.
[ Parent ]
Re: Confusion of Identities (none / 0) (#29)
by fluffy grue on Fri Jun 09, 2000 at 12:45:36 AM EST

You can email Rusty and ask him real nice... it worked for me (my username on here was originally just 'grue', not 'fluffy grue')
--
"Is not a quine" is not a quine.
I have a master's degree in science!

[ Hug Your Trikuare ]
[ Parent ]

Put the idle cycles to good use... (none / 0) (#15)
by Nio Spartan on Thu Jun 08, 2000 at 01:23:46 PM EST

If the system is allowed to run overnight, the Casino-21 Climatology Project would be a great option; it's a distributed research effort that runs through a screensaver,like CETI@home.
What does courage mean? You can't program it. -Hugo Pratt
3D Modelling (none / 0) (#17)
by Neuromancer on Thu Jun 08, 2000 at 02:48:18 PM EST

I think that the students would like to use it for 3D modelling/animation. There is POVPVM. Could be a lot of fun. Or they could write games for it. I'm sure that if you have a chapter of ACM, that would be a popular one ;-)

Re: 3D Modelling (none / 0) (#24)
by dufke on Thu Jun 08, 2000 at 05:02:32 PM EST

I think that the students would like to use it for 3D modelling/animation.

But do these ancient boxen have enough FPU power to make it worthwhile? Somehow I think 10 Athlons would do more good than 50 JavaStations...
__
I am a Lurker. If you are reading this, I surfaced momentarily.
[ Parent ]

Re: 3D Modelling (none / 0) (#26)
by hattig on Thu Jun 08, 2000 at 06:26:53 PM EST

I reckon 2 Alphas could do better - the CPUs are only 100MHz and I doubt they are that powerful.

So 5 Athlons.

This is more for the "Don't want to waste the hardware" type of geek. They should put some shelves up and line them up on them in their room. There is 5GHz of power in them, there must be a use somewhere.

A teaching aid is a good idea - teaching how a Beowulf cluster works, and parallel algorithms. It is never going to be that powerful, but it ain't half bad, and if the processors are clock for clock sort of equal with a Pentium in terms of power, then having them work as a distributed renderfarm in an interesting idea - get each machine to either generate 1 frame each or each machine to do 1/50th of the image. 5GHz is good - especially for a student :-) Better than a 500MHz AMD K6-2 at any rate... and hopefully free. But do you have any interest in rendering?

Maybe you could try writing a distributed OS... that would be very interesting

You could sell them as ornaments.

[ Parent ]

Compile time distribution (none / 0) (#36)
by Neuromancer on Thu Jun 15, 2000 at 08:30:08 AM EST

Well, students always have a bitch of a time getting their labs done when the server is down/at peak usage. Perhaps they could use a redundant clustering system with these, and let students in for software compiles. With NFS access to the main server, it could be userful in those wee hours of the morning before finals.

[ Parent ]
teach a class on distributed programming (none / 0) (#18)
by johnmeacham on Thu Jun 08, 2000 at 03:16:39 PM EST

thats what we did here at caltech, it was a frosh single term class where people wrote for MPI (similar to PVM, a C interface to a cluster of computers) and did projects like a parallel mandlebrot set (easy) or a parallel solution to some heat transfer equations (harder.) it was a good time.

Well, once you've got the Beowulf cluster... (2.00 / 1) (#19)
by Denor on Thu Jun 08, 2000 at 03:22:57 PM EST

  Then you can pour hot grits down your pants!
  Oh, wait, I'm on K5 now. Seriously, though - I think teaching people how to use it would be the first priority. At least, that's the first thing I'd do with one - I'd want to know every in and out of the thing, how to tweak it, and what sort of problems it's useful for.
  Another good application would be (if the machine's powerful enogh) to show off. Run a webserver on it and get /.ed - watch it hold up as more powerful individual computers would not. Or run a mailserver on UUNet's network, and watch it ferry spam to and fro and not crash. Just having it around for hack value could be an interesting enough use. :)

-Denor


Do something useful... (none / 0) (#20)
by DesiredUsername on Thu Jun 08, 2000 at 03:32:26 PM EST

RC5 and seti@home would be pointless--they are already parallelized. To those programs, your "cluster" would just be 50 machines.

If this is a student project, go to one of your professors and ask for a good problem. I'm sure s/he's got a bunch of good ideas. For that matter, get out your "Algorithm Design" book and look up "parallel" or "distributed" processing.

Play 囲碁
I've (almost) got a beowulf cluster. What do I do now? | 36 comments (36 topical, 0 editorial, 0 hidden)
Display: Sort:

kuro5hin.org

[XML]
All trademarks and copyrights on this page are owned by their respective companies. The Rest 2000 - Present Kuro5hin.org Inc.
See our legalese page for copyright policies. Please also read our Privacy Policy.
Kuro5hin.org is powered by Free Software, including Apache, Perl, and Linux, The Scoop Engine that runs this site is freely available, under the terms of the GPL.
Need some help? Email help@kuro5hin.org.
My heart's the long stairs.

Powered by Scoop create account | help/FAQ | mission | links | search | IRC | YOU choose the stories!