1

For the longest time I've been interested in building a cluster of heterogeneous nodes in an attempt to have a home supercomputer since I am very interested in doing AI research.

However, the issue is even though I have a myriad of hardware, (2x dual quad rack mount servers, 8 285GTX Gpus, 6x PS3s 2x Hacked 360s (they can run linux) access to tonnes of common PCs as well as a few workstations) I have no large data set that needs to be crunched, or even any software that I can run distributed. I have messed with distributed code compiling but at best its made my kernel builds go from 10 minutes (at worst) to 30 seconds (and I think 20 of those seconds are just setup).

So where should I start? I have a decent understating of Obj-C/C/C++ so it shouldn't be too hard to write something, but what should I write?

3 Answers 3

1

If you want data to crunch, there's plenty out there:

As for "what should I build", the real question is, what interests you?

2
  • Although having a dataset helps, I'm still not sure what I should try and crunch with this system. See, I'd like data that would come to a conclusive answer so I can benchmark my system, however as I mentioned in the OP, I haven't found anything that takes more than a few seconds.
    – edude05
    May 18, 2009 at 16:49
  • Agreed, I was mostly responding to your earlier comment about not having any large datasets. If you want ideas about problems to solve, maybe try the previous KDD Cup challenges? (They're not strictly distributed problems, but usually have to do with analysing large datasets.) sigkdd.org/kddcup The canonical "hard" problems are factoring and primality testing; maybe running mprime or something? mersenne.org/various/freeware.htm Or maybe compute Pi? (if you want a known-answer problem...) defcon1.hopto.org/pi
    – Stobor
    May 19, 2009 at 6:27
0

Well I think it's best to determine which subset of the hardware you have available you'll be developing your application for. Software for the PS3 needs special attention and will require separate development from something built to run on typical linux servers.

You may also need to do some research on how you could develop an application for the 360; I'm not sure if it'd really give you what you're looking for to be honest.

Once you've decided on the subset of hardware you need to develop for it would be good to start with some basic development to ensure you can a foundation put together which enables communication. With a solid foundation you'll be able to expand your code to support a variety of distributed projects.

I hope I'm understanding your question correctly!

Cheers

1
  • I think your slightly off track. The idea is I want to write some software that will run on all these devices as a means of doing some research on what "works" in terms of distributed processing
    – edude05
    May 18, 2009 at 16:51
0

If you are really adventurous you could try porting either/both Sprite (Unix-y) or/and Plan 9 (just plain weird) to each of the flavours of hardware you have (Plan 9 already runs on PCs).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.