Xgrid Project

Krevinek

Krevinek

Evil PPC Tweaker
Well, now that we have Xgrid... which does nothing, why not do something just because it is there? I remember helping someone attempt the factorial of 1 million, but failing because we didn't have enough time or computing power. Here is the trick I am attempting... I want to get an Xgrid setup of a few dozen machines from MacOSX.com users and see how quickly we can crack the factorial of 1 million! It would be an interesting attempt, and if it goes smoothly, we can attempt the factorial of larger numbers and see what type of speed record for amateur clusters for processing of large numbers we can set.

I am currently writing the code needed to attempt this along with a nice clean plug-in for the attempt.

Anyone interested in contributing a machine or two to the cluster? Right now I mostly need machines that can run 24/7 on broadband so I can run some speed tests using the Mandlebrot example that comes with Xgrid to see performance levels we can achieve over the internet.

To contribute to the project:

1) Download and install Xgrid
2) Open System Preferences and the new Xgrid panel
3) In the Agent tab, select 'Connect to specific host' and fill in the text field with: mzed.homeip.net
4) You can configure when your machine can accept tasks, although selecting 'Always' will definitely help out more.
5) Open the Agent Security tab
6) Set your password to: megamacgrid
7) Back in the Agent tab, start the agent, it will automagically join the cluster.

This password is simply to ensure you want to join the cluster, and don't worry about security over the network, since SSL is used for communications. You can select the 'Xgrid' screen saver if you wish, which will devote all your resources to Xgrid while you are asleep.
 
I suppose I could turn sleep off for my iMac for a while... would you be able to run stuff while it's got a screen saver going?

I don't know how much power you can squeeze out of a 400 Mhz G3... :)
 
Yeah, there is actually a screensaver for Xgrid that allows you to do just that... and don't be afraid of the computing power, so far I am running a 400Mhz G4 8600 and my Lombard 333 in a cluster fairly successfully... a 400Mhz G3 would get similar performance as my 8600 since the factorial code isn't going to be optimized for altivec. I am going to post up instructions in the first post soon so people can setup Xgrid to help this project.

And don't worry about the security risk, Xgrid cluster apps are run as 'nobody', which means they have no access on your system, only read-access to the files needed to do the computation.
 
I'll give it a try. 1.4GHz G4.

XGrid looks cool but other than this I wouldn't have any use for it.

The thing is, it says it uses Rendezvous, but does it also let you add clusters manually?

Also, how does it split up the work load? It'd have to give us a large chunk to do and then collect the results like Seti@home or something because if it worked like a traditional cluster you'd need more bandwidth to get good performance.
 
Captain Code said:
I'll give it a try. 1.4GHz G4.

XGrid looks cool but other than this I wouldn't have any use for it.

The thing is, it says it uses Rendezvous, but does it also let you add clusters manually?

Also, how does it split up the work load? It'd have to give us a large chunk to do and then collect the results like Seti@home or something because if it worked like a traditional cluster you'd need more bandwidth to get good performance.
Click to expand...

Well, right now I have my GF contributing the use of her iBook currently and I was running a Mandlebrot test using my G4 8600, my Lombard, and her iBook, all done over broadband. We were averaging about 1.2Ghz for 60% of the time the Mandlebrot task was running, not too shabby for 3 older machines.

The task I am designing will split the workload into about 100 tasks, then do 25, then 5, then compute those 5 on a single task on the cluster. All that is being sent back and forth is text, and a plug-in people would have to install for this to work. Which means this is a sort of task that is actually pretty parallel already, and can VERY easily be split across the internet using Xgrid.

I think this would be a good test and give Apple some interesting feedback in the end... I am also looking at ways to do a minor re-write of OpenShiva to allow it to utilize Xgrid to do MPEG-4 conversions (at least for home use for myself, but think about what we could do with a few internet-friendly Xgrid plug-ins and a massive amateur cluster?)
 
It sounds like it'd work pretty well then. I have a 9500 on 10.1.5 as well that is just a mail server. Will XGrid run on it? I haven't tried yet, but if I could set it to be absolutly lowest priority then I'd install it on there as well.

I managed to crash XGrid BLAST the first time I ran it. Tried starting a local service and it said permission denied. I tried connecting to a localhost service cause I thought maybe it was already running from running XGrid before and it crashed.
 
XGrid needs 10.2.8 or later... the agent (the app that actually turns your system into a node) doesn't need any apps to run... you configure and start it from your System Preferences. I am going to re-tweak the controller node (my 8600 G4) to use a seperate password which will be given to those who participate in the project so you can have access to the cluster created and mess around with it while I am writing and testing the factorial plug-in.

I use my 8600 as a mail server as well, and it is configured to always accept tasks... I haven't noticed anything unusual since doing this with my mail services.
 
1. Very cool idea. Good luck. Sorry, no machines I can release for this project.

2. When you're done with this, use your talent to write some plug-ins to let me use After Effects network rendering easier, mpeg-2 encoding via Compressor, Maya rendering, etc.!!!! :)

3. Does anyone else remember reading that IBM press release from maybe half a year ago? It talked about how they were going to bank everything on a similar technology to Xgrid and develop a massive infrastructure of parallel computing. Their business model was then going to switch to providing "client" machines that tapped into this system. Basically Xgrid on steroids. Anyone have a link for that?

4. Can't wait to see where all of this leads. Great concept with so many possibilities.
 
Alrighty then, tell me when it's ready and give me a download link and I'll stick it in.
 
Hang on am minute!

How do we know this guy is for real?

For all we know, he could be passively enlisting you guys into a digital bank job!

He could be pooling your computer resources to crack some 40bit encryption on some transatlantic wire transfers.

Sorry about that. I've just been watching the X-Files box set back to back since christmas day. I'm feeling a little paranoid...
 
Ah, so how'd you get 10.2.8 on your 8500? I'd like to get my 9500 up to speed as much as I can but it kept KPing trying to boot off the 10.2 disk.
 
Captain Code said:
Ah, so how'd you get 10.2.8 on your 8500? I'd like to get my 9500 up to speed as much as I can but it kept KPing trying to boot off the 10.2 disk.
Click to expand...


Maybe the G4 400 upgrade is enough to get it to work?
 
I'd like to join this, but my prefs pane gives up an err=1 window when it opens and again when I click start agent. I reinstalled it, but no go. App runs fine and does local demos no problem.

Also, trying to join from app gives me a authentication failed message.

Any one else seeing this? :(

PB G4 1.25 Al Book, 512MB ram

mike
 
I'm in. For now at least. We'll see if the network admins shut me down, as they shut down SETI@home…

Oh, yeah.

G5 1.6 GHz on a fat-@ss pipe.
XGrid set to "only when idle", though. Sorry, need my cycles when I'm actually using my machine.
 
Captain Code said:
Maybe the G4 400 upgrade is enough to get it to work?
Click to expand...

Yup, Ryan of XPostFacto couldn't get 10.2 or later running on non-G3/4 upgraded machines.

For those worried about the project, the plug-in and binary I will be distributing for the project will also have the source for it, for those who can read a little C... mostly the app will be code to multiply two big strings together, nothing more.
 
Okay, I have a design for the command-line tool that will be running on people's machines, and will write it up over the next 2 hours or so. I will post up a tarball for people who wish to peruse the code as well as people to help audit the code. I may or may not make an altivec-enabled version of the code (8 digits per loop versus 1... tasty) by then, which would allow those with a G4/G5 system to help out with to crunch many more pieces. I also will be scouring the plug-in API so see if I can submit jobs in rounds (seems very possible based off the Mandlebrot task), which would help split more of the task (multiplying very large numbers, even if it is only 2, using the method I am working with, takes a lot of processing power).

So far 5 machines have connected to the grid for a period of time:
My control node, up 24/7
My laptop, up when I am
My gf's iBook, up when she is at home
A 1.8Ghz G5 system, from... someone.
An 800Mhz iBook, from... someone.

I will configure the grid so that I can allow others to connect to it and tinker with the demos here shortly, probably the same time I submit the code for auditing.
 
XGrid seems like it's the basis for the distributed build in XCode. Seems like they just took what they started with XCode and made it capable of using plugins for custom applications.
 
I'm all set up in System Prefs but it doesn't seem to be doing anything because the CPU usage is all going to Seti@home.

I also tried to connect via XGrid the program and it said authentication failed. Is it offline now or something?
 
You must log in or register to reply here.
Back
Top