"6 day compile" (1 GHz G4) DONE in 7 hours (8 core Intel 3.0 GHz over SSH)
"6 day compile" (1 GHz G4) DONE in 7 hours (8 core Intel 3.0 GHz over SSH)
- Subject: "6 day compile" (1 GHz G4) DONE in 7 hours (8 core Intel 3.0 GHz over SSH)
- From: "Helmut Brandt" <email@hidden>
- Date: Thu, 14 Jun 2007 20:28:51 -0700
An update for you all. Thanks for the numerous suggestions on how to do
this compile.
It has now been done, thanks to the generosity of Kevin Vanwulpen for
offering me a temporary SSH login on his 8 core 3.0 GHz Intel monster!
I did the compile in a little over 7 hours, instead of 6 days. I could
have probably done it in more like 6-7 hours if I knew a bit more (well,
a lot more!) about Unix.
Kevin (and others) suggested using xcodebuild in the terminal to compile
the program. Since this could be done over SSH, Kevin and I decided to
go this route. While I got xcodebuild to run over SSH, I kept getting
weird appendages on the names of my object files. Since I am a relative
XCODE newbie, I decided to just use gcc commands over SSH to compile the
files rather than waste the resource that Kevin loaned me trying to get
it to run.
So, I did some experimentation, and found the best way to feed the 8
cores was this:
(CAUTION: UNIX IGNORANCE AHEAD! Go easy on me!)
At first, I gave batches of 20 or so files to compile to the 8 core
machine. This proved to be not optimal at all. After some
experimentation with different approaches, I finally found something
very fast, (but very gross) that worked (and, I am DONE!). Here was my
approach:
I started 8 Terminal shells on my machine, each with an SSH login to the
remote 8 core machine. I gave each shell a single file to compile, and
watched for the execution to complete, at which time I pasted in a new
gcc compile command for a different file. Using this method was
extremely fast. However, due to my Unix ignorance I did not know how to
load multiple commands to a shell which would execute *consecutively*
rather than *concurrently* (I found before that I could seriously bog
down the system if I loaded too many compile requests to the cores).
So, I just did it manually. In this manner, each core chewed on a single
compile, and then got (manually) another one from me when it was done.
I had written a python script to generate a file of the commands, so I
just copied from that text file and pasted to the shell (yeah, I know,
gross).
Sometimes the compiles were so fast that I could not keep all 8 shells
fed, but I estimate that I kept them fed at least 80% of the time. My
actual time was 9 hours, but since I estimate that some of the time the
cores were not fed (like, when I was eating lunch!), the real time was
more like 7 hours.
Anyway, if anyone can tell me how to feed a stream of commands to a
shell that will execute one after another (no forking), I would
appreciate it.
Thanks again to all that responded.
I also got one other offer from Daniel Dalquen offering remote use of
his Quad G5, which I still might take him up on to try some other
optimizations for speed! Thanks, Daniel! I'll be in touch.
Helmut.
--
Helmut Brandt
email@hidden
--
http://www.fastmail.fm - One of many happy users:
http://www.fastmail.fm/docs/quotes.html
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Xcode-users mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden