Is there a max number of jobs the controller can handle?
Is there a good way to submit 15000 jobs? How about more than that?
I also had problem of this type before, then I realised that using
multiple tasks within one job was much better handled by the
controller rather than a lot of jobs with one task each.
This indeed also makes a lot of sense if your different jobs/tasks
are indeed the same program run with different input. My
understanding is that when you distribute your work in a lot of jobs
with one task each, you have a massive overhead because a lot of the
information is duplicated (transferred and stored independently for
each job), while when having multiple tasks in a single job the data/
executable which is common to all the tasks have to be transferred
only once and stored only once on the controller.
If you look at the manual for xgrid, towards the end there is the
description of how to submit through the command line a multiple
tasks' job. One 'easy' way to do that is to submit a single task jobs
using xrgrid command line, then get the corresponding xml description
through xgrid (-job specification) and then edit this file to add all
the tasks you want.
If you need to perform this kind of stuff frequently, it is also
fairly easy to generate the XML from any language which has a bit of
XML support (python, java, C++ or C...). The XML content is clearly
described in the xgrid man page.
Where do you guys learn about this stuff?
If I restart the controller, or the server, will I still be able to
retrieve my jobs out of the database?
Are my questions even common?
Thanks for the help, sorry for the rant.
Brett Grant
Hope this helps.
Serge.
*******************************************************************
Dr. Serge COHEN
GPG Key ID: 9CBB58FB
N.K.I.
Department of Molecular Carcinogenesis
Plesmanlaan 121
1066 CX Amsterdam; NL
E-Mail: email@hidden
Tel : +31 20 512 2053
*******************************************************************
iD8DBQFGJLnG5EPeG5y7WPsRApCpAKCyiHoFFQXHAkePFOxiHdQtuq7hxACfYKeP
QQIOSYdpHFlh8ozULHqgkzM=
=SCka
-----END PGP SIGNATURE-----
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Xgrid-users mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden
