FYI - this is exactly the behavior we were seeing on our system (Univ
Pitt - 130 Xserves, 3 Xserve RAIDs, Xsan), which made us implement a
number of changes to the way things were laid out. With our system,
we completely isolated not only the Xsan MDCs, but also the clients
(we have two MDCs, and 4 clients), such that the client/server ratio
is about 30:1 for each Xsan client. We also gave the Xsan machines
their own internal network to communicate metadata. This helped
immensely.
One other thing you didn't mention is the grid engine setup - what
machine is your grid engine controller? We had to create a separate
machine for the grid engine controller, and isolate the grid engine
directory to its own storage pool on the Xsan (so that the users
directories and grid engine directories were kept separate). Grid
engine generates a lot of filesystem traffic in our experience.
Lastly, train the users not to use the NFS directory for their runs
on the compute nodes, if at all possible. We've actually created
example scripts for our users that enable them to bundle up the files
for their runs, copy them to the local compute node, and then execute
everything on the local machine (in /tmp), and then copy back
relevant results files. This also makes a big difference.
Hope that helps!
--
M. Michael Barmada, Ph.D.
Associate Professor of Human Genetics
Director, Human Genetics Department Computational Resources Division
Director, Center for Computational Genetics
Graduate School of Public Health, University of Pittsburgh
Crabtree A300
130 Desoto Street
Pittsburgh, PA 15261
=====================================================================
There are three kinds of people in this world:
those that can count, and those that can't...
The requirements said: Windows 2000 or better.
So I got a Macintosh.
To know the mighty works of God; to comprehend His wisdom and
majesty and power; to appreciate, in degree, the wonderful working
of His laws, surely all this must be a pleasing and acceptable mode
of worship to the Most High, to whom ignorance can not be more
grateful than knowledge.
~Copernicus
=====================================================================
--
M. Michael Barmada, Ph.D.
Associate Professor of Human Genetics
Director, Human Genetics Department Computational Resources Division
Director, Center for Computational Genetics
Graduate School of Public Health, University of Pittsburgh
Crabtree A300
130 Desoto Street
Pittsburgh, PA 15261
=====================================================================
There are three kinds of people in this world:
those that can count, and those that can't...
The requirements said: Windows 2000 or better.
So I got a Macintosh.
To know the mighty works of God; to comprehend His wisdom and
majesty and power; to appreciate, in degree, the wonderful working
of His laws, surely all this must be a pleasing and acceptable mode
of worship to the Most High, to whom ignorance can not be more
grateful than knowledge.
~Copernicus
=====================================================================
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Xsan-Users mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/xsan-users/email@hidden
