I'm working with expensive software to parse a 5-300Gb csv file and
write out each column as a separate binary file. I'm running 10.5.2.
I find that I cannot process my huge files without running 'purge' a
few times throughout processing to clear the cache and return occupied
memory. Otherwise, the cache takes over memory and heavy swapping
starts.
I understand that a 'no cache' option is available in the open call
but I don't have access to the source code of the software I'm working
with.
Is there a command line tool or a set of system calls I can use to
limit the size of the cache to a certain amount of memory?
Would setting kern.maxvnodes with syscntl help here?
