Speculating here, as I can't find bsdgrep in the 10.8.1 source: http://opensource.apple.com/release/mac-os-x-1081/ bsdgrep is probably setting F_NOCACHE on either 1) all files or 2) files over a certain size threshold. It's doing this to avoid blowing the kernel buffer cache which could have a performance impact on all other apps. This is not a bug, but good default behavior, especially for large files. Unfortunately I don't see a flag in the man page to override the default. On Sep 11, 2012, at 12:50 PM, Jay Freeman <jay_reynolds_freeman@mac.com> wrote:
Sorry to let this one slide for a while; I thought I should explain what I have found out.
The two versions of MacOS are indeed running different versions of grep.
Under Mountain Lion:
grep -V grep (BSD grep) 2.5.1-FreeBSD
But for Snow Leopard:
/Volumes/"Snow Leopard"/usr/bin/grep -V grep (GNU grep) 2.5.1
When I run the Snow Leopard grep from mountain lion -- that is, when I substitute /Volumes/"Snow Leopard"/usr/bin/grep for grep in the tests described below, running when booted on my Mountain Lion partition -- I get back all of the performance that was lost using the Mountain Lion grep.
I will file a bug report on this one.
-- Jay Reynolds Freeman --------------------- Jay_Reynolds_Freeman@mac.com http://JayReynoldsFreeman.com (personal web site) On Aug 31, 2012, at 3:32 PM, Jay Freeman <jay_reynolds_freeman@mac.com> wrote:
I have encountered a large discrepancy in run times for grepping from an (evidently) paged-in file in Mountain Lion (10.8.1) versus Snow Leopard (10.6.8); Mountain Lion is about THIRTY TIMES slower:
I am running on a Mac Pro (Model MacPro3,1) with dual quad-core Xeons at 3.2 GHz and 16 GByte of RAM. I have a 4.5 GByte file, named "LotsOfLines", comprising 50 000 000 lines each about 100 characters long. The file is on an external firewire disc drive. The terminal command for this test is:
time grep CATS <full path to "LotsOfLines"> | grep DOGS
and there happens to be exactly one line in the file that contains both CATS and DOGS.
Suppose I have been doing other things with the Mac, or perhaps have not acccessed the file since boot time, so there is no chance that any of the file is paged into memory. Suppose also that my Mac is not very active -- Activity Monitor shows no substantial CPU use or disc activity. Then "time" reports, e.g:
Under Snow Leopard:
real 1m52.988s user 0m4.454s sys 0m2.952s
Under Mountain Lion:
real 2m49.208s user 2m45.289s sys 0m2.558s
The large discrepancy in user time is interesting but that is NOT the big problem; I note from the disk actvity pane in Activity Monitor that the real time is precisely consistent with the rate of reading from the firewire disc.
The problem occurs when I repeat the same command IMMEDIATELY after the first run terminated. What I get is this:
Under Snow Leopard:
real 0m5.591s user 0m3.472s sys 0m2.123s
Under Mountain Lion:
real 2m47.351s user 2m45.243s sys 0m2.058s
Thus the real time to execute this latter case is some THIRTY TIMES longer under Mountain Lion than under Snow Leopard. ?!?
I obtain similar results -- and a similar discrepancy -- when I put the file on one of the internal drives of my Mac Pro.
The high speed achieved under Snow Leopard appears to indicate that the OS has kept the file paged in (or at least most of it) between runs, which is what I would expect, and indeed, Activity Monitor shows little or no disc activity during the Snow Leopard run. An odd thing is that under Mountain Lion, Activity Monitor also shows little or no disc activity, which would suggest that the file is still paged in, but it certainly is not being accessed at what I would expect to be in-RAM speed.
The finder's report of times when the file was created, modified, and last opened, all are very old -- they reflect the time the file was actually written to disc, not the times of the tests just described.
As a double-check, I rebooted the Mac Pro (to Mountain Lion) and zapped the PRAM as I did so, and repeated the Mountain Lion tests -- and got the same result.
To add insult to injury, when I mount the external drive as a USB drive on my slow, small, 11-inch Macbook Air (MacbookAir3,1, dual core, 1.6 GHz, 4 GB RAM), also running Snow Leopard 10.6.8, the paged-in case runs in a real time of about 1 minute 45 seconds, still a lot faster than my Mac Pro running Mountain Lion, even though the Macbook Air doesn't have enough RAM to page in the entire file.
Does anyone have any idea of what is going on? I am not enough of a low-level system wizard to know if there is some setting somewhere that I can tweak to speed up Mountain Lion, or whether there is a feature I do not understand, or whether there is some kind of bug. Perhaps the page replacement algorithm for Mac OS changed between Snow Leopard and Mountain Lion in some way that adversely affects my situation?
FWIW, there is an issue with a real application underlying all this; I have a case in which it would be nice to keep a file paged in for repeated rapid access to parts of it. The data presented here are from a stripped-down test case based on the real application.
-- Jay Reynolds Freeman --------------------- Jay_Reynolds_Freeman@mac.com http://JayReynoldsFreeman.com (personal web site)
_______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app... This email sent to site_archiver@lists.apple.com