site_archiver@lists.apple.com Delivered-To: darwin-dev@lists.apple.com Hi, I made some quick prototype of the suggestions and I have now valloc buffer. It seems work well on most systems, but still we feel that the OS is caching on system that have large amount of free memory. For example we have 2 systems with 9 GB of RAM. And on those systems it looks like the OS still caches the files, because after a good 20 mins of playing files, we lost 5GB of free memory on those systems. Until the system has about 1.6GB of free memory left and then it seems to stabilize around that amount. On a machine with 3 GB of RAM the Free memory stabilize around 1.3GB of RAM. The memory usage of our app is about 1GBytes. When I look at the fs_usage output. it looks like both us and quicktime are opening the files with <CACHING OFF> (second one is quicktime F=13) 14:05:02.497 open F=12 (R_____) /video/clips_1080i_PAL/2500_i_1920x1080_AppleProRes.mov 0.000026 Our App 14:05:02.497 fcntl F=12 <CACHING OFF> 0.000003 Our App 14:05:02.497 write F=6 B=0x96 0.000008 Our App 14:05:02.497 fcntl F=12 <CMD=45> 0.000001 Our App 14:05:02.497 write F=6 B=0xbc 0.000007 Our App 14:05:02.497 getattrlist /video/clips_1080i_PAL/2500_i_1920x1080_AppleProRes.mov 0.000016 Our App 14:05:02.497 getattrlist /video/clips_1080i_PAL/2500_i_1920x1080_AppleProRes.mov 0.000021 Our App 14:05:02.497 getattrlist 0.000031 Our App 14:05:02.497 getattrlist /video/clips_1080i_PAL/2500_i_1920x1080_AppleProRes.mov 0.000021 Our App 14:05:02.497 getattrlist /video/clips_1080i_PAL/2500_i_1920x1080_AppleProRes.mov 0.000007 Our App 14:05:02.497 getattrlist /video/clips_1080i_PAL/2500_i_1920x1080_AppleProRes.mov 0.000008 Our App 14:05:02.497 getattrlist /video/clips_1080i_PAL/2500_i_1920x1080_AppleProRes.mov 0.000008 Our App 14:05:02.497 getattrlist /video/clips_1080i_PAL/2500_i_1920x1080_AppleProRes.mov 0.000007 Our App 14:05:02.497 getattrlist /video/clips_1080i_PAL/2500_i_1920x1080_AppleProRes.mov 0.000008 Our App 14:05:02.497 getattrlist /video/clips_1080i_PAL/2500_i_1920x1080_AppleProRes.mov 0.000007 Our App 14:05:02.497 getattrlist /video/clips_1080i_PAL/2500_i_1920x1080_AppleProRes.mov 0.000007 Our App 14:05:02.497 getattrlist /video/clips_1080i_PAL/2500_i_1920x1080_AppleProRes.mov 0.000008 Our App 14:05:02.497 open F=13 (R_____) /video/clips_1080i_PAL/2500_i_1920x1080_AppleProRes.mov 0.000011 Our App 14:05:02.497 getattrlist /video/clips_1080i_PAL/2500_i_1920x1080_AppleProRes.mov 0.000008 Our App 14:05:02.498 getattrlist /video/clips_1080i_PAL/2500_i_1920x1080_AppleProRes.mov 0.000009 Our App 14:05:02.498 getattrlist 0.000006 Our App 14:05:02.498 getattrlist /video/clips_1080i_PAL/2500_i_1920x1080_AppleProRes.mov 0.000008 Our App 14:05:02.498 fcntl F=13 <CACHING OFF> 0.000002 Our App 14:05:02.503 pread F=13 B=0x8 O=0x00000000 0.005277 W Our App 14:05:02.503 pread F=13 B=0x20 O=0x00000000 0.000304 W Our App 14:05:02.503 pread F=13 B=0x8 O=0x00000020 0.000260 W Our App 14:05:02.504 pread F=13 B=0x8 O=0x00000028 0.000197 W Our App 14:05:02.511 pread F=13 B=0x8 O=0x3b164f40 0.007767 W Our App 14:05:02.512 pread F=13 B=0x8 O=0x3b1680e1 0.000323 W Our App 14:05:02.512 pread F=13 B=0x8 O=0x3b1680e9 0.000202 W Our App 14:05:02.512 pread F=13 B=0x8 O=0x3b1680f5 0.000186 W Our App 14:05:02.512 pread F=13 B=0x8 O=0x3b16b520 0.000251 W Our App 14:05:02.514 getattrlist /video/clips_1080i_PAL/Softron_TestPattern_PRORESHD_i50.mov 0.000032 Our App 14:05:02.515 getattrlist /video/clips_1080i_PAL/Softron_TestPattern_PRORESHD_i50.mov 0.000020 Our App 14:05:02.516 getattrlist /video/clips_1080i_PAL/2500_i_1920x1080_AppleProRes.mov 0.000012 Our App 14:05:02.516 getattrlist /video/clips_1080i_PAL/2500_i_1920x1080_AppleProRes.mov 0.000013 Our App 14:05:02.520 pread F=13 B=0x34ca O=0x3b16b528 0.007802 W Our App 14:05:02.521 getattrlist /video/clips_1080i_PAL/2500_i_1920x1080_AppleProRes.mov 0.000078 Our App 14:05:02.521 getattrlist /video/clips_1080i_PAL/2500_i_1920x1080_AppleProRes.mov 0.000008 Our App 14:05:02.521 getattrlist /video/clips_1080i_PAL/2500_i_1920x1080_AppleProRes.mov 0.000031 Our App 14:05:02.522 getattrlist /video/clips_1080i_PAL/2500_i_1920x1080_AppleProRes.mov 0.000018 Our App 14:05:02.522 getattrlist /video/clips_1080i_PAL/2500_i_1920x1080_AppleProRes.mov 0.000009 Our App 14:05:02.522 getattrlist /video/clips_1080i_PAL/2500_i_1920x1080_AppleProRes.mov 0.000007 Our App 14:05:02.522 getattrlist /video/clips_1080i_PAL/2500_i_1920x1080_AppleProRes.mov 0.000009 Our App 14:05:02.522 getattrlist /video/clips_1080i_PAL/2500_i_1920x1080_AppleProRes.mov 0.000008 Our App 14:05:02.522 getattrlist /video/clips_1080i_PAL/2500_i_1920x1080_AppleProRes.mov 0.000032 Our App 14:05:02.522 getattrlist /video/clips_1080i_PAL/2500_i_1920x1080_AppleProRes.mov 0.000007 Our App 14:05:02.522 getattrlist /video/clips_1080i_PAL/2500_i_1920x1080_AppleProRes.mov 0.000010 Our App 14:05:02.522 getattrlist /video/clips_1080i_PAL/2500_i_1920x1080_AppleProRes.mov 0.000007 Our App 14:05:02.522 getattrlist /video/clips_1080i_PAL/2500_i_1920x1080_AppleProRes.mov 0.000009 Our App 14:05:02.522 getattrlist /video/clips_1080i_PAL/2500_i_1920x1080_AppleProRes.mov 0.000007 Our App 14:05:02.522 getattrlist /video/clips_1080i_PAL/2500_i_1920x1080_AppleProRes.mov 0.000009 Our App 14:05:02.522 getattrlist /video/clips_1080i_PAL/2500_i_1920x1080_AppleProRes.mov 0.000010 Our App 14:05:02.522 getattrlist /video/clips_1080i_PAL/2500_i_1920x1080_AppleProRes.mov 0.000008 Our App 14:05:02.522 getattrlist /video/clips_1080i_PAL/2500_i_1920x1080_AppleProRes.mov 0.000007 Our App 14:05:02.522 getattrlist /video/clips_1080i_PAL/2500_i_1920x1080_AppleProRes.mov 0.000025 Our App 14:05:02.522 getattrlist /video/clips_1080i_PAL/2500_i_1920x1080_AppleProRes.mov 0.000007 Our App 14:05:02.523 getattrlist /video/clips_1080i_PAL/2500_i_1920x1080_AppleProRes.mov 0.000012 Our App 14:05:02.523 getattrlist 0.000010 Our App 14:05:02.523 getattrlist /video/clips_1080i_PAL 0.000008 Our App 14:05:02.523 getattrlist /video 0.000007 Our App 14:05:02.523 getattrlist /video 0.000006 Our App 14:05:02.523 getattrlist /video 0.000005 Our App 14:05:02.523 getattrlist /video/clips_1080i_PAL 0.000034 Our App 14:05:02.523 getattrlist /video/clips_1080i_PAL 0.000006 Our App 14:05:02.523 getattrlist /video/clips_1080i_PAL 0.000006 Our App 14:05:02.523 getattrlist /video/clips_1080i_PAL 0.000006 Our App 14:05:02.523 close F=13 0.000013 Our App 14:05:02.523 write F=6 B=0x7c 0.000026 Our App 14:05:02.523 write F=6 B=0x9e 0.000006 Our App 14:05:02.523 write F=6 B=0x8e 0.000018 Our App 14:05:02.523 write F=6 B=0x98 0.000006 Our App 14:05:02.523 write F=6 B=0x86 0.000007 Our App 14:05:02.523 write F=6 B=0x78 0.000005 Our App 14:05:02.523 write F=6 B=0x87 0.000006 Our App 14:05:02.524 write F=6 B=0x79 0.000006 Our App 14:05:02.524 write F=6 B=0x85 0.000017 Our App 14:05:02.524 write F=6 B=0x76 0.000006 Our App 14:05:02.537 pread F=12 B=0x2ee00 O=0x00000200 0.012757 W Our App 14:05:02.537 write F=6 B=0x7d 0.000023 Our App 14:05:02.548 pread F=12 B=0x2ee00 O=0x00fd8400 0.011340 W Our App 14:05:02.888 pread F=12 B=0x69aa00 O=0x0008cc00 0.093243 W Our App 14:05:02.895 write F=6 B=0x7e 0.000040 Our App 14:05:02.895 pread F=12 B=0x2ee00 O=0x01f98e00 0.000167 Our App * The size of read I'm doing is not always a multiple of 4096. I'm wondering if that could play a role? * Also is it possible to see with fs_usage when something is added to global file cache? regards, marc Quick File Prototype code that implements suggested idea: //------------------------------------------------------------------------------------------- // cNoCachedFile::Open //------------------------------------------------------------------------------------------- OS_Error cNoCachedFile::Open ( OS_Char * theFilePath ) { OS_Error anError = kOS_NoError; int aControlReturn = 0; fFileRef = open(theFilePath,O_RDONLY); if (fFileRef == -1) { anError = kOS_Error_CouldntOpenFile; goto bail; } aControlReturn = fcntl(fFileRef, F_NOCACHE, 1); if (aControlReturn < 0) { #if FW_LOG_PROGRESS OS_OPTIONAL_DEBUG_LOG(gLog_Progress_On,">>>> cNoCachedFile::Open Open movie: No cache flag gave error %d \n",errno); #endif } aControlReturn = fcntl(fFileRef, F_RDAHEAD, 0); if (aControlReturn < 0) { #if FW_LOG_PROGRESS OS_OPTIONAL_DEBUG_LOG(gLog_Progress_On,">>>> cNoCachedFile::Open Open movie: No Read a Head flag gave error %d \n",errno); #endif } bail: return anError; } // cNoCachedFile::Allocate //------------------------------------------------------------------------------------------- // cNoCachedFile::Read //------------------------------------------------------------------------------------------- OS_Error cNoCachedFile::Read ( OS_Pointer pStoragePtr, OS_StreamSize pFilePosition, OS_MemorySize pDataSize) { OS_Error anError = kOS_NoError; off_t aSeekOffset; static void* gBufferPtr = NULL; static OS_MemorySize gBufferPtrSize = 0; ssize_t aLength; FW_ASSERT(fFileRef != -1); FW_ASSERT(gNoCachedFileMutex != NULL); // We can only read one block at the time, we should do this better and actually use the timestamp. // And then schedule which read goes first by timestamp. now we rely on the Kernel Thread scheduler. gNoCachedFileMutex->Lock(); if ( (gBufferPtr == NULL) || (gBufferPtrSize < pDataSize)) { if (gBufferPtr) { free(gBufferPtr); } else if (gBufferPtrSize == 0) { gBufferPtrSize = 8 * 1024 * 4096; } if (gBufferPtrSize < pDataSize) { gBufferPtrSize = pDataSize % 4096; if (gBufferPtrSize > 0) { gBufferPtrSize = pDataSize - pDataSize % 4096 + 4096; } else { gBufferPtrSize = pDataSize; } } gBufferPtr = valloc(gBufferPtrSize); FW_ASSERT(gBufferPtr != NULL); #if FW_LOG_PROGRESS OS_OPTIONAL_DEBUG_LOG(gLog_Progress_On,">>>> cNoCachedFile::Read allocated %u read buffer: %d \n",gBufferPtr,gBufferPtrSize); #endif } aLength = pread(fFileRef, gBufferPtr, pDataSize,pFilePosition); memcpy(pStoragePtr, gBufferPtr, pDataSize); gNoCachedFileMutex->Unlock(); if (aLength < 0) { anError = errno; if (anError == kOS_NoError) { anError = kOS_MovieError_FileReadError; } } return anError; } //------------------------------------------------------------------------------------------- // cNoCachedFile::Close //------------------------------------------------------------------------------------------- void cNoCachedFile::Close ( ) { if (fFileRef != -1) { close(fFileRef); fFileRef = -1; } } On Feb 4, 2010, at 12:10 PM, mvo@sky4studios.be wrote:
Chris,
Thanks for the input, I will do some test tonight.
I know in my case this isn't, that's why I mentioned this:
-------------- Not sure important: my memory pointer that i pass into the read, is an offset in a section of a big memory blob allocated with malloc. --------------
Question: is it sufficient that the pointer address I pass is a multiple of 4096 (in case page size on that platform is 4096) or do I have allocate the memory in a special way. Because as far I recall Malloc creates and aligned buffer, but not a page aligned buffer.
thanks,
marc
Marc --
To ensure proper F_NOCACHE behavior, you should really ensure you pass page-aligned buffers to read/pread. The buffer cache may decide it has to go through the copy path if your buffer is not well-aligned and your DMA controller requires better alignment.
-- Chris
On Feb 3, 2010, at 8:53 PM, Marc Van Olmen wrote:
hi,
Just trying to trace an issue with my app:
Our app runs fine under 10.5.8 (just retested this)
The app plays a playlist of HD videos and about 20Mbytes/second data rate. It plays a few of those clips in loop. I'm using the standard unix open function to read from these files.
fFileRef = open(fFilePath,O_RDONLY);
if (fFileRef == -1) { anError = kOS_Error_CouldntOpenFile; goto bail; } BAIL_OSERROR(fcntl(fFileRef, F_NOCACHE, 1));
(Also QuickTime has these files open, but I don't use Quicktime to read) but maybe this is a cause.
There is only one file open at the time. Also my app doesn't allocate much memory once the video is playing. So the Application memory is constant for most of the time.
Under 10.6.2 I see something strange happening.Under Activity Memory: the System Memory->Free is decreasing after playing above clips for like 20 min. Under System 10.5.8 this stays constant.
My system has 3GB of ram. Under 10.5.8 this stays around 1.5GB free available. Under 10.6.2 this goes to 200Mbytes after 20 min.
After a while I notice the "Swap used" starts to grow a little under 10.6.2 and this causes sometimes frame drops in my video playback app. Under 10.5.8 this doesn't happen.
When I go to the Terminal (10.6.2) and I use the 'purge' command Activity Memory: the System Memory->Free goes back 1.2 GBytes Free memory.
So this makes me assume that disk cache is being filled by my reading.
I also tested this on a system with 6GB and above symptoms happening also.
I also tried doing this:
BAIL_OSERROR(fcntl(fFileRef, F_RDAHEAD, 0));
no difference. Same behavior under 10.6.2
I tried: BAIL_OSERROR(fcntl(fFileRef, F_GLOBAL_NOCACHE, 1));
no difference. Same behavior under 10.6.2
Not sure important: my memory pointer that i pass into the read, is an offset in a section of a big memory blob allocated with malloc.
* Any suggestion on how to get rid of this problem? I * Did something changed under 10.6 regarding F_NOCACHE? * Any suggestion on how trace this issue better? Is there a way I can trace when a read data is being added to the disk cache for example? I didn't see anything in Instruments->Disk Activity. Maybe I missed something.
I can file a radar in case needed but just double checking if I'm overlooking something simple here.
cheers,
marc
_______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/sarcone%40apple.com
This email sent to sarcone@apple.com
------------------ 6 Infinite Loop M/S 306-2MS Cupertino CA 95014 phone: (408) 974-4033 fax: (408) 862-7577 email: sarcone@apple.com
------------------ 6 Infinite Loop M/S 306-2MS Cupertino CA 95014 phone: (408) 974-4033 fax: (408) 862-7577 email: sarcone@apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.appl... This email sent to site_archiver@lists.apple.com