November 2005 - Darwin-dev - lists.apple.com

Re: no Makefile in IOKitUser-277.6
by Jeff Nathan 07 Nov '05

07 Nov '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Certainly, http://www.opendarwin.org/en/articles/building_xnu/ - -Jeff On Nov 7, 2005, at 1:33 PM, Kevin Van Vechten wrote: Thanks, Kevin On Nov 7, 2005, at 6:52 AM, Josh Wisenbaker wrote: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I just noticed that IOKitUser-277.6 doesn't have a Makefile, making it difficult to follow the building XNU instructions on www.opendarwin.org. I'm resolving a problem with my ADC account and can't presently file a bug report. Just a heads up, in case anyone who can correct this is reading the list. Josh www.afp548.com - -- http://cerberus.sourcefire.com/~jeff (DSA key id 6923D3FD) Part-time software mechanic, full-time daredevil! -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (Darwin) iD8DBQFDb55cEqr8+Gkj0/0RAlJ0AKC1JV5r/Z2q1opq+Ep1drkFI+S9YwCfS/m6 WIzDux8VK2BoDxpioFxOt/M= =VVn7 -----END PGP SIGNATURE----- _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… I'll make an attempt to find existing build instructions and rewrite them to be in terms of DarwinBuild. It would be very helpful if everyone could reply with URLs of build instructions on either Apple's or OpenDarwin's site. I would just use darwinbuild now. All the old build instructions out there on the internet are out of date now. This email sent to site_archiver(a)lists.apple.com

1 0

Re: no Makefile in IOKitUser-277.6
by Kevin Van Vechten 07 Nov '05

07 Nov '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Thanks, Kevin On Nov 7, 2005, at 6:52 AM, Josh Wisenbaker wrote: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I just noticed that IOKitUser-277.6 doesn't have a Makefile, making it difficult to follow the building XNU instructions on www.opendarwin.org. I'm resolving a problem with my ADC account and can't presently file a bug report. Just a heads up, in case anyone who can correct this is reading the list. Josh www.afp548.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… I'll make an attempt to find existing build instructions and rewrite them to be in terms of DarwinBuild. It would be very helpful if everyone could reply with URLs of build instructions on either Apple's or OpenDarwin's site. I would just use darwinbuild now. All the old build instructions out there on the internet are out of date now. This email sent to site_archiver(a)lists.apple.com

1 0

Re: no Makefile in IOKitUser-277.6
by Josh Wisenbaker 07 Nov '05

07 Nov '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Importance: Normal User-agent: SquirrelMail/1.4.4 > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > > I just noticed that IOKitUser-277.6 doesn't have a Makefile, making > it difficult to follow the building XNU instructions on > www.opendarwin.org. I'm resolving a problem with my ADC account and > can't presently file a bug report. > > Just a heads up, in case anyone who can correct this is reading the > list. I would just use darwinbuild now. All the old build instructions out there on the internet are out of date now. Josh www.afp548.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: no Makefile in IOKitUser-277.6
by Shantonu Sen 07 Nov '05

07 Nov '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Which instructions (so I can delete them)? Shantonu On Nov 6, 2005, at 5:42 PM, Jeff Nathan wrote: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Thanks for clarifying. Out of curiosity, will the build instructions be updated? - -Jeff On Nov 6, 2005, at 8:01 PM, Kevin Van Vechten wrote: - Kevin On Nov 6, 2005, at 4:58 PM, Jeff Nathan wrote: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 - -Jeff - -- http://cerberus.sourcefire.com/~jeff (DSA key id 6923D3FD) "Problems cannot be solved at the same level of awareness that created them." - Albert Einstein -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (Darwin) This email sent to kevin(a)opendarwin.org - -- The original EZ-bake packet oven. http://nemesis.sourceforge.net -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (Darwin) _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… The answer is no. You should be using darwinbuild (<http:// opendarwin.org/projects/darwinbuild/>) which handles all dependencies and whether to use xcodebuild(1) or make(1). IOKitUSer-277.6 has an Xcode project. Starting with 10.4.3, you must use Xcode to build IOKitUser. I just noticed that IOKitUser-277.6 doesn't have a Makefile, making it difficult to follow the building XNU instructions on www.opendarwin.org. I'm resolving a problem with my ADC account and can't presently file a bug report. Just a heads up, in case anyone who can correct this is reading the list. iD8DBQFDbqa8Eqr8+Gkj0/0RAniUAJ9UP2n3UkatIBHUTPxUQdbXNf4zHgCdFQG8 ugICF/oHJkGMGbj8c902EAg= =zJsf -----END PGP SIGNATURE----- _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/kevin% 40opendarwin.org iD8DBQFDbrEkEqr8+Gkj0/0RAveKAJ9FQb1wR3QA3YrPxtsvOvlzJhexugCfR0UQ lPgUXz3TZO0H83VisrlJ3TI= =nyDx -----END PGP SIGNATURE----- _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/ssen% 40opendarwin.org This email sent to ssen(a)opendarwin.org This email sent to site_archiver(a)lists.apple.com

1 0

Re: no Makefile in IOKitUser-277.6
by Jeff Nathan 07 Nov '05

07 Nov '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Thanks for clarifying. Out of curiosity, will the build instructions be updated? - -Jeff On Nov 6, 2005, at 8:01 PM, Kevin Van Vechten wrote: - Kevin On Nov 6, 2005, at 4:58 PM, Jeff Nathan wrote: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 - -Jeff - -- http://cerberus.sourcefire.com/~jeff (DSA key id 6923D3FD) "Problems cannot be solved at the same level of awareness that created them." - Albert Einstein -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (Darwin) This email sent to kevin(a)opendarwin.org - -- The original EZ-bake packet oven. http://nemesis.sourceforge.net -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (Darwin) iD8DBQFDbrEkEqr8+Gkj0/0RAveKAJ9FQb1wR3QA3YrPxtsvOvlzJhexugCfR0UQ lPgUXz3TZO0H83VisrlJ3TI= =nyDx -----END PGP SIGNATURE----- _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… IOKitUSer-277.6 has an Xcode project. Starting with 10.4.3, you must use Xcode to build IOKitUser. I just noticed that IOKitUser-277.6 doesn't have a Makefile, making it difficult to follow the building XNU instructions on www.opendarwin.org. I'm resolving a problem with my ADC account and can't presently file a bug report. Just a heads up, in case anyone who can correct this is reading the list. iD8DBQFDbqa8Eqr8+Gkj0/0RAniUAJ9UP2n3UkatIBHUTPxUQdbXNf4zHgCdFQG8 ugICF/oHJkGMGbj8c902EAg= =zJsf -----END PGP SIGNATURE----- _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/kevin% 40opendarwin.org This email sent to site_archiver(a)lists.apple.com

1 0

Re: no Makefile in IOKitUser-277.6
by Kevin Van Vechten 07 Nov '05

07 Nov '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com - Kevin On Nov 6, 2005, at 4:58 PM, Jeff Nathan wrote: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 - -Jeff - -- http://cerberus.sourcefire.com/~jeff (DSA key id 6923D3FD) "Problems cannot be solved at the same level of awareness that created them." - Albert Einstein -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (Darwin) _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… IOKitUSer-277.6 has an Xcode project. Starting with 10.4.3, you must use Xcode to build IOKitUser. I just noticed that IOKitUser-277.6 doesn't have a Makefile, making it difficult to follow the building XNU instructions on www.opendarwin.org. I'm resolving a problem with my ADC account and can't presently file a bug report. Just a heads up, in case anyone who can correct this is reading the list. iD8DBQFDbqa8Eqr8+Gkj0/0RAniUAJ9UP2n3UkatIBHUTPxUQdbXNf4zHgCdFQG8 ugICF/oHJkGMGbj8c902EAg= =zJsf -----END PGP SIGNATURE----- _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/kevin% 40opendarwin.org This email sent to kevin(a)opendarwin.org This email sent to site_archiver(a)lists.apple.com

1 0

no Makefile in IOKitUser-277.6
by Jeff Nathan 07 Nov '05

07 Nov '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 - -Jeff - -- http://cerberus.sourcefire.com/~jeff (DSA key id 6923D3FD) "Problems cannot be solved at the same level of awareness that created them." - Albert Einstein -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (Darwin) iD8DBQFDbqa8Eqr8+Gkj0/0RAniUAJ9UP2n3UkatIBHUTPxUQdbXNf4zHgCdFQG8 ugICF/oHJkGMGbj8c902EAg= =zJsf -----END PGP SIGNATURE----- _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… I just noticed that IOKitUser-277.6 doesn't have a Makefile, making it difficult to follow the building XNU instructions on www.opendarwin.org. I'm resolving a problem with my ADC account and can't presently file a bug report. Just a heads up, in case anyone who can correct this is reading the list. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Thread Priority
by AgentM 04 Nov '05

04 Nov '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On Nov 4, 2005, at 1:20 PM, Marc Van Olmen wrote: hi, I'm wondering if this is a good idea: My application has 3 threads: 1. Reading 2. Image Composing 3. Display so I had the idea to play with thread priority. When 15 frame buffers or free for rendering I give thread priority 80 When 14 frame buffers or free for rendering I give thread priority 73 ... When 1 frame buffers or free for rendering I give thread priority 20 0 frames the thread sleeps. Would this be a good idea? Or is it better that the thread has a constant priority? Should I also use the FixedPriority option: |-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|- AgentM agentm(a)themactionfaction.com |-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|- _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… In general, thread prioritizing should be avoided because it almost never does what you might expect. Several issues crop up immediately such as priority inversion and starvation. pthreads have APIs just to handle that. The plan you suggest below is pretty much certain to backfire- prioritizing ad-hoc will probably decrease performance instead of improve it. The priority/code maintainability and predictability trade-off is seldom valuable so Butenhof in "Programming POSIX Threads" recommends against thread prioritizing. If you are having performance issues now, there are a number of things you should experiment with: 1) increase/decrease your buffer pools 2) measure time spent per frame in each thread (use Shark or a different sampler) 3) add code to monitor how quickly the buffers are passed through the system 4) reduce polling- if you polling on a camera, make sure you aren't grabbing frames faster than the camera can produce them (the iSight pumps out 15 fps at 640x480) Shark (in the CHUD tools) can help you pin-point where you spending the most time. Optimize that and don't bother with the thread priorities. Good luck! The image composing thread (2) has a buffer of 15 frames, so 15 frames in advance can be generated. When ever a buffer comes available the thread wakes up and generates another one. The questions I have now is that I want to make sure that other threads have more time available to increase responsiveness or even in the future have more displays busy> Another question related, to make sure that my 2nd thread generates enough frames in advance. if (mFixedPriority) { thread_extended_policy_data_t theFixedPolicy; theFixedPolicy.timeshare = false; // set to true for a non- fixed thread BAIL_MSG_OSERROR(thread_policy_set(aMachPort, THREAD_EXTENDED_POLICY, (thread_policy_t)&theFixedPolicy, THREAD_EXTENDED_POLICY_COUNT), "cOS_Thread::SetPriority: failed to set the fixed-priority policy"); } I don't care about user responsive I want my application to run smooth so sending the frames out at correct timing for smooth animation. This email sent to site_archiver(a)lists.apple.com

1 0

Thread Priority
by Marc Van Olmen 04 Nov '05

04 Nov '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com hi, I'm wondering if this is a good idea: My application has 3 threads: 1. Reading 2. Image Composing 3. Display so I had the idea to play with thread priority. When 15 frame buffers or free for rendering I give thread priority 80 When 14 frame buffers or free for rendering I give thread priority 73 ... When 1 frame buffers or free for rendering I give thread priority 20 0 frames the thread sleeps. Would this be a good idea? Or is it better that the thread has a constant priority? Should I also use the FixedPriority option: regards, mvo _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… The image composing thread (2) has a buffer of 15 frames, so 15 frames in advance can be generated. When ever a buffer comes available the thread wakes up and generates another one. The questions I have now is that I want to make sure that other threads have more time available to increase responsiveness or even in the future have more displays busy> Another question related, to make sure that my 2nd thread generates enough frames in advance. if (mFixedPriority) { thread_extended_policy_data_t theFixedPolicy; theFixedPolicy.timeshare = false; // set to true for a non-fixed thread BAIL_MSG_OSERROR(thread_policy_set(aMachPort, THREAD_EXTENDED_POLICY, (thread_policy_t)&theFixedPolicy, THREAD_EXTENDED_POLICY_COUNT), "cOS_Thread::SetPriority: failed to set the fixed-priority policy"); } I don't care about user responsive I want my application to run smooth so sending the frames out at correct timing for smooth animation. This email sent to site_archiver(a)lists.apple.com

1 0

Re: 64-bit question
by AgentM 04 Nov '05

04 Nov '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com The program is listed belowl, to compile it just execute: gcc -arch ppc64 c-bigmem-test.c -o c-bigmem-test.exe "c-bigmem-test.exe" is the output - an executable file you may run. That depends on your definition of "correct". Here is the code -------- #include <stdio.h> #include <string.h> #include <stdlib.h> #define ARRAY_SIZE 1000000 int main(){ char **bigbuf; char buf [ARRAY_SIZE]; long long int i=0; for(i=0;i<ARRAY_SIZE;i++){ buf[i]='A'; } bigbuf=malloc(ARRAY_SIZE*sizeof(char *)); if(bigbuf==NULL){ printf("Main malloc failed!\n"); } i=0; while(++i){ char * str_tmp=malloc(ARRAY_SIZE*sizeof(char)); if(str_tmp==NULL){ printf("Internal malloc failed!\n"); } bigbuf[i]=str_tmp; strcpy(bigbuf[i], buf); if(i%10==0) {printf("MB allocated: %lld\n",i);}; } return 0; } -------- This email sent to agentm(a)themactionfaction.com |-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|- AgentM agentm(a)themactionfaction.com |-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|- _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On Nov 3, 2005, at 1:30 PM, Bob Bader wrote: ---------------- Observations from my users ----------------------- 1. The "top" command does not display the correct memory allocation status for a given process. To be precise, it is not able to show the values for memory greater than 4GB so it's a kind of hard to test high-mem applications. Looks like it's just the old 32bit "top", without 64bit capability. Yes. Unfortunately, even the 64-bit app can't query how much memory it is using because the kernel structures involved don't support 64- bit-sized reporting. Oops! http://lists.apple.com/archives/darwin-kernel/2005/Sep/msg00055.html and the follow-up: http://lists.apple.com/archives/darwin-kernel/2005/Sep/msg00061.html 2. We wrote and executed a custom C program to test the 64bit C compiler the system came with. It also tests 64bit environment and a system's general ability to handle high-mem apps. The application was able to allocate about 15GB on your test box and ... then the machine rebooted! Unacceptable! The program is straightforward and it just tries to allocate more and more memory and it doesn't stop by itself (leave it or kill it!). On the screen you'll see the value of current memory taken. We tested it on the 64bit Linux /the memory status obtained from 64bit "top" matched the values printed out by our C program/ and when all system resources (physical memory + swap/hdd/ memory ) ran out, it was killed by the kernel which is correct. Linux (by default) employs the so-called Out-Of-Memory Killer (OOM Killer) which allows malloc to overcommit and effectively lie about how much memory is available for allocation. If the overcommitted pages are then written (as you do below) then a heuristic takes over in the kernel to kill (with SIGKILL) some program-- cross your fingers and hope it doesn't kill your database or long-running number- cruncher. More details and the way to shut this off at: http://lwn.net/Articles/104179/ Obviously, you revealed a bug in the Darwin kernel's 64-bit handling (and an easy-to-use DOS), but POSIX has a well-defined method of notifying the program that there is no more memory- namely returning NULL from malloc(). So please don't rely on the KILL behavior. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/agentm% 40themactionfaction.com This email sent to site_archiver(a)lists.apple.com

1 0