March 2003 - Darwin-kernel - lists.apple.com

Re: ...Unhappy Childhood Persists
by Jim Magee 11 Mar '03

11 Mar '03

On Tuesday, March 11, 2003, at 02:56 PM, Lance Drake wrote: Does your application link against any Apple-specific frameworks other than libSystem? There should be quite a few references in the archives about "No other Apple frameworks, other than libSystem, can be combined with fork() and expect consistent results." You must immediately call exec() or _exit() (with only signal-safe calls between these and the fork() call). The process of elimination was finally brought to bear on this effort. Yes, there ARE other frameworks linked with the tool - so I went to the trouble of pulling out all frameworks and code - down to the minimum required to execute to the point it was crashing (Linked foundations are now: CoreFoundation and IOKit - removing Carbon, SystemConfiguration and DiskArbitration). Things appear now to be stable enough to execute that "runLoopSource = IONotificationPortGetRunLoopSource( sNotifyPort );" call which is where the 'complete' version konks out. Note that even IOKit potentially can get you in trouble. CoreFoundation isn't officially supported in fork()ing applications either, but it does go out of its way to try and make things work "post-fork()" anyway. One thing it doesn't do is reset the main runloop post-fork(). There should be something in the archives about doing that manually in your own code post-fork() (if you insist on doing the unsupported thing and continue to use these frameworks post-fork). Where I am at the moment is that it looks like I will have to take the fork/execve route - which seems unnecessary, considering the fact that it all works fine (as is) in 10.2.4 and nobody has anything to point to that represents a big change in the mach world from the 10.1 days. I suppose I should say (in my best Jerry Seinfeld) "HEY!... whatever works!", but I'd really like to find out why this is failing. You are just getting "luckier" in the 10.2.4 runs. What you are trying to do is equally unsupported in either build. All those other frameworks have globals initialized with names of Mach ports embedded in them. After the fork(), the memory is copied into the child, but not the ports. Eventually the port names (copies of which are lying in memory waiting to jump out and bite you) get reused in the child for some other purpose. When you do execute any of the code from those other frameworks that depend on those ports names, all kinds of craziness ensues. Some of those data structures may actually affect dyld's ability to resolve symbol names in the child (like the setsid() call which only the child calls). That's why being pre-bound sometimes alters this behavior. We have tried to "fix" this issue by putting in "at-fork()" handlers in these other frameworks. The problem is that the dependency ordering of the frameworks can be different in the child during the run of these handlers than they were during the first initialization in the parent. There is no good way (that we have found) to encapsulate these new dependencies so that the child could run them in the proper order. So, that's still a craps-shoot as to whether it would work or not. Until we have a solution that works reliably, we have to declare this behavior "offlimits." We understand that it is frustrating (especially to Unix programmers who are forced into using some of these other frameworks for "just one thing"), and have been working hard on a proper solution. Sorry for the inconvenience. --Jim _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

task_info and virtual_size
by joe udder 11 Mar '03

11 Mar '03

Hello. I am a little curious on how virtual memory per process works in Darwin. I am calling "task_info()" using TASK_BASIC_INFO to retrieve the amount of memory my process is using, but I receive very doubtful numbers. According to a webpage[1], the values are number of pages, but to me they look very much like the size in bytes instead. I came to this conclusion using the "top" and comparing the values for resident memory sizes. I suspect it is merely a typo in the headerfiles? However, the real confusion is "virtual_size", which is a value around 700MB while "top" shows ~90MB. And that one I can't understand at all. Anyone has an idea on how this works? .ju [1] : http://web.mit.edu/darwin/src/modules/xnu/osfmk/man/task_basic_info.html _________________________________________________________________ Add photos to your e-mail with MSN 8. Get 2 months FREE*. http://join.msn.com/?page=features/featuredemail _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: ...Unhappy Childhood Persists
by Lance Drake 11 Mar '03

11 Mar '03

Hi Jim, (et al...) In the words of Chester A. Riley - as played by William Bendix in 'The Life Of Riley', "What a revoltin' development this is!" For whatever reason, MacOSX 10.2 seems to be far more forgiving on this point as this tool has never crashed in that environment. As others have said, these are moral equivalents. If one fails, the other should as well. You are correct, sir! - This was not the problem - although I am especially miffed at how a light change in this area caused the failure mode to 'move' - thus making me look silly as I announced that the problem had been solved. For a time, the tool did not crash, but it was not receiving any notifications and I thought, "Well, time to drill down to the NEXT level of incompatibility". Actually, I was just lucky it was not tripping on its own shoelaces - which were still tied together. Are both versions pre-bound? I tried checking and unchecking the pre-binding box in PB. No effect. Often, people look at the effect of trivial changes like this and forget that the execution life-cycle of a process is radically changed whether it is pre-bound or not. In fact, a common dyld failure mode on 10.1 is to call abort() when it detects that it has painted itself in a corner. If this is what's happening (which sounds reasonable), I surely do not know how we got headed into that corner. Does your application link against any Apple-specific frameworks other than libSystem? There should be quite a few references in the archives about "No other Apple frameworks, other than libSystem, can be combined with fork() and expect consistent results." You must immediately call exec() or _exit() (with only signal-safe calls between these and the fork() call). The process of elimination was finally brought to bear on this effort. Yes, there ARE other frameworks linked with the tool - so I went to the trouble of pulling out all frameworks and code - down to the minimum required to execute to the point it was crashing (Linked foundations are now: CoreFoundation and IOKit - removing Carbon, SystemConfiguration and DiskArbitration). Things appear now to be stable enough to execute that "runLoopSource = IONotificationPortGetRunLoopSource( sNotifyPort );" call which is where the 'complete' version konks out. Not having the source-code for performFork(), I can't comment on whether it could trigger such problems. <SEE BELOW FOR 'performFork()' CODE> Where I am at the moment is that it looks like I will have to take the fork/execve route - which seems unnecessary, considering the fact that it all works fine (as is) in 10.2.4 and nobody has anything to point to that represents a big change in the mach world from the 10.1 days. I suppose I should say (in my best Jerry Seinfeld) "HEY!... whatever works!", but I'd really like to find out why this is failing. For now, your suggestion that the cause is a fallout from linking in other frameworks seems to have the highest probability of significance. I'll keep forging ahead with my 'process of addition' until I hit paydirt. Film at 11... Again, THANK YOU VERY MUCH for taking the time to consider this problem and to offer your valuable comments. Lance Drake // ------------------------------------------------------------------------ -------- int performFork(void) { uid_t pid; // prep parent to receive sigterm from child signal(SIGTERM, parent_handle_sigterm); pid = fork(); switch (pid) { case -1: // error return(0); break; case 0: // child task /* Set a new session for the child process. */ if (setsid() == -1) { //syslog(LOG_INFO, "setsid = -1, errno = %d", errno); return 0; } /* run relative to the root of the filesystem, just in case. */ if (chdir("/") == -1) { //syslog(LOG_INFO, "chdir('/') == -1"); return 0; } return (1); break; default: // parent task = wait for signal, then exit { int status; wait4(pid, (int *)&status, 0, 0); if (WIFEXITED(status)) { syslog(LOG_INFO, "ERROR: failed to start, exit status = %d", WEXITSTATUS(status)); } else { syslog(LOG_INFO, "ERROR: failed to start, received signal = %d", WTERMSIG(status)); } } break; } return(0); } // ------------------------------------------------------------------------ ---- _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Re[2]: Happy child (w/10.2), gets SIGABRT (w/10.1)
by Jim Magee 11 Mar '03

11 Mar '03

On Monday, March 10, 2003, at 05:08 PM, Lance Drake wrote: It appears the SIGABRT was caused by an invalid reference to a local stack var in the main function which was accepting the value returned by my call to the fork activities routine. The original code was: int result; ... result = performFork(); // create long-lived version of this process if(0 == result) // parent returning?... goto ExitPath; // say goodbye to parent pause(); // <-- inserted for debug, thanks to Pelle Johansson ... The code was changed to: if(0 == performFork()) // parent returning?... goto ExitPath; // say goodbye to parent // pause(); // <-- inserted for debug, thanks to Pelle Johansson For whatever reason, MacOSX 10.2 seems to be far more forgiving on this point as this tool has never crashed in that environment. As others have said, these are moral equivalents. If one fails, the other should as well. Are both versions pre-bound? Often, people look at the effect of trivial changes like this and forget that the execution life-cycle of a process is radically changed whether it is pre-bound or not. In fact, a common dyld failure mode on 10.1 is to call abort() when it detects that it has painted itself in a corner. Does your application link against any Apple-specific frameworks other than libSystem? There should be quite a few references in the archives about "No other Apple frameworks, other than libSystem, can be combined with fork() and expect consistent results." You must immediately call exec() or _exit() (with only signal-safe calls between these and the fork() call). Not having the source-code for performFork(), I can't comment on whether it could trigger such problems. --Jim _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Happy child (w/10.2), gets SIGABRT (w/10.1)
by Peter Lovell 11 Mar '03

11 Mar '03

On Monday, March 10, 2003, at 05:45 PM, Paul Ripke wrote: What is said in "Design of the UNIX Operating System" is true, however, as soon as your performFork() function returns, each process will have its own copy of "result". What's said in the book is that neither process can access the other processes *copy* of the variable. Like I said, glad we could help, but I don't know what fixed it :) Hi Lance, I agree with Paul - the change you made isn't the real "fix". After the fork, each process has its own space. I don't know what "performFork()" is doing but I presume it's more than just "fork()" as you check only for zero/nonzero. Have you checked the possible return values from the child? Make sure that your main() routine actually returns a proper exit value as that's what will be passed back to the parent. I don't think that would have caused the parent to think the child died with SIGABRT, but it is worth checking. Don't let main() finish without exit(). Regards.....Peter _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Happy child (w/10.2), gets SIGABRT (w/10.1)
by Paul Ripke 10 Mar '03

10 Mar '03

On Tuesday, Mar 11, 2003, at 09:08 Australia/Sydney, Lance Drake wrote: Hi Peter, Paul and Pelle, After getting up this morning, I discovered your three VERY helpful letters in my inbox. Thanks to your input and suggestions, I believe the problem is solved. Glad we could help! It appears the SIGABRT was caused by an invalid reference to a local stack var in the main function which was accepting the value returned by my call to the fork activities routine. The original code was: int result; ... result = performFork(); // create long-lived version of this process if(0 == result) // parent returning?... goto ExitPath; // say goodbye to parent pause(); // <-- inserted for debug, thanks to Pelle Johansson When the pause was reached, the SIGABRT had already been generated and none of the syslog error messages I had deposited in the performFork function were showing via the 'tail -f /var/log/system.log' command in a separate terminal window. The only other activity that had taken place was the evaluation of the return value from the performFork function. Evidently, when the child process returned from the performFork routine, the reference to 'result' was bogus and caused the exception. Maurice J. Bach would seem to confirm this, on page 196 in 'Design Of The Unix Operating system' (Prentice Hall, 1986), where he reports that "neither process can access the variables of the other process". The code was changed to: if(0 == performFork()) // parent returning?... goto ExitPath; // say goodbye to parent // pause(); // <-- inserted for debug, thanks to Pelle Johansson For whatever reason, MacOSX 10.2 seems to be far more forgiving on this point as this tool has never crashed in that environment. Thank you all VERY MUCH for giving me that extra legup of information and moral support! Well, I for one, am still confused. At almost any level of optimisation, the machine code emitted by gcc for the above two samples should be the same... What is said in "Design of the UNIX Operating System" is true, however, as soon as your performFork() function returns, each process will have its own copy of "result". What's said in the book is that neither process can access the other processes *copy* of the variable. Like I said, glad we could help, but I don't know what fixed it :) Cheers, -- Paul Ripke Unix/OpenVMS/TSM/DBA 101 reasons why you can't find your Sysadmin: 68: It's 9AM. He/She is not working that late. -- Koos van den Hout _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re[2]: Happy child (w/10.2), gets SIGABRT (w/10.1)
by Lance Drake 10 Mar '03

10 Mar '03

Do you know whether the abort happens right at the start, or after the child has done all (or some) of its work? - p.s. I assume that both the parent and child are re-built using 10.1.x Peter Lovell If it helps, you'll get a SIGABRT calling abort(3) or a failed assert()ion. Alternately, "ulimit -c unlimited" and let it core - SIGABRT should coredump if default signal handlers haven't been changed. Paul Ripke If that's not the case I'd recommend installing a SIGABRT handler, brake in it with gdb, and take a look at the backtrace. To make gdb work with the subprocess, make it pause() right after the fork and then look up the pid with ps, start gdb without any arguments, and type attach <pid>. Pelle Johansson ------------------------------------------------------------------------ ----- Hi Peter, Paul and Pelle, After getting up this morning, I discovered your three VERY helpful letters in my inbox. Thanks to your input and suggestions, I believe the problem is solved. It appears the SIGABRT was caused by an invalid reference to a local stack var in the main function which was accepting the value returned by my call to the fork activities routine. The original code was: int result; ... result = performFork(); // create long-lived version of this process if(0 == result) // parent returning?... goto ExitPath; // say goodbye to parent pause(); // <-- inserted for debug, thanks to Pelle Johansson When the pause was reached, the SIGABRT had already been generated and none of the syslog error messages I had deposited in the performFork function were showing via the 'tail -f /var/log/system.log' command in a separate terminal window. The only other activity that had taken place was the evaluation of the return value from the performFork function. Evidently, when the child process returned from the performFork routine, the reference to 'result' was bogus and caused the exception. Maurice J. Bach would seem to confirm this, on page 196 in 'Design Of The Unix Operating system' (Prentice Hall, 1986), where he reports that "neither process can access the variables of the other process". The code was changed to: if(0 == performFork()) // parent returning?... goto ExitPath; // say goodbye to parent // pause(); // <-- inserted for debug, thanks to Pelle Johansson For whatever reason, MacOSX 10.2 seems to be far more forgiving on this point as this tool has never crashed in that environment. Thank you all VERY MUCH for giving me that extra legup of information and moral support! Lance Drake _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Happy child (w/10.2), gets SIGABRT (w/10.1)
by Peter Lovell 10 Mar '03

10 Mar '03

On Monday, March 10, 2003, at 12:50 AM, Lance Drake wrote: After a call to wait4(ch_pid...), the parent, examining WIFEXITED, reports the child process has been quit via SIGABRT. (100% > repeatable) Discussion: ----------- In my tool, main() performs a fork(), the parent is supposed to quit and leave the child remaining and running. This is exactly what happens with the 10.2.4 implementation. When my manager asked for 10.1 support, it seemed like a simple enough thing - but noooooo... I have loaded a separate disk with OS 10.1 and installed the 2.0 PB tools, re-built the project and the formerly, completely reliable code sputters to a total standstill. If I skip the fork and simply execute, all of my code runs with no problems or exceptions - it works fine! But, as soon as the fork takes place, the child process dies a horrible death... Hi Lance, do you know whether the abort happens right at the start, or after the child has done all (or some) of its work? Regards.....Peter p.s. I assume that both the parent and child are re-built using 10.1.x _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: darwin-kernel digest, Vol 2 #243 - 3 msgs
by Brian Bergstrand 10 Mar '03

10 Mar '03

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Monday, March 10, 2003, at 08:26 AM, William Kucharski wrote: >> In the last case [ if (ISSET(vp->v_flag, VUINIT)) ], where the >> thread >> sleeps waiting for the VNODE to finish init, the interlock is dropped >> before waiting, but after returning from the wait it jumps immediately >> to retry: w/o checking flags for LK_INTERLOCK. If LK_INTERLOCK is set, >> then the interlock won't be acquired at the beginning of retry:. Maybe >> I'm missing something, but before going to retry, shouldn't >> LK_INTERLOCK be cleared or if set then then re-acquire the lock before >> the jump? > > What am I missing? > > The first lines of the routine are: > > retry: > > /* > * If the vnode is in the process of being cleaned out for > * another use, we wait for the cleaning to finish and then > * return failure. Cleaning is determined by checking that > * the VXLOCK flag is set. > */ > if ((flags & LK_INTERLOCK) == 0) > simple_lock(&vp->v_interlock); > > so when the VUINIT leg calls simple_unlock and sleeps, when it wakes > and > does a goto retry the lock will be reacquired if LK_INTERLOCK is still > 0. Right, but if the flag is set the VUINIT block still drops the interlock, and since that flag is set, it won't be re-acquired at the beginning of the loop. No lock == race condition or possible vnode corruption. > LK_INTERLOCK is not set by the simple_lock() call, or by any > intervening code > in vget(), so no flags are set that need to be unset; the release of > v_interlock is sufficient. > LK_INTERLOCK is set by the caller when they have already acquired the vnode interlock before calling vget. vget/vn_lock/others need to know this because they mess with the lock state. In this case, it looks like vget has a bug in its "messing" code. Brian Bergstrand <http://www.classicalguitar.net/brian/> PGP Key ID: 0xB6C7B6A2 All programmers are playwrights and all computers are lousy actors. - Unknown -----BEGIN PGP SIGNATURE----- Version: PGP 8.0 iQA/AwUBPmy5ennR2Fu2x7aiEQJLQwCfTx7Q94xIWmQM3UbqFn1veuQixiIAn0c4 FZT/UHdmCDnSVEw/t+UMCN6U =jAVC -----END PGP SIGNATURE----- _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Locking bug in vget()?
by Brian Bergstrand 10 Mar '03

10 Mar '03

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Monday, March 10, 2003, at 03:52 AM, Quinn wrote: > At 1:21 -0600 9/3/03, Brian Bergstrand wrote: >> I think I may have found a bug in vget(). > > I took a quick look at this and it looks like a problem to me too. > Please file a bug so that one of our kernel engineers can look at this > and makes a final determination. > > <http://developer.apple.com/bugreporter/> Thanks Quinn. I'll do that. I just wanted some feedback before filing a bug. Extra set of a eyes and all... Brian Bergstrand <http://www.classicalguitar.net/brian/> PGP Key ID: 0xB6C7B6A2 I can't complain, but sometimes I still do. - Joe Walsh -----BEGIN PGP SIGNATURE----- Version: PGP 8.0 iQA/AwUBPmy5d3nR2Fu2x7aiEQK0MQCdFVoJpSiCCuQo0PoJAszivSfiv/kAoKjP +0pKIke/bjoNgTt4An4XSi0u =FQPi -----END PGP SIGNATURE----- _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0