March 2008 - Darwin-kernel - lists.apple.com

RE: per process file attributes
by Manish Chaturvedi 20 Mar '08

20 Mar '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Thread-index: AciKTqf1Zl+LkrURS5qLGAeNK0ritgAATvpg Thanks Terry for quick reply, but using lsof in our program will make me to parse the output, which seems to be a complicated job, is there is any alternate solution to this. Will be helpful if I follow the source code of lsof to do the same? Manish -----Original Message----- From: Terry Lambert [mailto:tlambert@apple.com] Sent: Thursday, March 20, 2008 11:24 AM To: Manish Chaturvedi Cc: darwin-kernel(a)lists.apple.com Subject: Re: per process file attributes The man page is cribbed from BSD, and not updated. We consider sysctl as SPI due to not having gone through API review, and therefore both unstable and not for use by third parties. If you need this information, consider using "lsof". -- Terry On Mar 19, 2008, at 10:36 PM, Manish Chaturvedi wrote: > Hi All, > > I am writing application to list all process and opened files by the > process, where I am able to get the process information like > Pid,ppid, path, process commadline arguments etc using sysctl() > ( followed this link http://developer.apple.com/qa/qa2001/qa1123.html) > . > > I am using the following code to get the file information about > every process. > > int mib[3]; > struct extern_file *file = NULL ; // Is it the correct structure ? > > mib[0] = CTL_KERN; > mib[1] = KERN_MAXFILES; > mib[2] = 1693 ; > > size = sizeof(argmax) ; > > if (sysctl((int *)mib, 2, &argmax, &size, NULL, 0) < 0) { > printf("No args found\n"); > printf("Error is %s\n", strerror(errno)) ; > > } > else > { > printf("size is Given as %d\n",argmax) ; // argmax is > giving me the correct values (i.e. count of all opened files ) > size = (size_t)argmax ; > > mib[0] = CTL_KERN; // Here I am resetting the mib arry > for getting per process file information . > mib[1] = KERN_FILE; > mib[2] = 1693 ;//1693 ;//procList[i].kp_proc.p_pid; > > file =(extern_file*) malloc(argmax); > if(file == NULL) > { > printf("file pointe not allocated yes\n") ; > } > if (sysctl((int *)mib, 3, fileNew, &size, NULL, 0) < 0) > { > printf("Error in getting file struct %s\n", > strerror(errno)) ; > } > else > { > // Print file attributes > } > > I want to list all file descriptors file paths and othet file > attributes , which I am not getting with this program, Getting an > error "Not a directory" during second sysctl call, where as getting > the correct file count after first sysctl() call. > In the man page of sysctl() KERN_FILE will return a file structure > which is not at all documented ( not confirm about the structure to > use ), there is no file structure exposed on Mac OS X (apart from > extern_file and filedesc), > > I am not sure which structure to use ? is it possible to get per > process file information using KERN_FILE? > OS version is 10.4.7 > Thanks in advance! > > Regards, > Manish. > _______________________________________________ > Do not post admin requests to the list. They will be ignored. > Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) > Help/Unsubscribe/Update your Subscription: > http://lists.apple.com/mailman/options/darwin-kernel/tlambert%40apple.com > > This email sent to tlambert(a)apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: per process file attributes
by Terry Lambert 20 Mar '08

20 Mar '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com -- Terry Hi All, int mib[3]; struct extern_file *file = NULL ; // Is it the correct structure ? mib[0] = CTL_KERN; mib[1] = KERN_MAXFILES; mib[2] = 1693 ; size = sizeof(argmax) ; if (sysctl((int *)mib, 2, &argmax, &size, NULL, 0) < 0) { printf("No args found\n"); printf("Error is %s\n", strerror(errno)) ; Regards, Manish. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/tlambert%40apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… The man page is cribbed from BSD, and not updated. We consider sysctl as SPI due to not having gone through API review, and therefore both unstable and not for use by third parties. If you need this information, consider using "lsof". On Mar 19, 2008, at 10:36 PM, Manish Chaturvedi wrote: I am writing application to list all process and opened files by the process, where I am able to get the process information like Pid,ppid, path, process commadline arguments etc using sysctl() ( followed this link http://developer.apple.com/qa/qa2001/qa1123.html) . I am using the following code to get the file information about every process. } else { printf("size is Given as %d\n",argmax) ; // argmax is giving me the correct values (i.e. count of all opened files ) size = (size_t)argmax ; mib[0] = CTL_KERN; // Here I am resetting the mib arry for getting per process file information . mib[1] = KERN_FILE; mib[2] = 1693 ;//1693 ;//procList[i].kp_proc.p_pid; file =(extern_file*) malloc(argmax); if(file == NULL) { printf("file pointe not allocated yes\n") ; } if (sysctl((int *)mib, 3, fileNew, &size, NULL, 0) < 0) { printf("Error in getting file struct %s\n", strerror(errno)) ; } else { // Print file attributes } I want to list all file descriptors file paths and othet file attributes , which I am not getting with this program, Getting an error “Not a directory” during second sysctl call, where as getting the correct file count after first sysctl() call. In the man page of sysctl() KERN_FILE will return a file structure which is not at all documented ( not confirm about the structure to use ), there is no file structure exposed on Mac OS X (apart from extern_file and filedesc), I am not sure which structure to use ? is it possible to get per process file information using KERN_FILE? OS version is 10.4.7 Thanks in advance! This email sent to tlambert(a)apple.com This email sent to site_archiver(a)lists.apple.com

1 0

per process file attributes
by Manish Chaturvedi 20 Mar '08

20 Mar '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Thread-index: AciKTGRtrNUFmTlcTIG27b9DlXcHUw== _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: kernel panic on wakeup from hibernation
by Ratheesh Ramachandran 19 Mar '08

19 Mar '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:MIME-Version:Content-Type:Message-ID; b=ARwX0rx9Hd9wsSqskLG2xl12wa2JLWzmuzzprvrp9F56hZTa3XQplJZ6sb0WK/V3eXMyL9sLh9aZ9lTSC3ak94ix8WRWFz2pT4Ij2vMVMxOoheBcBx4Jgo44rNj63fBs9DuS2sT9e4IddEeu/ZPGdyqnSkA0rO1Q2/8g9G3k4xs=; _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: kernel panic on wakeup from hibernation
by Terry Lambert 19 Mar '08

19 Mar '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com > See execargs_alloc in xnu in bsd/kern/kern_exec.c for an example of > how to do an allocation using semaphore_wait() without breaking. - If you truly want to go down this path, then you should get a copy of: Mac OS X Internals Amit Singh Addison Wesley Publishers ISBN 0-321-27854-2 ...and look up assert_wait(), thread_block(), and thread_should_abort(). - My personal recommendation would be that: -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… On Mar 19, 2008, at 1:40 AM, Ratheesh Ramachandran wrote: I looked at this. It is returning without allocating if the semaphore_wait() returns a KERN_ABORTED... which is fine. Your allocation does not do this, and that's what's wrong with your allocation, at least according to your pseudo-code. You try to get the lock, if you don't get it, you allocate anyway, because you have a lock() function that internally uses semaphore_wait(), and you don't check the return on your lock() function to decide to abort the allocation attempt. So then you attempt the allocation and stomp over a pointer that's supposed to be protected by the lock (semaphore), but isn't, and then you wonder why you have problems. Then you compound the problem by calling the unlock(0 function, which if it has transactional semantics, means you drop the semaphore that someone else allocated, causing them to be unprotected in their data access as well. You need to follow rigid rules when you involved locking; you have to pick whether you are locking code (critical sectioning) or data (object access protection). Locking data is typically a better approach, just from a general CS standpoint, since it increases parallelism. And then you never, never, never violate the locking protocol that protects the integrity of whatever it is you lock. > Alternately, indicate that your thread us uninterruptible until it > gets the lock (and make sure that you _get_ the lock sometime, or you > will block forever -- better to just do it right). Will this approach work with semaphores ? I mean -- does semaphore_wait require thread to be interruptible ? And, how do you set the thread to be uninterruptible ? Any samples for this around ? thread_set_state seems promising, but, not sure if thats the one. Can this be done on the current_thread ? Forget that I said this. "Doing it right" means handling the interruption. The alternatative requires a deeper understanding of how the Mach synchronization primitives work. I could point you at xnu bsd/kern/kern_synch.c, but I doubt that without the context behind it that that would be very useful to you. + Keep it simple + Don't "roll your own locks" using semaphores + Use lck_mtx_* or IOLock* primitives (depending on if you are an IOKit driver) + lock access to data objects + forget trying to play with preventing aborts happening + just catch when they do and implement error recovery This email sent to site_archiver(a)lists.apple.com

1 0

Re: kernel panic on wakeup from hibernation
by mm w 19 Mar '08

19 Mar '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; bh=H/3Ii8OHWb+Cf7O5xAbJbH+DLyGX7QAwV2apAYbWVfs=; b=ib0AzJd8deUWhF+wTEytoPsHRq/Vb7FJ2sjGxxCoJ7X3i6FG0lRRYChCYj9N2dORLi4weKYMf0AWltLcZkLL3Lj8/hv77mHzjDa6uKQOW6vant1YhyBcV9CgghPe+TfyaUgb3XkF8p8Hk+9fKXvvSvsW+4UcXV1GTi6KsYYHYik= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=dA45M2r92B9yxpa5bkWpQ+X/VmjT20fzLwAevR2evjNsSfNYQ5iw7XK2PWRi/qeibeSwPoLPOQdWrgtrocPyK7LxbDzxaWoVvIijLr9M9FXBEJoAak/X4pwasPRxpCUa66ad01CEzHXMxfnBIq4mO+0MCzE96ILCi09skbGp6QM= On Wed, Mar 19, 2008 at 1:40 AM, Ratheesh Ramachandran <ratheesh_jakarta(a)yahoo.com> wrote: > > > > See execargs_alloc in xnu in bsd/kern/kern_exec.c for an example of > > how to do an allocation using semaphore_wait() without breaking. > > I looked at this. It is returning without allocating if the semaphore_wait() > returns a KERN_ABORTED... which is fine. > > > > Alternately, indicate that your thread us uninterruptible until it > > gets the lock (and make sure that you _get_ the lock sometime, or you > > will block forever -- better to just do it right). > > Will this approach work with semaphores ? I mean -- does semaphore_wait > require thread to be interruptible ? > I think so > And, how do you set the thread to be uninterruptible ? Any samples for this assert_wait thread_block > around ? thread_set_state seems promising, but, not sure if thats the one. > Can this be done on the current_thread ? > the thread_set_state methods is machine dependent, however you can the status using thread_info ( thread_basic_info struct ) > Regards, > Ratheesh > > > ----- Original Message ---- > From: Terry Lambert <tlambert(a)apple.com> > To: Ratheesh Ramachandran <ratheesh_jakarta(a)yahoo.com> > Cc: Michael Smith <drivers(a)mu.org>; Darwin-Kernel List > <darwin-kernel(a)lists.apple.com> > Sent: Wednesday, 19 March, 2008 2:48:29 AM > Subject: Re: kernel panic on wakeup from hibernation > > > On Mar 18, 2008, at 1:33 AM, Ratheesh Ramachandran wrote: > > My thinking is that this is not a "regular" bad access scenario. > > But, something getting triggered by thread-safety issues (ie. the > > semaphore_wait's KERN_ABORTED error). I have a memory allocator > > function which looks something like this: > > > > void *my_malloc(int size, char *tag) { > > > > lock(); //calls semaphore_wait internally > > > > //get memory > > > > unlock(); > > } > > > > Notice, I am not checking the result of lock. > > > > If somebody does a thread_terminate() on one of the threads waiting > > on lock, thread-safety is compromised ie, multiple threads may > > access the critical section. The memory allocator may assign the > > same chunk of memory to different threads, one of them may free it, > > crashing the others. > > > > Is my reasoning correct ie. consistent with how darwin works ? > > > > What is the best way to handle such a situation ? Should my_malloc() > > have returned if lock() returned a KERN_ABORTED ? Should it have > > retried ? Will KERN_ABORTED always imply thread termination ? > > You are doing this wrong. > > See execargs_alloc in xnu in bsd/kern/kern_exec.c for an example of > how to do an allocation using semaphore_wait() without breaking. > > Alternately, indicate that your thread us uninterruptible until it > gets the lock (and make sure that you _get_ the lock sometime, or you > will block forever -- better to just do it right). > > -- Terry > > > ________________________________ > > Share files, take polls, and discuss your passions - all under one roof. > Click here. > _______________________________________________ > Do not post admin requests to the list. They will be ignored. > Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) > Help/Unsubscribe/Update your Subscription: > > http://lists.apple.com/mailman/options/darwin-kernel/openspecies%40gmail.com > > This email sent to openspecies(a)gmail.com > -- -mmw _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Locking a shared memory object (crash safe )
by Jonas Maebe 19 Mar '08

19 Mar '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On 19 Mar 2008, at 14:50, Nicholaz Beresford wrote: Indeed. Jonas _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… Jonas Maebe wrote: Note that if you unlink the semaphore right after creating it, trying to sem_open() the same semaphore (i.e., passing the same name to sem_open) will get you a new semaphore rather than the previously opened/unlinked one (and this may or may not be a problem). As far as I understand it, other processes (started later) would then create their own (independent) semaphore which would defy the purpose of what I am trying to achieve (allowing a cross process lock). That approach is indeed only useful if you use it for communication across different threads (so you can pass along the pointer to the created semaphore) rather than communication across different processes. In this case you can always create a launcher wrapper app which creates/cleans up the semaphores (possibly already cleaning them up if they existed in the first place, so you also work around force quitting the launcher app). This email sent to site_archiver(a)lists.apple.com

1 0

Re: Locking a shared memory object (crash safe )
by Nicholaz Beresford 19 Mar '08

19 Mar '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com User-agent: Thunderbird 2.0.0.12 (Windows/20080213) Thanks Nick _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… Jonas Maebe wrote: Note that if you unlink the semaphore right after creating it, trying to sem_open() the same semaphore (i.e., passing the same name to sem_open) will get you a new semaphore rather than the previously opened/unlinked one (and this may or may not be a problem). As far as I understand it, other processes (started later) would then create their own (independent) semaphore which would defy the purpose of what I am trying to achieve (allowing a cross process lock). This email sent to site_archiver(a)lists.apple.com

1 0

Re: Locking a shared memory object (crash safe )
by Jonas Maebe 19 Mar '08

19 Mar '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On 19 Mar 2008, at 14:28, Nicholaz Beresford wrote: creation: sem=sem_open(...) sem_unlink(sem) Jonas _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… I'm porting an application from WIN32 which does shared memory. Under Windows we use a Named Mutex to lock the memory. This works well even if the app crashes (Win32 releases the mutex when crashing). Under darwin we're doing it via posix shared-mem and posix sems, but posix sems retain their states when the app goes down, so from that point every access to the shared mem will lock until the machine is rebooted. You can manually delete named semaphores using the ipcs/ipcrm command line program, there's no need to reboot (or you can write a helper program which does a sem_open() of the semaphore without O_CREAT| O_EXCL, followed by a sem_close()). Furthermore, you can ensure that the semaphore is freed when your program ends (crashing or not) by immediately unlinking the semaphore after you open it. So, destruction (which also happens automatically when the process stops due to the unlink above): sem_close(sem) Note that if you unlink the semaphore right after creating it, trying to sem_open() the same semaphore (i.e., passing the same name to sem_open) will get you a new semaphore rather than the previously opened/unlinked one (and this may or may not be a problem). This email sent to site_archiver(a)lists.apple.com

1 0

Locking a shared memory object (crash safe )
by Nicholaz Beresford 19 Mar '08

19 Mar '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com User-agent: Thunderbird 2.0.0.12 (Windows/20080213) #### We're catching the following signals: SIGHANDLER::InterceptSignal(SIGHUP, yesno); SIGHANDLER::InterceptSignal(SIGINT, yesno); SIGHANDLER::InterceptSignal(SIGABRT, yesno); SIGHANDLER::InterceptSignal(SIGSTOP, yesno); SIGHANDLER::InterceptSignal(SIGKILL, yesno); SIGHANDLER::InterceptSignal(SIGTERM, yesno); SIGHANDLER::InterceptSignal(SIGTRAP, yesno); SIGHANDLER::InterceptSignal(SIGBUS, yesno); SIGHANDLER::InterceptSignal(SIGSEGV, yesno); SIGHANDLER::InterceptSignal(SIGILL, yesno); SIGHANDLER::InterceptSignal(SIGFPE, yesno); #### I'm setting up signal handlers like this: struct sigaction new_action= NULLSTRUCT, old_action= NULLSTRUCT; ASSERT(signum<ARRAYMAX(SIGHANDLER::OldActions)); // Set up the structure to specify the new action new_action.sa_handler= SIGHANDLER::SigHandler; sigemptyset(&new_action.sa_mask); new_action.sa_flags = SA_RESETHAND; // only once sigaction(signum, NULL, &SIGHANDLER::OldActions[signum]); if (old_action.sa_handler!=SIG_IGN) sigaction(signum, &new_action, &SIGHANDLER::OldActions[signum]); sigaction(signum, &SIGHANDLER::OldActions[signum], NULL); memset(&SIGHANDLER::OldActions[signum], 0, sizeof(struct sigaction)); raise(signum); // raise the signal again to the previous handler Thanks! Nick _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… I'm porting an application from WIN32 which does shared memory. Under Windows we use a Named Mutex to lock the memory. This works well even if the app crashes (Win32 releases the mutex when crashing). Under darwin we're doing it via posix shared-mem and posix sems, but posix sems retain their states when the app goes down, so from that point every access to the shared mem will lock until the machine is rebooted. To avoid this I want to intercept the crash signals, post semaphores and proceed with the standard signal handling. Details are below, but what I'd like to know if this is sensible or if I'm doing something utterly stupid (we're not using signal()/sigaction() anywhere else in the code, so we're only dealing with the default behavior and want to get a chance to clean up the most urgent stuff between the crash and the standard crash-handler). From observation it seems to work as planned, but I'm using this family of functions for the first time, so I'd want to be sure (unless of course someone has a better way than using posix sems in the first place, but flock() doesn't seeem to work on shared mem). #### Then in the signal handler I'm restoring the old behavior (and we're using sigaction with RESETHAND), then raise the signal again (after posting my locked semaphores of course). This email sent to site_archiver(a)lists.apple.com

1 0