February 2004 - Darwin-kernel

Re: Deadlock in My KEXT with NKE
by Matt Jaffa 12 Feb '04

12 Feb '04

Ok I tend to not explain what I am doing real well, sorry about that. I don't want to hold the funnel while it is sleeping, but keeping the state as it was before it slept. But the most important thing I am trying to do here is make it so that the Thread is sleeping while work is being done by the daemon so that it doesn't do a while loop without sleeping in the kernel, here is my while loop: work_to_do = 1; //int stuff = 0; while(conditional_val == 0) { (void) tsleep((caddr_t) & so->so_timeo, PSOCK, "Testing", 2 * hz); } //*top = NULL; conditional_val = 0; //return EACCES; work_to_do = 0; so as you see in the while loop it sleeps every time it goes through the loop so it doesn't spend lots of CPU trudging through the loop over and over. It does the while loop while the dameon is processing information. Also is there a way for the KEXT to detect if a daemon is running? On Feb 11, 2004, at 7:03 PM, Shawn Erickson wrote: > I don't think you can or should be able to hold a funnel when/if you > sleep your thread. Once your thread wakes I believe it gets the funnel > back before continuing if it held the funnel when it slept. > > Given what you have listed so far I cannot make much sense of what you > are attempting to do... maybe some else can...? > > -Shawn > > On Feb 11, 2004, at 5:52 PM, Matt Jaffa wrote: > >> SO I was reading in the archives that if my NKE send function is >> blocked(or in this case sleeping) it releases control of the funnel, >> this is what could be causeing the Deadlock maybe. >> >> And I was also reading that you have to lock it so that the state >> isn't changed and won't cause this deadlock, it pointed me to >> sys/lock.h >> >> Using this how do I lock that so that it doesn't give up the funnel >> while it is sleeping. >> >> thanks, >> Matt >> >> >> On Feb 11, 2004, at 1:39 PM, Matt Jaffa wrote: >> >>> Hi all, >>> >>> I have done alot of researching on this and can't find a way to do >>> this. >>> >>> I have my KEXT which is a NKE, and I am specifically playing with >>> the send function so that >>> I can monitor and control all outgoing traffic from my computer. >>> I have succeeded in finishing my program, but after using it for a >>> little bit >>> it goes into Deadlock. >>> >>> I look at my code and I can see why. >>> >>> In my send function I have it surrounded by the the thread_tunnel >>> involving network_flock as to make sure only one send function is >>> being dealed with at a time, After which my send function signals a >>> global static variable work_to_do = 1, that informs the daemon >>> which is polling using sysctlbyname(...), that there is work to be >>> done, and then in the KEXT's NKE's send function it does a >>> while(conditional_val == 0) { >>> >>> the functions that causes it to sleep; >>> } >>> >>> then when the dameon is finished with the work, it signals by >>> sysctlbyname that conditional_val = 1 which causes the NKE's >>> send function to get out of the while loop. >>> >>> This way is working pretty well, except after awhile of using it, it >>> gums up and creates deadlock and also the CFUserNotificationCreate >>> in my Daemon stops popping up windows asking for input on what to do >>> for a connection, >>> >>> And finally the only thing i can do then is hold down the poser >>> button to reboot. >>> >>> I know probably the while loop is very stupid and dangerous to use >>> in the KEXT, it brings up questions like, what if the Daemon failed, >>> then it would be in a inifinite loop forever. >>> >>> So any suggestions of what to do now, of how to know if the Daemon >>> is running and also to block the send function until my dameon is >>> done with work and then to notify the NKE. Presently my daemon when >>> it first starts up it signals to the KEXT it is running and sets >>> that global static variable, but if the daemon were to die, the >>> KEXT wouldn't know. >>> >>> Thanks for listening to my blabbing on. >>> Matt >>> >>> P.S. I really did research this but I can't find anything on it, >>> yeah maybe I don't know how to search well, but please help. >>> I hope I explained this as best as possible. >>> _______________________________________________ >>> darwin-kernel mailing list | darwin-kernel(a)lists.apple.com >>> Help/Unsubscribe/Archives: >>> http://www.lists.apple.com/mailman/listinfo/darwin-kernel >>> Do not post admin requests to the list. They will be ignored. >> _______________________________________________ >> darwin-kernel mailing list | darwin-kernel(a)lists.apple.com >> Help/Unsubscribe/Archives: >> http://www.lists.apple.com/mailman/listinfo/darwin-kernel >> Do not post admin requests to the list. They will be ignored. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Deadlock in My KEXT with NKE
by Jim Magee 12 Feb '04

12 Feb '04

On Feb 11, 2004, at 8:52 PM, Matt Jaffa wrote: > SO I was reading in the archives that if my NKE send function is > blocked(or in this case sleeping) it releases control of the funnel, > this is what could be causeing the Deadlock maybe. > > And I was also reading that you have to lock it so that the state > isn't changed and won't cause this deadlock, it pointed me to > sys/lock.h > > Using this how do I lock that so that it doesn't give up the funnel > while it is sleeping. You absolutely MUST give up the funnel while sleeping. Did you mean "How do I take another lock without that potentially blocking - thus giving up the funnel anyway - and putting me right back in the same boat?" You have to use non-blocking lock tries combined with flags (set while holding the funnel) to coordinate it all. As a result, it's not easy to get right. And now you start to realize why we like to "pigeon hole" all the KEXTs. As long as you stay in category of KEXT, we have done all the dirty work for you (and largely hidden it behind somewhat standard interfaces). But once you start writing "mixed-mode" KEXTs, the encapsulation we have provided falls away. That's why it is sometimes easier to deal with a less than ideal arrangement that fits the pre-conceived notions of what a KEXT is, rather than trying to find the optimal combination of services to use. --Jim [demime 0.98b removed an attachment of type application/pkcs7-signature which had a name of smime.p7s] _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Deadlock in My KEXT with NKE
by Justin Walker 12 Feb '04

12 Feb '04

On Wednesday, February 11, 2004, at 05:52 PM, Matt Jaffa wrote: SO I was reading in the archives that if my NKE send function is blocked(or in this case sleeping) it releases control of the funnel, this is what could be causeing the Deadlock maybe. And I was also reading that you have to lock it so that the state isn't changed and won't cause this deadlock, it pointed me to sys/lock.h Using this how do I lock that so that it doesn't give up the funnel while it is sleeping. You have to give up the funnel whilst sleeping. It's not just a good idea; it's the law. The idea is that you are sleeping, waiting for state to change. If you have a lock on that state, you won't permit other threads to modify that state. Hence... Thumb's Rule says: if you want to sleep, make sure your house is in order; you can't tell what's going to happen while you are asleep, so assure that nothing bad happens. Who knows? Your thread may never get to run again. Regards, Justin -- Justin C. Walker, Curmudgeon-At-Large * Institute for General Semantics | If you're not confused, | You're not paying attention *--------------------------------------*-------------------------------* -- /~\ The ASCII Justin C. Walker, Curmudgeon-at-Large \ / Ribbon Campaign X Help cure HTML Email / \ _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Deadlock in My KEXT with NKE
by Shawn Erickson 12 Feb '04

12 Feb '04

I don't think you can or should be able to hold a funnel when/if you sleep your thread. Once your thread wakes I believe it gets the funnel back before continuing if it held the funnel when it slept. Given what you have listed so far I cannot make much sense of what you are attempting to do... maybe some else can...? -Shawn On Feb 11, 2004, at 5:52 PM, Matt Jaffa wrote: SO I was reading in the archives that if my NKE send function is blocked(or in this case sleeping) it releases control of the funnel, this is what could be causeing the Deadlock maybe. And I was also reading that you have to lock it so that the state isn't changed and won't cause this deadlock, it pointed me to sys/lock.h Using this how do I lock that so that it doesn't give up the funnel while it is sleeping. thanks, Matt On Feb 11, 2004, at 1:39 PM, Matt Jaffa wrote: Hi all, I have done alot of researching on this and can't find a way to do this. I have my KEXT which is a NKE, and I am specifically playing with the send function so that I can monitor and control all outgoing traffic from my computer. I have succeeded in finishing my program, but after using it for a little bit it goes into Deadlock. I look at my code and I can see why. In my send function I have it surrounded by the the thread_tunnel involving network_flock as to make sure only one send function is being dealed with at a time, After which my send function signals a global static variable work_to_do = 1, that informs the daemon which is polling using sysctlbyname(...), that there is work to be done, and then in the KEXT's NKE's send function it does a while(conditional_val == 0) { the functions that causes it to sleep; } then when the dameon is finished with the work, it signals by sysctlbyname that conditional_val = 1 which causes the NKE's send function to get out of the while loop. This way is working pretty well, except after awhile of using it, it gums up and creates deadlock and also the CFUserNotificationCreate in my Daemon stops popping up windows asking for input on what to do for a connection, And finally the only thing i can do then is hold down the poser button to reboot. I know probably the while loop is very stupid and dangerous to use in the KEXT, it brings up questions like, what if the Daemon failed, then it would be in a inifinite loop forever. So any suggestions of what to do now, of how to know if the Daemon is running and also to block the send function until my dameon is done with work and then to notify the NKE. Presently my daemon when it first starts up it signals to the KEXT it is running and sets that global static variable, but if the daemon were to die, the KEXT wouldn't know. Thanks for listening to my blabbing on. Matt P.S. I really did research this but I can't find anything on it, yeah maybe I don't know how to search well, but please help. I hope I explained this as best as possible. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Deadlock in My KEXT with NKE
by Matt Jaffa 12 Feb '04

12 Feb '04

SO I was reading in the archives that if my NKE send function is blocked(or in this case sleeping) it releases control of the funnel, this is what could be causeing the Deadlock maybe. And I was also reading that you have to lock it so that the state isn't changed and won't cause this deadlock, it pointed me to sys/lock.h Using this how do I lock that so that it doesn't give up the funnel while it is sleeping. thanks, Matt On Feb 11, 2004, at 1:39 PM, Matt Jaffa wrote: Hi all, I have done alot of researching on this and can't find a way to do this. I have my KEXT which is a NKE, and I am specifically playing with the send function so that I can monitor and control all outgoing traffic from my computer. I have succeeded in finishing my program, but after using it for a little bit it goes into Deadlock. I look at my code and I can see why. In my send function I have it surrounded by the the thread_tunnel involving network_flock as to make sure only one send function is being dealed with at a time, After which my send function signals a global static variable work_to_do = 1, that informs the daemon which is polling using sysctlbyname(...), that there is work to be done, and then in the KEXT's NKE's send function it does a while(conditional_val == 0) { the functions that causes it to sleep; } then when the dameon is finished with the work, it signals by sysctlbyname that conditional_val = 1 which causes the NKE's send function to get out of the while loop. This way is working pretty well, except after awhile of using it, it gums up and creates deadlock and also the CFUserNotificationCreate in my Daemon stops popping up windows asking for input on what to do for a connection, And finally the only thing i can do then is hold down the poser button to reboot. I know probably the while loop is very stupid and dangerous to use in the KEXT, it brings up questions like, what if the Daemon failed, then it would be in a inifinite loop forever. So any suggestions of what to do now, of how to know if the Daemon is running and also to block the send function until my dameon is done with work and then to notify the NKE. Presently my daemon when it first starts up it signals to the KEXT it is running and sets that global static variable, but if the daemon were to die, the KEXT wouldn't know. Thanks for listening to my blabbing on. Matt P.S. I really did research this but I can't find anything on it, yeah maybe I don't know how to search well, but please help. I hope I explained this as best as possible. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Deadlock in My KEXT with NKE
by Matt Jaffa 11 Feb '04

11 Feb '04

Hi all, I have done alot of researching on this and can't find a way to do this. I have my KEXT which is a NKE, and I am specifically playing with the send function so that I can monitor and control all outgoing traffic from my computer. I have succeeded in finishing my program, but after using it for a little bit it goes into Deadlock. I look at my code and I can see why. In my send function I have it surrounded by the the thread_tunnel involving network_flock as to make sure only one send function is being dealed with at a time, After which my send function signals a global static variable work_to_do = 1, that informs the daemon which is polling using sysctlbyname(...), that there is work to be done, and then in the KEXT's NKE's send function it does a while(conditional_val == 0) { the functions that causes it to sleep; } then when the dameon is finished with the work, it signals by sysctlbyname that conditional_val = 1 which causes the NKE's send function to get out of the while loop. This way is working pretty well, except after awhile of using it, it gums up and creates deadlock and also the CFUserNotificationCreate in my Daemon stops popping up windows asking for input on what to do for a connection, And finally the only thing i can do then is hold down the poser button to reboot. I know probably the while loop is very stupid and dangerous to use in the KEXT, it brings up questions like, what if the Daemon failed, then it would be in a inifinite loop forever. So any suggestions of what to do now, of how to know if the Daemon is running and also to block the send function until my dameon is done with work and then to notify the NKE. Presently my daemon when it first starts up it signals to the KEXT it is running and sets that global static variable, but if the daemon were to die, the KEXT wouldn't know. Thanks for listening to my blabbing on. Matt P.S. I really did research this but I can't find anything on it, yeah maybe I don't know how to search well, but please help. I hope I explained this as best as possible. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: dynamic_pager and context switching
by Jim Magee 09 Feb '04

09 Feb '04

Actually, the daemon is involved in compacting swap. We add and subtract swap files based on total available free space in the swap files. When we decide to remove a swap file, there may still be contents in it. If there are, the dynamic_pager daemon's thread is the one used to bring those contents back into memory and out to its new home on another swap file. But it never leaves the kernel while doing this (but it is unavailable to respond to additional swap file need/excess notifications while doing it). --Jim On Feb 9, 2004, at 12:53 PM, Mike Smith wrote: dynamic_pager just creates/deletes the swapfiles. It's not involved in the I/O path, nor does it actually compact swap. = Mike On Feb 8, 2004, at 3:44 PM, Brian Tabone wrote: I was digging through the darwin kernel lists, and did some cursory searches on developer.apple.com, but I didn't find the answer. My question is , does the kernel turn to dynamic_pager every time it swaps pages in and out, or does dynamic_pager simply manage the swap file(s) and grow/shrink them based on usage by the kernel? The reason I ask is that it would seem a large performance penalty to have to context switch out to a user-space daemon for every page in/out request. Not only would you be contesting with page swaps, but you would further be slowed down by expensive context switching from kernel to user space. Thanks in advance, -Brian T _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: dynamic_pager and context switching
by Mike Smith 09 Feb '04

09 Feb '04

dynamic_pager just creates/deletes the swapfiles. It's not involved in the I/O path, nor does it actually compact swap. = Mike On Feb 8, 2004, at 3:44 PM, Brian Tabone wrote: I was digging through the darwin kernel lists, and did some cursory searches on developer.apple.com, but I didn't find the answer. My question is , does the kernel turn to dynamic_pager every time it swaps pages in and out, or does dynamic_pager simply manage the swap file(s) and grow/shrink them based on usage by the kernel? The reason I ask is that it would seem a large performance penalty to have to context switch out to a user-space daemon for every page in/out request. Not only would you be contesting with page swaps, but you would further be slowed down by expensive context switching from kernel to user space. Thanks in advance, -Brian T _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Debugging in the field
by Sam Vaughan 09 Feb '04

09 Feb '04

Hi everyone, Just wondering what people are doing to get debugging information from testers in the field, e.g. beta sites. If they get a kernel panic then the panic log will give us a backtrace, but what about e.g. a file system deadlock? Is it possible to get a dump of a machine's kernel stacks without doing an NMI and connecting to it from gdb on another box? Any tips on existing tools or writing my own would be appreciated! Sam _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: dynamic_pager and context switching
by Brian Bergstrand 09 Feb '04

09 Feb '04

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I believe (someone correct me if I'm wrong) that dynamic_pager acts like an nfsd process. Upon launch, it immediately makes a syscall and never returns. So it's basically a process wrapper around a kernel thread that never exits the kernel. Hence, I don't think there is much context switch overhead (besides a thread switch within the kernel [stack and registers]). HTH. On Feb 8, 2004, at 5:44 PM, Brian Tabone wrote: > I was digging through the darwin kernel lists, and did some cursory > searches on developer.apple.com, but I didn't find the answer. My > question is , does the kernel turn to dynamic_pager every time it > swaps pages in and out, or does dynamic_pager simply manage the swap > file(s) and grow/shrink them based on usage by the kernel? The reason > I ask is that it would seem a large performance penalty to have to > context switch out to a user-space daemon for every page in/out > request. Not only would you be contesting with page swaps, but you > would further be slowed down by expensive context switching from > kernel to user space. > > Thanks in advance, > -Brian T > _______________________________________________ > darwin-kernel mailing list | darwin-kernel(a)lists.apple.com > Help/Unsubscribe/Archives: > http://www.lists.apple.com/mailman/listinfo/darwin-kernel > Do not post admin requests to the list. They will be ignored. > > Brian Bergstrand <http://www.bergstrand.org/brian/>, AIM: triryche206 PGP Key: <http://www.bergstrand.org/brian/misc/public_key.txt> Objects in calendars are closer than they appear. -----BEGIN PGP SIGNATURE----- Version: PGP 8.0.3 iQA/AwUBQCbIvHnR2Fu2x7aiEQKxpgCg1iMjfe1TFc6giqbWa3/EzRGIzvkAoJkp aFhjw3RL2MrvggcR7ln1181r =dmOw -----END PGP SIGNATURE----- _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0