August 2003 - Darwin-kernel

Re: Q: Darwin panicing too soon?
by Stéphane Sudre 06 Aug '03

06 Aug '03

Le mercredi, 6 ao{ 2003, ` 16:12 Europe/Paris, Jim Magee a icrit : On Wednesday, August 6, 2003, at 7:04 AM, Jonas Maebe wrote: On woensdag, aug 6, 2003, at 12:50 Europe/Brussels, Stiphane Sudre wrote: } else if (retval != KERN_RESOURCE_SHORTAGE) { /* would like to cause a zone_gc() */ (1) panic("zalloc"); } ---- Q: Instead of panicing at (1), couldn't it just be possible to return NULL? KERN_RESOURCE_SHORTAGE is described as: /* A system resource could not be allocated to fulfill * this request. This failure may not be permanent. */ The issue is that once you "Kernel Panic"ed, it's 'permanent' till the next reboot. Note that it panics if retval is *different* from KERN_RESOURCE_SHORTAGE". So in case of KERN_RESOURCE_SHORTAGE, whatever comes below will be executed. Correct. The issue is that most kernel code expects zalloc() to wait until memory is available. They only expect a NULL return in the cases that they specified that they are unwilling to block for memory allocation. However, one of the "different from KERN_RESOURCE_SHORTAGE" errors actually occurs as a result of the kernel's virtual address space being exhausted (rather than just being short of free page frames at the moment). It is extremely unlikely that this "exhausted virtual memory space" situation will rectify itself by having the current thread wait. So we panic. This is somewhat analogous to malloc(). It will normally wait for memory to become available (including waiting for the kernel to select and page out other application pages, etc...). But if the virtual address space of your application is completely exhausted, there's not much that can be done. The good news is that the Darwin 10.7 (Panther) PowerPC[1] kernel supports a significantly larger kernel virtual address space. So the odds of hitting that panic go WAY down (most existing machines can't handle enough physical memory to trigger it). There are also additional hand and soft resource limits added to keep applications from inducing run-away kernel allocations. Side question: Is there a way or a tool to monitor kernel memory (free) space? _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Q: Darwin panicing too soon?
by Jonas Maebe 06 Aug '03

06 Aug '03

On woensdag, aug 6, 2003, at 16:12 Europe/Brussels, Jim Magee wrote: [1]. The i386 kernel still maps the kernel as part of each task, so it's kernel virtual space is still much smaller (and the application's available virtual space is also smaller). The PowerPC kernel has separate 4GB address spaces for the kernel vs. each application. Are there more juicy tidbits like this you can share with the list? The Unix section on Panther at apple.com isn't exactly all-telling :) Also, the WWDC sessions at connect.apple.com (which I can't watch yet as student member, but I can see the titles already) on the Core OS don't seem to include any kernel architectural improvement sessions. And is there maybe a technote about this kind of stuff planned? (a bit like <http://developer.apple.com/technotes/tn2002/tn2053.html> on Mac OS X 10.2) Jonas _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Q: Darwin panicing too soon?
by Jim Magee 06 Aug '03

06 Aug '03

On Wednesday, August 6, 2003, at 7:04 AM, Jonas Maebe wrote: On woensdag, aug 6, 2003, at 12:50 Europe/Brussels, Stiphane Sudre wrote: } else if (retval != KERN_RESOURCE_SHORTAGE) { /* would like to cause a zone_gc() */ (1) panic("zalloc"); } ---- Q: Instead of panicing at (1), couldn't it just be possible to return NULL? KERN_RESOURCE_SHORTAGE is described as: /* A system resource could not be allocated to fulfill * this request. This failure may not be permanent. */ The issue is that once you "Kernel Panic"ed, it's 'permanent' till the next reboot. Note that it panics if retval is *different* from KERN_RESOURCE_SHORTAGE". So in case of KERN_RESOURCE_SHORTAGE, whatever comes below will be executed. Correct. The issue is that most kernel code expects zalloc() to wait until memory is available. They only expect a NULL return in the cases that they specified that they are unwilling to block for memory allocation. However, one of the "different from KERN_RESOURCE_SHORTAGE" errors actually occurs as a result of the kernel's virtual address space being exhausted (rather than just being short of free page frames at the moment). It is extremely unlikely that this "exhausted virtual memory space" situation will rectify itself by having the current thread wait. So we panic. This is somewhat analogous to malloc(). It will normally wait for memory to become available (including waiting for the kernel to select and page out other application pages, etc...). But if the virtual address space of your application is completely exhausted, there's not much that can be done. The good news is that the Darwin 10.7 (Panther) PowerPC[1] kernel supports a significantly larger kernel virtual address space. So the odds of hitting that panic go WAY down (most existing machines can't handle enough physical memory to trigger it). There are also additional hand and soft resource limits added to keep applications from inducing run-away kernel allocations. --Jim [1]. The i386 kernel still maps the kernel as part of each task, so it's kernel virtual space is still much smaller (and the application's available virtual space is also smaller). The PowerPC kernel has separate 4GB address spaces for the kernel vs. each application. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Q: Darwin panicing too soon?
by Stéphane Sudre 06 Aug '03

06 Aug '03

On mercredi, ao{ 6, 2003, at 13:04 Europe/Paris, Jonas Maebe wrote: On woensdag, aug 6, 2003, at 12:50 Europe/Brussels, Stiphane Sudre wrote: } else if (retval != KERN_RESOURCE_SHORTAGE) { /* would like to cause a zone_gc() */ (1) panic("zalloc"); } ---- Q: Instead of panicing at (1), couldn't it just be possible to return NULL? KERN_RESOURCE_SHORTAGE is described as: /* A system resource could not be allocated to fulfill * this request. This failure may not be permanent. */ The issue is that once you "Kernel Panic"ed, it's 'permanent' till the next reboot. Note that it panics if retval is *different* from KERN_RESOURCE_SHORTAGE". So in case of KERN_RESOURCE_SHORTAGE, whatever comes below will be executed. Correct. I thought it was a == . This explains all. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Q: Darwin panicing too soon?
by Jonas Maebe 06 Aug '03

06 Aug '03

On woensdag, aug 6, 2003, at 12:50 Europe/Brussels, Stiphane Sudre wrote: } else if (retval != KERN_RESOURCE_SHORTAGE) { /* would like to cause a zone_gc() */ (1) panic("zalloc"); } ---- Q: Instead of panicing at (1), couldn't it just be possible to return NULL? KERN_RESOURCE_SHORTAGE is described as: /* A system resource could not be allocated to fulfill * this request. This failure may not be permanent. */ The issue is that once you "Kernel Panic"ed, it's 'permanent' till the next reboot. Note that it panics if retval is *different* from KERN_RESOURCE_SHORTAGE". So in case of KERN_RESOURCE_SHORTAGE, whatever comes below will be executed. Jonas _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Q: Darwin panicing too soon?
by Stéphane Sudre 06 Aug '03

06 Aug '03

In zalloc.c at line 662, this code can be found: if (zone->collectable) { vm_offset_t space; vm_size_t alloc_size; if (vm_pool_low()) alloc_size = round_page(zone->elem_size); else alloc_size = zone->alloc_size; retval = kernel_memory_allocate(zone_map, &space, alloc_size, 0, KMA_KOBJECT|KMA_NOPAGEWAIT); if (retval == KERN_SUCCESS) { zone_page_init(space, alloc_size, ZONE_PAGE_USED); zcram(zone, space, alloc_size); } else if (retval != KERN_RESOURCE_SHORTAGE) { /* would like to cause a zone_gc() */ (1) panic("zalloc"); } [...] ---- Q: Instead of panicing at (1), couldn't it just be possible to return NULL? KERN_RESOURCE_SHORTAGE is described as: /* A system resource could not be allocated to fulfill * this request. This failure may not be permanent. */ The issue is that once you "Kernel Panic"ed, it's 'permanent' till the next reboot. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: system read blocks.
by Quinn 05 Aug '03

05 Aug '03

At 10:57 +0530 5/8/03, Kaza Sudheer wrote: I had created Named Pipes(FIFO's) "Xid" in process A and that is shared to B thr STDIN( 0 ). Process A and B communicates thr named pipes"Xid" Process "A " writes on "Xid" and process "B" reads from "Xid". My probelm is Lets say if Process "A" is terminated, still Process B waits on named pipe"Xid". It blocks on read system call. How to over come this probelm. This list is for kernel-level development, such as device drivers, file system plug-ins, NKEs, and so on. Your question is about user space programming, that is, using the standard UNIX APIs. The folks on this list aren't really your best for answering your questions. You may be better off on the darwin-development mailing list. <http://www.lists.apple.com/mailman/listinfo/darwin-development> Beyond that, I highly recommend the book "Advanced Programming in the UNIX Enivronment", by Stevens,, which covers these sorts of topics in great detail. S+E -- Quinn "The Eskimo!" <http://www.apple.com/developer/> Apple Developer Technical Support * Networking, Communications, Hardware _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: mbuf question
by Stéphane Sudre 05 Aug '03

05 Aug '03

On mardi, ao{ 5, 2003, at 17:24 Europe/Paris, Justin Walker wrote: On Tuesday, Aug 5, 2003, at 10:06 America/Chicago, Stiphane Sudre wrote: Q: Can a non PacketHeader mbuf use an external storage? i.e. can the M_EXT flag be set on a mbuf with no M_PKTHDR flag? I had a look at the book "The Design and Implementation of the 4.4 BSD OS" but did not find an answer to this. The theoretical answer is 'yes'. Mbuf processing should deal with each mbuf individually, without regard to the chain its in. It is possible that some code doesn't obey this rule, but I don't know of any off-hand. It's easy to screw this up, so the possibility can't be ruled out, but it should work. Are you having problems, or just checking? I'm checking to see if I can have problems :-) As I don't think mbuf is the acronym for My Best Unix Friend, I'm trying to avoid potential problems due to false assumptions. I had some code which was making the assumption that an outgoing TCP/UDP packet could not be made of mbuf with an external storage. I removed this assumption, and now I'm trying to find if this second assumption is just an assumption or an axiom. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: mbuf question
by Justin Walker 05 Aug '03

05 Aug '03

On Tuesday, Aug 5, 2003, at 10:06 America/Chicago, Stiphane Sudre wrote: Q: Can a non PacketHeader mbuf use an external storage? i.e. can the M_EXT flag be set on a mbuf with no M_PKTHDR flag? I had a look at the book "The Design and Implementation of the 4.4 BSD OS" but did not find an answer to this. The theoretical answer is 'yes'. Mbuf processing should deal with each mbuf individually, without regard to the chain its in. It is possible that some code doesn't obey this rule, but I don't know of any off-hand. It's easy to screw this up, so the possibility can't be ruled out, but it should work. Are you having problems, or just checking? Regards, Justin -- Justin C. Walker, Curmudgeon-At-Large * Institute for General Semantics | Men are from Earth. | Women are from Earth. | Deal with it. *--------------------------------------*-------------------------------* _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

mbuf question
by Stéphane Sudre 05 Aug '03

05 Aug '03

Q: Can a non PacketHeader mbuf use an external storage? i.e. can the M_EXT flag be set on a mbuf with no M_PKTHDR flag? I had a look at the book "The Design and Implementation of the 4.4 BSD OS" but did not find an answer to this. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0