February 2006 - Darwin-kernel

Re: KPI Interface filter and filter detachment
by Stephane Sudre 03 Feb '06

03 Feb '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com OK, so this would not explain a recursive lock attempt. On 2 févr. 06, at 20:29, Josh Graessley wrote: -josh On Feb 2, 2006, at 9:36 AM, Stephane Sudre wrote: Maybe someone can shed some light on this. the callback can be sum-up to: myCallBack { mutex lock do something but never return anything mutex unlock return a value } Question: _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… Mmm, is it possible for a kernel control message to be received when you are within a iff_input callback? I would say yes. But would it be considered the same "thread" or not? I believe the detach is a special case. The detach is there to notify you that your filter has been detached. In this case, that is the last call you will receive and it should only happen when we are not calling any other entry points in your filter for that interface. Let's say a KPI Interface filter is attached to every available interfaces. Let's say this KPI Interface filter is implementing a iff_input callback. now at one point an interface is going away => a detach filter is called. can the detach filter (iflt_detach) be called while we are in myCallBack? _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/ jgraessley%40apple.com This email sent to jgraessley(a)apple.com This email sent to site_archiver(a)lists.apple.com

1 0

IOSyncer
by Herbert 03 Feb '06

03 Feb '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Is anyone using this? So I'm leaking and I hate that ;) Any help? _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… I'm trying but it's apparently not being released when I'm done with it, even though the header seems to indicate that it should auto- release. This email sent to site_archiver(a)lists.apple.com

1 0

kmem_alloc replacement
by Herbert 03 Feb '06

03 Feb '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Thanks, Quinn! Now I can do my kernel-level wavelet encryption in peace... _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… After studying Quinn's diagram, I figured that IOMallocAligned is the one to use if you need big chunks of wired memory. This email sent to site_archiver(a)lists.apple.com

1 0

Re: KPI Interface filter and filter detachment
by Josh Graessley 02 Feb '06

02 Feb '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com -josh On Feb 2, 2006, at 9:36 AM, Stephane Sudre wrote: Maybe someone can shed some light on this. the callback can be sum-up to: myCallBack { mutex lock do something but never return anything mutex unlock return a value } Question: _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com I believe the detach is a special case. The detach is there to notify you that your filter has been detached. In this case, that is the last call you will receive and it should only happen when we are not calling any other entry points in your filter for that interface. Let's say a KPI Interface filter is attached to every available interfaces. Let's say this KPI Interface filter is implementing a iff_input callback. now at one point an interface is going away => a detach filter is called. can the detach filter (iflt_detach) be called while we are in myCallBack? _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/jgraessley% 40apple.com This email sent to jgraessley(a)apple.com smime.p7s

1 0

Re: kmem_alloc replacement in Tiger
by Mike Smith 02 Feb '06

02 Feb '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On Feb 2, 2006, at 5:26 AM, Andrew Gallatin wrote: So I'm worried that on a DART-less system, there is no way to get a DMA address for a physical address over 4GB that does not involve bounce pages. I'm hoping that I'm wrong (like the bouncer is a gIOSystemMapper; that would only waste memory..). Drew, = Mike _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… Sadly, any concrete discussion on this subject involves speculation about future products; unless Dean Reece has something to say about a possible '64-bit I/O Kit' it's going to have to sit and wait for a bit. This email sent to site_archiver(a)lists.apple.com

1 0

KPI Interface filter and filter detachment
by Stephane Sudre 02 Feb '06

02 Feb '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Maybe someone can shed some light on this. the callback can be sum-up to: myCallBack { mutex lock do something but never return anything mutex unlock return a value } Question: _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… Let's say a KPI Interface filter is attached to every available interfaces. Let's say this KPI Interface filter is implementing a iff_input callback. now at one point an interface is going away => a detach filter is called. can the detach filter (iflt_detach) be called while we are in myCallBack? This email sent to site_archiver(a)lists.apple.com

1 0

Re: kmem_alloc replacement in Tiger
by Andrew Gallatin 02 Feb '06

02 Feb '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Terry Lambert writes: > On Feb 1, 2006, at 3:58 PM, Andrew Gallatin wrote: > > > Terry Lambert writes: > >> [...] > > > > You've confused me. What 32-bit value? I wasn't asking about how > > to handle a 64-bit user virtual address. We already do this. > > It sounded like you were trying to use a 32 bit physical address... No, I'm complaining a 32-bit physical address (may be) all that IOKit will give me, even though I'd love to get a 64-bit physical address. See below. > > My concern was with avoiding bouncing a 64-bit physical address which > > backs a 32-bit kernel virtual address (or a 32- or 64-bit user virtual > > address). > > This should not be a problem, and should not require bouncing, unless > there's a hardware limitation. The kernel itself only really deals > with virtual addresses. Physical addresses are handled by the pmap > layer. In practice, it's impossible to have a machine where you are > using memory that has a pmap entry referring to it that can't refer to > it because it didn't have enough bits to establish the mapping in the > first place. The attempt to map it would have failed. I don't argue with this. The problem is that there is no way for a driver to call down to the pmap layer. I must go through IOKit. And IOKit considers a physical address to be 32-bit: typedef UInt32 IOPhysicalAddress; and any supported way to get a physical address seems to wind its way through a 32-bit varible at some point. There does seem to be a 64-bit IOMemoryDescriptor::getPhysicalSegment64 (see xnu-792.6.56/iokit/Kernel/IOMemoryDescriptor.cpp). However it starts by getting a 32-bit 'physical' address. If there is a gIOSystemMapper (a DART I would assume), I think it finds the 64-bit physical address by doing a lookup in the IOMMU based on the 32-bit physical address. If there is no gIOSystemMapper (like on a G4 or an Intel box), it just casts the 32-bit address to 64 bits. So I'm worried that on a DART-less system, there is no way to get a DMA address for a physical address over 4GB that does not involve bounce pages. I'm hoping that I'm wrong (like the bouncer is a gIOSystemMapper; that would only waste memory..). Drew _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: kernel dump server
by Sam Vaughan 02 Feb '06

02 Feb '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On 02/02/2006, at 4:13 PM, Derek Kumar wrote: <http://oss.sgi.com/projects/lkcd/> Sam _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… We've also added a "stack snapshot" facility, implemented as a system call, which will gather a coherent stack trace for every thread in the system (user and kernel stacks). Internally, this is being used to sample the state of the system when the spinny cursor appears for too long, and collate that data in an automated fashion for later analysis. This is actually present (sans any documentation whatsoever) in the current x86 kernel, but a more general release and documented interface probably won't be available until Leopard. I don't know if it will be very useful to external developers, but I can see it being used to track down inexplicable latencies and so on when performance tuning. I think it would be quite useful to all third party kernel developers who spawn multiple threads. If you could expose an API to it then it would be a great first step towards a Mac equivalent of icrash and lcrash. This is why it's been so great to see Apple's enthusiasm with the continual development of the CHUD tools. The better the debugging tools the more stable the platform gets, which is especially important as Apple's third party kernel development community grows. This email sent to site_archiver(a)lists.apple.com

1 0

Re: kernel dump server
by Derek Kumar 02 Feb '06

02 Feb '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On Feb 1, 2006, at 7:46 PM, Sam Vaughan wrote: On 02/02/2006, at 1:30 AM, Derek Kumar <drk(a)apple.com> wrote: Something else that I wanted to note was that a feature that was asked for last WWDC, the ability to examine the live, running kernel on the same machine, has been in use internally for quite some time, and should be made available externally in a forthcoming tools release. Tremendous news, Derek! I remember asking for KPI support for walking kernel threads live at WWDC '04. I'd love to see an Apple- supported tool made available to do it. I haven't had the time to port my own 10.3 implementation to 10.4 and I really miss it when I need to look for deadlocks or stuck threads, especially on remote test machines in far off lands. Remote debugging isn't always feasible and I've never found Shark quite up to the task. We've also added a "stack snapshot" facility, implemented as a system call, which will gather a coherent stack trace for every thread in the system (user and kernel stacks). Internally, this is being used to sample the state of the system when the spinny cursor appears for too long, and collate that data in an automated fashion for later analysis. This is actually present (sans any documentation whatsoever) in the current x86 kernel, but a more general release and documented interface probably won't be available until Leopard. I don't know if it will be very useful to external developers, but I can see it being used to track down inexplicable latencies and so on when performance tuning. Derek _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Re(5): kernel dump server
by Derek Kumar 02 Feb '06

02 Feb '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com 1. isolate the network prefix used by the interface. IMPLEMENTATION ... (since the lack of ARP resolution will be addressed in a future update This is actually the "biggie" for us, and I suspect other developers too. If you have a full-blown network with routers then you can put the dump server on another network and things "just work". But for those of us with smaller networks (even large-ish companies with fairly simple networks) this is a killer. It makes kernel-dump entirely useless, so please can we have it in a "sooner" update. I must note that this generalization is something of an exaggeration. Section 5.2.4.2 of RFC 1812 ("Requirements for IP Version 4 Routers") appears to require compliant implementations to forward packets to the appropriate attached interface, regardless of the origin of the packet. Here's an excerpt describing the forwarding algorithm: 5.2.4.2 Local/Remote Decision After it has been determined that the IP packet needs to be forwarded according to the rules specified in Section [5.2.3], the following algorithm MUST be used to determine if the Immediate Destination is directly accessible (see [INTERNET:2]). ... # If no network interface has been selected in the first step, for each IP address assigned to the router: The result of this operation will usually have been computed and saved during initialization. 2. Isolate the corresponding set of bits from the IP Destination Address of the packet. 3. Compare the resulting network prefixes. If they are equal to each other, the packet can be transmitted through the corresponding network interface. Empirically, on the (RFC 1918 private) network that I'm composing this message on, where the router is a garden variety cable modem, crashdumps work fine when both source and destination are on the same network. The same is true of the routers on the Apple and MIT campuses. Additionally, I've helped engineers at external installations on multiple occasions in the past (Cisco and Oracle spring to mind) where crashdumps have been similarly configured. I don't believe it is completely unreasonable for a minimalist implementation to rely on this behavior. That being said, it is both inefficient and problematic in certain scenarios (such as yours) to direct packets exclusively to the router. Also, if there is no router present (as on a test network), you may have to use the workaround of setting the crashdump server as your gateway, as has been noted on this list in the past. I can also appreciate that there are many instances where it is not feasible to change router behavior (I believe this was the case at your installation) and, as I've noted previously, this is something we intend to address in an update as soon as possible. Derek _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0