November 2007 - Darwin-kernel

Re: pthread_cancel and cancelation points still broken in Mac OS X 10.5 Leopard?
by Amanda Walker 27 Nov '07

27 Nov '07

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com As an outside observer, I can see this clearly as a bug; it is a rather obvious omission of POSIX-defined behavior. I don't see Per as belaboring his point too much or that he is trying to get it more attention. It seems to me he is just responding to the defensiveness that you and others have exhibited about this bug, as if the bug might be dismissed as not really being a bug. --Amanda _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… On Nov 26, 2007, at 5:17 PM, Daniel Eischen wrote: Well, Per did start the thread by saying, in effect, "Apple is making a false claim that Mac OS X is UNIX 2003 compliant." Since it does pass the compliance tests, a bit of defensiveness on Apple's part does not seem unreasonable. I also raised questions about whether or not it's really a bug, though I certainly wasn't intending to be defensive--I have no vested interest in whether or not Leopard is UNIX03 compliant. I just know from sitting on both sides of the standardization table that compliance with any standard is sometimes not obvious, even when the language seems plain. Test suites exist in order to provide objective data points. If this turns out to be a bug (which, as I noted, does seem to be a reasonable interpretation in the case of file descriptors for interactive devices such as ptys/ttys), I'd describe it as a bug in the Open Group test suite as much as with Apple's implementation. Implementors rely on the accuracy and coverage of test suites. Just as there will always be cases that automated tests will miss, the certification for an implementation only means that is passes the compliance tests, not that it is somehow error free. This email sent to site_archiver(a)lists.apple.com

1 0

Re: panic crash on Leopard when assigning secondary IP to reattached interface
by Adi Masputra 27 Nov '07

27 Nov '07

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Ondrej, Please do file a radar for this; it would also be good if you can attach a skeleton version of the KEXT so that this can be easily reproduced. Thanks, Adi On Nov 23, 2007, at 8:00 AM, Ondrej Slanina wrote: Thanks, Ondrej Slanina developer .................................................................. Kerio Technologies s.r.o. Anglicke nabrezi 1, 301 49 Plzen Czech Republic http://www.kerio.com ................................................................... Connect. Communicate. Collaborate. Securely. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/adi%40apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com Hi, I'am new in Mac OS X kernel development and I'am experienced this problem: 1. I have clear OS with tun/tap interface installed. (This tun/tap is developed by me but the same behaviour occured with free tun/tap which you can find on Internet) 2. open device => attach net iface (ifnet_attach with uniqueid == MAC) 3. ifconfig "name-of-iface" 172.16.1.1-24 alias (yes, we are trying to add secondary IP to interface with no one IP) 4. everything is ok, we have iface with IP 172.16.1.1 5. close device => detach iface with iface_detach (during process off detaching we called ifnet_detach_protocol and another relevant detaching functions) 6. everything is ok, we have not iface in OS 7. open device => attach net iface (iface_attach with uniqueid == MAC [identical to MAC in second step]) 8. ifconfig "name-of-iface" 172.16.1.1-24 alias 9. panic crash This crash occures only on Leopard OS by reason of minority changes in BSD kernel (xnu-1228\bsd\netinet\in.c:1192) ifafree function is trying to dereference NULL pointer previously obtained by ifa_ifpgetprimary() Why this pointer is NULL ? The reason is in incoherence between if_addrhead (list with addresses on iface) and in_ifaddrhead (list with all(?) IP ) Another reason is reusing detached iface. When you detach iface from system, iface still remain in system waiting for another chance to live. No iface memory is released, no address memory is released. If you will call iface_attach with the same uniqueid, system will return back the same iface which you are allocated for the first time. BUT during processing of iface_attach function, if_addrhead list is initialized and filled with link layer address. This means that in this list will be only one address with family AF_LINK but no address with AF_INET family. But in in_ifaddrhead is still AF_INET address which leads to our iface. And this is incoherence mentioned above. When you call ifconfig ... alias, system try to ADD ip but not call REMOVE (as in the case of ifconfig .... wihout alias) OS will find address in in_ifaddrhead (xnu-1228\bsd\netinet\in.c:307) and will not call if_attach_ifa to add new address to if_addrhead. Next, in_ifinit() is called (xnu-1228\bsd\netinet\in.c:702) in this functionifa_ifpgetprimary() is called but this routine search in if_addrhead on iface) In this list is only one address (link layer) and no AF_INET. This means that NULL pointer is returned and panic crash will occure on next call to ifafree. My question is: Is this a bug in BSD portion of kernel and should be reported to Apple, or behaviour of my kernel module is inccorect ? Workaround is to call ifnet_attach with unique id, but in this case some leaks can occure.... P.S. I'am sorry for my bad english, I will try to improve my language skills :-) This email sent to adi(a)apple.com smime.p7s

1 0

Re: pthread_cancel and cancelation points still broken in Mac OS X 10.5 Leopard?
by mm w 27 Nov '07

27 Nov '07

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; bh=9O6+MqzHfDU4RZ2Jo+pU7S8uDGX7L9nsC4A50CDxilM=; b=U1b6GLw3oed8GvhAxudTaN5Wfqq+nZslUDCO8yRQZKp10yhrPxkVPyQsKOFwbJsE3z3egHIVvALsxbnJU7wXzXp2e3/GqTE/A6aDxKz+wX0DC36MNVmwTWnOHYHOFAGm/iBZydZV7P5D+zNVKSXWxeuS7NdRcmPvgxUrbTCChyQ= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=DSU4wy4MQybD0vq2EBMNLGvM2RWrrt2WM3iFP/IgmkfrHUDOJkhoWC3TI2a2JzZy8re2tx/S5LovlFmzo5D002Jf+DnaZfpuflf5Hj/08fG6UAgt4XfiqKLfSgDfa0w9kuKO/gSWf3VQumXAHae2dbcygD+VO25QlgElQ6704Xk= Hi Daniel, as an extern observer, i can say Terry had already answered on the path, "Thanks, Per. Looks like we will need to fix something. I'll see if I can route the radar three doors down the hall from me faster. -- Terry" 25 Nov and it is not acceptable Per was on one's case (Terry) like Terry was the Evil, even if it's a bug, it's only a bug, there is time to fix it. Hey Terry let's go do your job... provide a macosx update right away ... crazy?, i was in the same situation (about implementing a standard) we have to follow the reference tests, and sure if the test cases change..., and this kind of stuff is not simple to change right away it engages the whole OS, Terry is a really nice guy, this is the first time here that I see him anger... and i think if you take the OS in the 'globality', it's only a small small piece, you can't revoke a cert on a simple bug? the document is clear? This is not a court and Terry is not guilty of hate crime. -mmw -- echo zapydapntpd.rxltw@nzx | tr a-z@. p-za-o.@ _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: UDF driver location?
by mm w 26 Nov '07

26 Nov '07

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; bh=L90RbjVTrmFyNb3vhP4pdaAh13CjolEgxOwLEHwLewQ=; b=xsIKbDqh0WtKGe8N8lxRNbk5Rr7tZQPF7XiV4ZWMd1yE9FE8YAH+h/CJzCsTKa+KEQM0N/NlesGcjP3fGb1sfENcI0sp35E17J4+vBckLAIS8yyJenqBck5cNYz83Uu4GNF6nv4HXwp/Zshs4gMWaeWUdfNwXOMriOizNOAK1eo= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=GBbG7zmAhc1NTcbgISLxndpLORuflcoRJrA74sZquMNTzIV+M2wDs3cVDPjTCoKMjZn4AHIxg9WHsG3+1Qn4vwqBfzBh1nW7jTcOUMicPmAxRAerzQ2Y13DrJqQBP6ei7SLEn1TOAI385a25H2LNnvwymVlIqMFpAje7EOQUHyk= Hi, the UDF VFS plugin is not open-source (never ever) On Nov 26, 2007 3:18 PM, Paul Wagland <pwapple(a)kungfoocoder.org> wrote: > Hi all, > > Quick question. I have downloaded the xnu source from > http://www.opensource.apple.com/darwinsource/10.5/, but it does not > appear to include the UDF driver. It does include the iso 9660 driver, > but I am most interested in the UDF driver. Does anyone know which > package this might be in? > > Thanks in advance for any help that can be offered, > Paul > > _______________________________________________ > Do not post admin requests to the list. They will be ignored. > Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) > Help/Unsubscribe/Update your Subscription: > http://lists.apple.com/mailman/options/darwin-kernel/openspecies%40gmail.com > > This email sent to openspecies(a)gmail.com > -- echo zapydapntpd.rxltw@nzx | tr a-z@. p-za-o.@ _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

UDF driver location?
by Paul Wagland 26 Nov '07

26 Nov '07

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com User-agent: Mutt/1.5.17 (2007-11-01) Hi all, Quick question. I have downloaded the xnu source from http://www.opensource.apple.com/darwinsource/10.5/, but it does not appear to include the UDF driver. It does include the iso 9660 driver, but I am most interested in the UDF driver. Does anyone know which package this might be in? Thanks in advance for any help that can be offered, Paul _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: pthread_cancel and cancelation points still broken in Mac OS X 10.5 Leopard?
by Daniel Eischen 26 Nov '07

26 Nov '07

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On Mon, 26 Nov 2007, Terry Lambert wrote: Part of the quote was missing: <quote> Asynchronous cancelability is explicitly NOT required to allow a cancel to interrupt a thread blocked at a cancellation point in the "shall occur" table. That would defeat the entire purpose of cancellation. </quote> As an outside observer, I can see this clearly as a bug; it is a rather obvious omission of POSIX-defined behavior. I don't see Per as belaboring his point too much or that he is trying to get it more attention. It seems to me he is just responding to the defensiveness that you and others have exhibited about this bug, as if the bug might be dismissed as not really being a bug. I am sometimes guilty of having this attitude myself, but it is much easier to observe in others :-) At any rate, I hope you all had a good vacation - I'm sure it was well-deserved. -- DE _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… That interpretation of async vs. deferred renders async practically useless, which I doubt is the intent. As to when things have to be fixed or not, that's lawyer and release schedule territory, way above my pay grade, and we do not discuss release schedules here, ever. Certification from my perspective has to do with passing the requisite tests. If a test failure happens because a new test is added, we will in general fix it or request a PIN (like both we and Tru64 UNIX got for AST based signals already) ASAP. Belaboring your point here isn't winning your bug any more attention than it's already getting. Give us a break, we are all just getting back from vacation. This email sent to site_archiver(a)lists.apple.com

1 0

Re: pthread_cancel and cancelation points still broken in Mac OS X 10.5 Leopard?
by Terry Lambert 26 Nov '07

26 Nov '07

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Thanks, -- Terry On Nov 26, 2007, at 12:27 PM, Per Mildner <Per.Mildner(a)sics.se> wrote: <quote> Mac OS X is just broken. 10.5 apparently passed the VSTH suite, which simply means that the test suite has missed a few more relatively obvious test cases. </quote> "In particular, no test suite can detect every possible way that an implementation can be non-conforming." <quote> I would seem to me that Apple should correct the problem, once reported, by its next Open Brand Certificate renewal date, which si 18 May 2008.[sic] </quote> <quote> Apple has an obligation to correct any reported non-conformities "within the prescribed timescale", regardless of what may not be detected by a test suite. (See http://www.opengroup.org/openbrand/ .)" </quote> And, quoting from <http://www.opengroup.org/openbrand/>: "If ... the non-conformance is not corrected in the predefined timescale, then the Product Registration, and the ability to use the trademark, will be lost." <quote> It would, in fact, be INCORRECT to call read(), or most other POSIX functions, with asynchronous cancel enabled. Regards, _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… That interpretation of async vs. deferred renders async practically useless, which I doubt is the intent. As to when things have to be fixed or not, that's lawyer and release schedule territory, way above my pay grade, and we do not discuss release schedules here, ever. Certification from my perspective has to do with passing the requisite tests. If a test failure happens because a new test is added, we will in general fix it or request a PIN (like both we and Tru64 UNIX got for AST based signals already) ASAP. Belaboring your point here isn't winning your bug any more attention than it's already getting. Give us a break, we are all just getting back from vacation. I just added the following information to my bug report (misspellings and all...): I brought this up on the austin-group-l mailing list (Nov 23 2007, Item 11231 "A question about cancellation points" on https://www.opengroup.org/sophocles/show_archive.tpl?listname=austin-group-l ). Presumably the Austin list members have more experience than most in judging in matters related to SUSv3 conformance. The consensus seems to be that the current Mac OS 10.5 behavior does not conform to SUSvw3 and that: The thread also points out that the behavior with PTHREAD_CANCEL_ASYNCHRONOUS is irrelevant since: >From POSIX, "The pthread_cancel( ), pthread_setcancelstate( ), and pthread_setcanceltype( ) functions are defined to be async-cancel safe. No other functions in this volume of POSIX. 1-200x are required to be async-cancel-safe." </quote> This email sent to site_archiver(a)lists.apple.com

1 0

Re: pthread_cancel and cancelation points still broken in Mac OS X 10.5 Leopard?
by Per Mildner 26 Nov '07

26 Nov '07

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com <quote> Mac OS X is just broken. 10.5 apparently passed the VSTH suite, which simply means that the test suite has missed a few more relatively obvious test cases. </quote> "In particular, no test suite can detect every possible way that an implementation can be non-conforming." <quote> I would seem to me that Apple should correct the problem, once reported, by its next Open Brand Certificate renewal date, which si 18 May 2008.[sic] </quote> <quote> Apple has an obligation to correct any reported non-conformities "within the prescribed timescale", regardless of what may not be detected by a test suite. (See http://www.opengroup.org/openbrand/ .)" </quote> And, quoting from <http://www.opengroup.org/openbrand/>: "If ... the non-conformance is not corrected in the predefined timescale, then the Product Registration, and the ability to use the trademark, will be lost." <quote> It would, in fact, be INCORRECT to call read(), or most other POSIX functions, with asynchronous cancel enabled. From POSIX, "The pthread_cancel( ), pthread_setcancelstate( ), and pthread_setcanceltype( ) functions are defined to be async-cancel safe. No other functions in this volume of POSIX.1-200x are required to be async-cancel-safe." </quote> Regards, _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… I just added the following information to my bug report (misspellings and all...): I brought this up on the austin-group-l mailing list (Nov 23 2007, Item 11231 "A question about cancellation points" on https://www.opengroup.org/sophocles/show_archive.tpl?listname=austin-group-l) . Presumably the Austin list members have more experience than most in judging in matters related to SUSv3 conformance. The consensus seems to be that the current Mac OS 10.5 behavior does not conform to SUSvw3 and that: The thread also points out that the behavior with PTHREAD_CANCEL_ASYNCHRONOUS is irrelevant since: This email sent to site_archiver(a)lists.apple.com

1 0

Re: MTRRs and PATs
by Victor Vedovato 26 Nov '07

26 Nov '07

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Victor On Nov 26, 2007, at 8:57 AM, Michael Smith wrote: On Nov 25, 2007, at 11:29 PM, Victor Vedovato wrote: = Mike _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… On x86, I believe the MTRR/PAT registers are the final say (over MMU PTEs) as to the read/write/allocation/eviction policy for CPU caches on system memory. We're concerned about these since we need to coordinate the CPU cache policy with GPU cache policy - flushing is a really bad strategy to mitigate these issues. We need a holistic view of CPU, GPU, System and GPU memory caching policy - reggie_se is a good start for this. Derek - thanks for the excellent response. We can use reggie_se to set/reset the registers for debugging. It's not clear from others that OS X uses the MTRRs/PATs for cache control. Can you confirm that? Thanks again. If you were to describe what you meant by "cache control", you might be able to get a clearer answer. This email sent to site_archiver(a)lists.apple.com

1 0

Re: MTRRs and PATs
by Derek Kumar 26 Nov '07

26 Nov '07

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Victor, Derek Victor Vedovato wrote: Victor On Nov 20, 2007, at 3:17 PM, Derek Kumar wrote: _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… It's not clear from others that OS X uses the MTRRs/PATs for cache control. Can you confirm that? Technically, cacheability attributes for all virtual->physical address translations are governed by the PAT mechanism. The "classical" PCD and PWT bits in a pagetable entry, in combination with the PAT bit in the pagetable entry form a "mux" which selects a field in the IA32_CR_PAT MSR. For compatibility, the power-on value of that MSR is such that if the PCD (page cache disable) bit is lit in the PTE, the page is marked non-cacheable and so on. As I described previously, the only PAT MSR field that the kernel reprograms currently (Tiger/ Leopard) is the PA6 field, which is set to write combine. Thus, if you use say IOSetProcessorCacheMode() to request write combining for a particular virtual page (perhaps mapped to a framebuffer), the PCD and PAT bits are set in the PTE, thus selecting the PA6 field within the IA32_CR_PAT MSR, which in turn is programmed for write combining. If you specify kIOMapInhibitCache, the PCD bit is set in the PTE. (There are some currently unresolved issues such as X's lack of support for the strong uncacheable attribute, but that's not immediately relevant) Currently, a few MTRR ranges may be programmed by firmware (EFI) prior to control transfer to the kernel, and there's a set of legacy exports (mtrr_range_add/remove) which can be used to alter the cacheability attributes for a physical range. I'm unaware of any current use of these exports--the IOKit APIs do not alter MTRRs at present. It would help if you clarified the nature of the problem. I understand if you cannot disclose why you need this information, but if your ultimate objective is to accurately discover the cacheability attributes of a virtual or physical range from first principles (perhaps to debug an issue), you will have to discover the union of the MTRR and pagetable based cache attributes for that range. You can identify the MTRRs covering the range in question using the tools I described in my previous message. To discover the pagetable based cacheability attributes, you'd have to identify the set of all pagetables which contain a mapping to the physical address in question (unless you're confident that it can only be a kernel mapping, in which case you can restrict yourself to the kernel pagetable). You'd then examine the relevant pagetable entries to discover the cache attribute type for each mapping. Note that specifying disparate cacheability attributes for multiple mappings to the same physical address is illegal, and can lead to hard to debug problems. If the MTRR and PAT ranges overlap, there are rules for resolving conflicts. Contact me if you need to do go down this route, there are internal tools that can help. Derek - thanks for the excellent response. We can use reggie_se to set/reset the registers for debugging. It's not clear from others that OS X uses the MTRRs/PATs for cache control. Can you confirm that? Thanks again. reggie_se -r -a 0x277 -B 64 This email sent to site_archiver(a)lists.apple.com

1 0