October 2006 - Darwin-dev - lists.apple.com

Re: bringing pf (4) to OS X via NKE
by Ryan McGann 31 Oct '06

31 Oct '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com I'm posting my intention to port pf (4) (http://www.freebsd.org/cgi/ man.cgi?query=pf&sektion=4) to an NKE for use as a replacement or complement to the current ipfw2. According to the Network Kernel Extensions Programming Guide (http:// developer.apple.com/documentation/Darwin/Conceptual/NKEConceptual/ index.html) it seems I might use an Interface Filter KPI mechanism to accomplish such a task. So I ask those who are more familiar with NKEs, is this a reasonable task, am I sane to try it, and do you have any words of advice? Ryan _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… I can tell you from experience you're going to have a hard time doing things at the IP filter layer. IPFW is kind of hacked in the Mac OS X stack between the interface layer and the IP layer. Therefore it can do things that are not possible at either layer alone. My experience comes from trying to write a similar replacement for IPFW. I'm not familiar with PF per-se but I imagine our requirements were pretty close to what PF and IPFW provide. - An IP filter only receives unfragmented packets. You cannot filter on unfragmented packets. - An IP filter receives outgoing packets before they have been routed, so you won't be able to filter based on the interface. And sometimes the mbuf_pkthdr_rcvif wasn't set for incoming packets either. (That might have been a bug.) - An IP filter receives packets after they have been "filtered" out by the IP stack, which is both good and bad. If you want to filter those packets yourself, then you're SOL. If you don't care then it's a good thing because checking for those packets can be tedius. - An interface filter receives fragmented packets and you can filter on the interface. However an interface filter is harder to write correctly/safely, especially for IPv6 (finding the protocol headers in IPv6 was not very fun). In summary, to completely emulate IPFW I concluded I'd need an interface filter, not an IP filter. But if you're OK with not implementing some parts of pf's interface, or if pf doesn't support those tests, then use an IP filter. It's easy to switch between the two implementations, btw. Just write the entrypoints to your packet filtering so that it takes a pointer to the packet start and a pointer to the protocol header. This was all back in 10.4's early days as proof-of-concept. I never finished it because we moved on to other projects. I might have been running into bugs in 10.4.0, but I know some of these are not bugs but are limitations. Somebody from Apple might be able to comment more. This email sent to site_archiver(a)lists.apple.com

1 0

Possible issues with PowerMac12_1PlatformPlugin
by Benjamin Herrenschmidt 31 Oct '06

31 Oct '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Hi folks ! In case anybody on this list is interested in picking this up (I don't own the machine and have no direct interest in seeing that fixed, I just felt it might be of some interest to some apple folks so I'm posting it here). While looking at the various thermal control loops for PowerMac12,1, I've found what might be bugs in there. I'm not 100% certain as the code is fairly intricated but here they are anyway. It's based on AppleMacRISC4PE-185.0.0 source code which is the latest version of this module available in the Apple public source repository. Note that all of these might not be bugs, they might be done on purpose, though that would be nasty... * PowerMac12_1_SystemFansCtrlLoop.cpp::calculateNewTarget(), unlike the usual versions of that code, applies the PID values as an offset to the value -fetched- back from the target control, it's even commented with a big fat "IMPORTANT NOTE" in the code. The problem is that the target control is a PowerMac12_1VirtualControl.cpp, whose implementation of getTargetValue() does pretty much nothing but call it's superclass, which will get the target value from the "target-value" of the dictionary. However, as the name indicates, PowerMac12_1VirtualControl is a "virtual" control, which sits between a real HW control and one or many loops, doing all sort of things irrelevant to this discussion... except for one... when setting a new target value, it sets it to the target real control... but never to itself as far as I can see. Thus it will never update the device-tree value that getTargetValue() fetches. It never calls super::setTargetValue() that is. And I haven't found anybody calling sendTargetValue() for it neither. In fact, PowerMac12_1_SystemFansCtrlLoop::sendNewTarget, _unlike_ other PID control loops, specifically does _not_ call sendTargetValue() on the target control. That means that PowerMac12_1_SystemFansCtrlLoop always apply the PID value as an offset on top of something that is never updated nor initialized in the plist as far as I can see. * The exact same thing happens with PowerMac12_1_CPUFanCtrlLoop in the sense that it also uses getTargetValue() as a reference and never calls sendTargetValue() while the target is a virtual control, so the reference is a never updated (and never initialized in the plist) value, thus I suspect 0 (I haven't verified, no HW at hand). * That one might be on purpose, but I find it a bit weird. In PowerMac12_1VirtualControl::setTargetValue, in the case where multiple control loops are attached. The code will "remember" for each control loop the value & reference and tick a "control loop was here" flag. Then, it checks if all loop have been here, and if yes, calculate a new real value for the target control and clears all the "was here" flags. That means that basically, it will only send a new value to the target after all control loops have set at least a new value once. Which means that the target control will be updated at the lowest common denominator of the update rates of all control loops. In practice, according to the plist, this is actually how it's wired with 2 loops: one is ticking at 1 seconds interval and the other at 5. I just find this weird... having a loop ticking every secodn (thus supposed to have fast reactivity) but defeated by the fact that the target control will only be updated based on the slowest of the loops (every 5 seconds). Of course, there will still be the metastates to make sure we don't overtemp, but it looks to me like the code is defeating the purpose of having one loop tick faster than the other... That's it for now :) It might also be that the public version of that code isn't what is actually shipped in 10.4.8 too, since darwin latest update of that module source was around 10.4.4 I think... Somebody feel free to file a RADAR for it, or just ignore it, as you wish. Cheers, Ben. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Directory Services only returns 1000 users from Active Directory/All Domains/Users
by Paul Nelson 30 Oct '06

30 Oct '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Thread-index: Acb8bZZO1N31wGhgEduyJQAWy4i6sA== Thread-topic: Directory Services only returns 1000 users from Active Directory/All Domains/Users User-agent: Microsoft-Entourage/11.2.5.060620 You can write your own LDAP code to query AD instead of using the plug-in. Then implement the page controls - it is not very difficult. It sounds like you are writing some kind of code anyway. on 10/30/06 2:59 PM, Martin Bestmann at martin(a)poleposition-sw.com wrote: > > On 30.10.2006, at 18:37, Paul Nelson wrote: > >> Apple's AD plug-in does not implement the LDAP paged results >> control. As a >> result, you only get the first 1000 records. >> > > Does anyone know of a work around for this problem, beside switching > to AdmitMac (which is definitely an option). > > Thanks, > > Martin > >> See http://www.ietf.org/rfc/rfc2696.txt >> >> Thursby's ADmitMac product does implement paged results in their AD >> plug-in: >> http://www.thursby.com >> >> Paul Nelson >> Thursby Software Systems, Inc. >> >> >> >> >> on 10/30/06 11:27 AM, Martin Bestmann at martin(a)poleposition-sw.com >> wrote: >> >>> Hi, >>> >>> I'm have problems getting all users from in an Active Directory >>> Domain using Directory Services. We have a test domain with about >>> 65000 users. When trying to get the list of users only exactly 1000 >>> are returned. Same with computers. >>> This also happens when using the dscl tool (dscl localhost -list >>> "Active Directory/All Domains/Computers") >>> I can get all AD users when I use a Windows machine and run some VB >>> scripts. So the AD controller is working correctly. >>> >>> Is the any secret setting that I need to turn on to get all users? >>> >>> Thanks, >>> >>> Martin >>> >>> _______________________________________________ >>> Do not post admin requests to the list. They will be ignored. >>> Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) >>> Help/Unsubscribe/Update your Subscription: >>> http://lists.apple.com/mailman/options/darwin-dev/nelson% >>> 40thursby.com >>> >>> This email sent to nelson(a)thursby.com >>> >> >> > > _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Directory Services only returns 1000 users from Active Directory/All Domains/Users
by Martin Bestmann 30 Oct '06

30 Oct '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On 30.10.2006, at 18:37, Paul Nelson wrote: Thanks, Martin See http://www.ietf.org/rfc/rfc2696.txt Paul Nelson Thursby Software Systems, Inc. Hi, I'm have problems getting all users from in an Active Directory Domain using Directory Services. We have a test domain with about 65000 users. When trying to get the list of users only exactly 1000 are returned. Same with computers. This also happens when using the dscl tool (dscl localhost -list "Active Directory/All Domains/Computers") I can get all AD users when I use a Windows machine and run some VB scripts. So the AD controller is working correctly. Is the any secret setting that I need to turn on to get all users? Thanks, Martin This email sent to nelson(a)thursby.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Apple's AD plug-in does not implement the LDAP paged results control. As a result, you only get the first 1000 records. Does anyone know of a work around for this problem, beside switching to AdmitMac (which is definitely an option). Thursby's ADmitMac product does implement paged results in their AD plug-in: http://www.thursby.com on 10/30/06 11:27 AM, Martin Bestmann at martin(a)poleposition-sw.com wrote: _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/nelson% 40thursby.com This email sent to site_archiver(a)lists.apple.com

1 0

Re: bringing pf (4) to OS X via NKE
by gorse＠mps.ohio-state.edu 30 Oct '06

30 Oct '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com My link to the man page seems to have been clobbered, but it should describe fairly accurately what I will attempt to implement: http://www.freebsd.org/cgi/man.cgi?query=pf&sektion=4 The OpenBSD guide for pf: http://www.openbsd.org/faq/pf/ And Daniel Hartmeier's (the original author) website: http://www.benzedrine.cx/pf.html In brief, pf (4) brings NAT, QoS, and stateful packet filtering to the kernel, whereas currently we must rely on userland programs like natd and throttled in order to do NAT and QoS, respectively, via Divert sockets. Thus, we take a big hit on performance and flexibility. For instance, to change a port forwarding rule in the current setup one must kill natd, modify either the command line arguments or the config file, then restart natd which results in all NAT connections being dropped in order to change a rule on the fly. Not to mention the time lost in communication from kernel to user and back to kernel a couple of times per packet. Hence the motivation for pf for OS X. Cheers, Joe Josh Graessley <jgraessley(a)apple.com> said: > > Those familiar with the KPIs may not be familiar with pf. > > An interface filter or an IP filter are the most likely places to tie > in to the stack. I know nothing of pf, so I can't really help with a > better answer. > > -josh > > On Oct 29, 2006, at 9:55 PM, Joseph Gorse wrote: > > > Hello all, > > > > I'm posting my intention to port pf (4) (http://www.freebsd.org/cgi/ > > man.cgi?query=pf&sektion=4) to an NKE for use as a replacement or > > complement to the current ipfw2. > > > > According to the Network Kernel Extensions Programming Guide > > (http://developer.apple.com/documentation/Darwin/Conceptual/ > > NKEConceptual/index.html) it seems I might use an Interface Filter > > KPI mechanism to accomplish such a task. So I ask those who are > > more familiar with NKEs, is this a reasonable task, am I sane to > > try it, and do you have any words of advice? > > > > Thank you for your time, > > Joe Gorse > > _______________________________________________ > > Do not post admin requests to the list. They will be ignored. > > Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) > > Help/Unsubscribe/Update your Subscription: > > http://lists.apple.com/mailman/options/darwin-dev/jgraessley% > > 40apple.com > > > > This email sent to jgraessley(a)apple.com > > -- _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: bringing pf (4) to OS X via NKE
by Josh Graessley 30 Oct '06

30 Oct '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Those familiar with the KPIs may not be familiar with pf. -josh On Oct 29, 2006, at 9:55 PM, Joseph Gorse wrote: Hello all, _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com An interface filter or an IP filter are the most likely places to tie in to the stack. I know nothing of pf, so I can't really help with a better answer. I'm posting my intention to port pf (4) (http://www.freebsd.org/cgi/ man.cgi?query=pf&sektion=4) to an NKE for use as a replacement or complement to the current ipfw2. According to the Network Kernel Extensions Programming Guide (http://developer.apple.com/documentation/Darwin/Conceptual/ NKEConceptual/index.html) it seems I might use an Interface Filter KPI mechanism to accomplish such a task. So I ask those who are more familiar with NKEs, is this a reasonable task, am I sane to try it, and do you have any words of advice? Thank you for your time, Joe Gorse _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/jgraessley% 40apple.com This email sent to jgraessley(a)apple.com smime.p7s

1 0

Re: Directory Services only returns 1000 users from Active Directory/All Domains/Users
by Paul Nelson 30 Oct '06

30 Oct '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Thread-index: Acb8ShNnUfs8Ymg9EdupyQAWy4i6sA== Thread-topic: Directory Services only returns 1000 users from Active Directory/All Domains/Users User-agent: Microsoft-Entourage/11.2.5.060620 Apple's AD plug-in does not implement the LDAP paged results control. As a result, you only get the first 1000 records. See http://www.ietf.org/rfc/rfc2696.txt Thursby's ADmitMac product does implement paged results in their AD plug-in: http://www.thursby.com Paul Nelson Thursby Software Systems, Inc. on 10/30/06 11:27 AM, Martin Bestmann at martin(a)poleposition-sw.com wrote: > Hi, > > I'm have problems getting all users from in an Active Directory > Domain using Directory Services. We have a test domain with about > 65000 users. When trying to get the list of users only exactly 1000 > are returned. Same with computers. > This also happens when using the dscl tool (dscl localhost -list > "Active Directory/All Domains/Computers") > I can get all AD users when I use a Windows machine and run some VB > scripts. So the AD controller is working correctly. > > Is the any secret setting that I need to turn on to get all users? > > Thanks, > > Martin > > _______________________________________________ > Do not post admin requests to the list. They will be ignored. > Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) > Help/Unsubscribe/Update your Subscription: > http://lists.apple.com/mailman/options/darwin-dev/nelson%40thursby.com > > This email sent to nelson(a)thursby.com > _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Directory Services only returns 1000 users from Active Directory/All Domains/Users
by Martin Bestmann 30 Oct '06

30 Oct '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Hi, Is the any secret setting that I need to turn on to get all users? Thanks, Martin _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… I'm have problems getting all users from in an Active Directory Domain using Directory Services. We have a test domain with about 65000 users. When trying to get the list of users only exactly 1000 are returned. Same with computers. This also happens when using the dscl tool (dscl localhost -list "Active Directory/All Domains/Computers") I can get all AD users when I use a Windows machine and run some VB scripts. So the AD controller is working correctly. This email sent to site_archiver(a)lists.apple.com

1 0

bringing pf (4) to OS X via NKE
by Joseph Gorse 30 Oct '06

30 Oct '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Hello all, Thank you for your time, Joe Gorse _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… I'm posting my intention to port pf (4) (http://www.freebsd.org/cgi/ man.cgi?query=pf&sektion=4) to an NKE for use as a replacement or complement to the current ipfw2. According to the Network Kernel Extensions Programming Guide (http:// developer.apple.com/documentation/Darwin/Conceptual/NKEConceptual/ index.html) it seems I might use an Interface Filter KPI mechanism to accomplish such a task. So I ask those who are more familiar with NKEs, is this a reasonable task, am I sane to try it, and do you have any words of advice? This email sent to site_archiver(a)lists.apple.com

1 0

Re: Logging from kernel
by Thulli 30 Oct '06

30 Oct '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:in-reply-to:references:mime-version:content-type:message-id:cc:content-transfer-encoding:from:subject:date:to:x-mailer; b=JceAKYRxJNcV93hhy9iz2GWQu9Aj+SvETIjiFvhNMAVcPoqOjSP4Mb0Vp6MpKrkVJt4pilwIDC5L/GTus1J7T4u0HxKXkHOan9TjSuqiBNPitC2HXj5Mr3RW2ubuhFGAkXa9whwgU24aYiw5aT+cT5+omkpbpuL3181f7hHGrq0= >> the other is syslog() void logpri(int); void log(int, const char *, ...); My kext has the following dependencies. <key>com.apple.kpi.mach</key> <string>8.0.0</string> <key>com.apple.kpi.libkern</key> <string>8.0.0</string> <key>com.apple.kpi.bsd</key> <string>8.0.0</string> Now kextload fails, saying.. kextload: extension ./MyKext.kext appears to be valid kld(): Undefined symbols: _log _logpri thanks, Thulli. On 26-Oct-06, at 7:57 PM, Justin C. Walker wrote: On Oct 26, 2006, at 24:42 , Thulli wrote: Not really a solution, for the same reasons the above aren't perfect. Justin -- Justin C. Walker, Curmudgeon at Large Institute for the Absorption of Federal Funds ----------- I'm beginning to like the cut of his jibberish. ----------- _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… really ? I assume you are refering to "Kernel.framework/Headers/sys/ syslog.h" After including the header <sys/syslog.h> i tried using the following calls in my kext. Changing the dependency to a more generic "com.apple.kernel" version 8.8.1 solves the kextload problem. Looks like log() and logpri() of syslog.h are unsupported ? b'coz i could trace the dependency of log() to "com.apple.kpi.unsupported" logpri() is there only in the header. Can somebody advise on the best method for logging from a kext. Considered the following. 1. printf and IOLog. I couldn't catch the logs just before my kext caused panic :) These are two choices; the other is syslog(). They all are pretty much the same (i.e., they are wrappers around the same code), although IOLog is designed for IOKit components. 2. Custom event logging: This method is very heavy. I don't want to write a user land tool, just for logging purpose. 3. asl looks very good. Is this available for a kext ? Well, the header file is in /usr/include, which is a great indicator for kernel use: can't (or shouldn't) be used in the kernel. Your basic problem is that, if a panic occurs, the system grinds to a halt and therefore, any scheme that makes use of system mechanisms will halt along with it. There is not a very good way to deal with this in the field, due to the above. Andrew's suggestion is about the best that I'm aware of. The ability to send a kernel dump to a (local) dump server might be of some help, depending on how your customers are set up. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/thullishah% 40gmail.com This email sent to thullishah(a)gmail.com This email sent to site_archiver(a)lists.apple.com

1 0