June 2009 - Darwin-kernel - lists.apple.com

Understanding kexts in the early boot process
by Thomas Tempelmann 14 Jun '09

14 Jun '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type:content-transfer-encoding; bh=RDgUOBVvFR9F0T3qjEYzrp/fCDywevzd//GF6DslN4A=; b=IsNklOwq6rBf8gAECn+wYwXK9BkU2KdyICGeCnimhAHfYWnmZikeXQN36ua+VO4Pm1 5GL00xNMa17k8MOG1PiVzxB9n12S0CUEZNsdZskiNGa2P9k6eBY4TDPwWplmZe+7AV2x qoodvYMkH3uNhb2yfLI3nl3SsF3Qj4sl3zmYM= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type :content-transfer-encoding; b=eKn0l0V5Q3DQHgiIReqzqCsrnQwplRx3kfMNpW3tEInjqjDnJnPus6H+DpgP3/YyaH cueZ/dNrgHnzI07ubELmxu7tabIRkjQMn3Vs3iiNpvAXdbaS3PQfGAyJWRatGatYUA2O yXHYoo69h4YgdgrJg/CxWItLko9Q+FArXmsqg= Despite reading Amit Singh's book "OS X Internals" and searching the web, I still am missing some details about how the OS X boot process works. Maybe someone here can enlighten me? The gap I am trying to fill is how the basic kexts are loaded and initialized: When the kernel starts, it appears to need a few drivers which are implemented in external kexts. Only later, when all is set up, there's a daemon which dynamically loads other kexts. I like to understand the binding of the former type of drivers. I understand that there's a kextcache, which binds the kernel with a set of kexts and stores it in /System/Library/Caches/com.apple.kernelcaches/ Question 1: What rule specifies which kexts get linked to that kernel? Is it simply the entire Extensions.mkext? Question 2: What if this file does not exist, can the system still boot, and which component is then taking care of finding the kexts (e.g. is it boot.efi or the kernel or what?) Now to the Extensions.mkext. I understand that this gets automatically created from a running OS X in the background whenever it detects changes to the Extensions folder (or at least in most such cases). Question 3: Does this mkext contain _all_ kexts from the Extensions folder then, or does it sort them out somehow (by what rule)? Amit writes that the boot loader (boot.efi) loads the kernel and Extensions.mkext. Is he referring to the prebound kernel cache file here? Provided that file does not exist (question 2), it probably loads the mktext file and kernel (from the root dir) separately. However, due to Amit's book, the boot_args struct does not provide for passing a reference to the mkext file to the kernel. Question 4: How does the mkext, if loaded separately, get linked to the kernel? (i.e. does the boot loader contain a linker that works like the kextcache command?) Question 5: How do these basic kexts get initialized then? Does the boot loader init them before calling the kernel, or does the kernel find them somehow and init them? -- Thomas Tempelmann, http://www.tempel.org/ _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: How to get Total cpu usage
by mm w 11 Jun '09

11 Jun '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=lsST5bWWjX1/M9YHC/sla9kAE83lHelR1oqtjckfJ9s=; b=LaMR1Zdg1HvJ7ZlD4r6bP4O0HkBb6bttzZaEetQVpxqbcTHY49F/HZLOuH+4vEF3Jj 9bEzpnxzX8V2TrltPi0sn8dwKa2P/4tr7fTZMiyACOUpIiWEoxWNHXpNoPZFDP+LcyJF 59sbh/0x5cTf9YDOx29xv24wXiuafc8SgefFI= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=CRSlYRAQGSjZX3MDG1u2U9NnrjNtf0OvNMJkdV0ecdASkcWVcM9TvHZBdtNwoY1EkT xxHOXIxrBYVjOzaOhW54qra6MiXhgSXxUHNNVUnwxGi+KOxCMzDdbzNtymwCOt+xxoae syxr/Nwe/HY5SLrusoHmGOEe2hJLOMGSaUHVI= Hi you did not make the effort to push your google button, basically something like the following snap but there are other options to estimate it for instance: - you could popen vm_stat - open "Activity Monitor" - you may also be able to list all proc with sysctl familly - getrusage now it's in your hands: to find the most significant average calculus uint32_t count; processor_cpu_load_info_t info; mach_msg_type_number_t message; kern_return_t status = host_processor_info( host, PROCESSOR_CPU_LOAD_INFO, &count, &info, &message ); do { // info[i].cpu_ticks[CPU_STATE_SYSTEM]; // info[i].cpu_ticks[CPU_STATE_USER]; // info[i].cpu_ticks[CPU_STATE_IDLE]; /* calculate your average */ } while (++i < count); PS: NOT A KERNEL QUESTION Best, On Wed, Jun 10, 2009 at 10:52 PM, Mathu kannan<mathueie(a)gmail.com> wrote: > Hi, > > How to get Total cpu usage programmetically in MAC. > Any API is there? > > > Regards, > Mathu. > > > > _______________________________________________ > Do not post admin requests to the list. They will be ignored. > Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) > Help/Unsubscribe/Update your Subscription: > http://lists.apple.com/mailman/options/darwin-kernel/0xcafefeed%40gmail.com > > This email sent to 0xcafefeed(a)gmail.com > _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

How to get Total cpu usage
by Mathu kannan 11 Jun '09

11 Jun '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type; bh=TtOUGIjJGyybVN/YN/saDBjgGeZWxYOrCYtpG7l7yAo=; b=hF0JMCK7n9v/WoEjlTZ2B5UgVJBuCq3gXMFgs1TAaQgYjMdkQdt/4XvfzmwiD9IDdU ZtZd5pqUHVSq1zlBN1Pwouuz2NjxvjcjIS5CU+Dp7//kej+1HlYmphtscO1TqqKTwLrj 0GIXGL48MX2o+zJtgKihZPFSYA6UeB3+sfYc8= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=ZZjLIv29wwemAIwSKZRtpH/7lHLCDJ9+c2U5wFH2Wxh8CUPv9O/HO/beXmFhzHUfc+ 56X1w6nQ8yVp3IFKz0gTO6P8lb9m6dJY2Bp9eki8OMFHbqo+hUe5ZeTmu7zg43T0cWFr 6QRpgY/tmepG7xbKH52RGwuEhAlOWGUjrPj48= _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Host tool for KDP over serial
by Michael Crawford 10 Jun '09

10 Jun '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=+BObW91zPp1iuNv8ukNdbU3jOWkorhk1wPbLR2QnM7s=; b=AbDl6raEmigoz9yEq/4MDqTWs1l1KeUfIK/uWYLzS7uQ6YXLNNQuNSypvsbVWcnLt5 TbkucOL50zcjaoZs0nwtJNc5PKMBzvLxcr7bCdlfgLVXJHcVprRV/86Jf9A5UbsNa7Yo aiYKS6uKH9TWVe0+RNsDJMhVA4dUGASG3Fjfg= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=tYwg40v6ncV/+Kx2oVO1l7FTVCYGTnDPjXOg5B7JdEwFJaENeAn9oytsUpfk6pc3NM 2n5Hc7fQ+o15Vy7nDOummcbUbzugDN8ltIOzNk6joe9VlP5jlhC2+RTAfb2mNkAUGHKO 0SIlUlP6RYty0gpUjqdVQwSlwWlkkO5OA1O9c= On Wed, Jun 10, 2009 at 10:06 AM, David Elliott<dfe(a)tgwbd.org> wrote: > I'll go ahead and write the host-side tool and see if I can get that > working. It seems to me that this code must have been used fairly recently > since someone bothered to add an #if CONFIG_EMBEDDED to it. So I have to > assume the tool exists at Apple. I assume you have your reasons for not > wanting to give it up so that's that. I figured it didn't hurt to ask > though. I don't know firsthand, but I would expect that the embedded target is for the version of Mac OS X that runs on the iPhone and iPod Touch. I understand they use ARM processors. The retail units don't have serial ports, but possibly the target boards that Apple's engineers use for development do. Mike -- Michael David Crawford mdcrawford at gmail dot com GoingWare's Bag of Programming Tricks http://www.goingware.com/tips/ _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Host tool for KDP over serial
by David Elliott 10 Jun '09

10 Jun '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Hi Shantonu, Thanks, -Dave On Jun 10, 2009, at 1:47 AM, Shantonu Sen wrote: Shantonu Sent from my MacBook On Jun 9, 2009, at 9:46 PM, David Elliott wrote: Hi, -Dave _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/ssen%40apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… I am bringing up Darwin on a different type of i386 and I need debug earlier than either Ethernet or FireWire can provide. Ordinarily in a case like this I'd use VMware's debug stub but the problem I'm having is that I have a particular machine that seems to hang during startup with the last IOLog message indicating that IOPCIConfigurator::attach is running. Obviously it's not possible to debug stuff in a VM that only happens on a particular physical machine. Both the firewire and ethernet debuggers match on the PCI device nub so they aren't available until after PCI configuration has occurred. With a serial-based KDP connection that starts before the PE is started I would presumably be able to figure out where it's hanging and what's happening. I'll go ahead and write the host-side tool and see if I can get that working. It seems to me that this code must have been used fairly recently since someone bothered to add an #if CONFIG_EMBEDDED to it. So I have to assume the tool exists at Apple. I assume you have your reasons for not wanting to give it up so that's that. I figured it didn't hurt to ask though. I don't believe we have any plans to provide a tool for this, since the only Mac with built-in serial is the Xserve, which has both firewire and ethernet. Those are the supported transports for doing GDB interactive debugging and saving kernel core dumps. I'd like to run KDP over RS232 serial on an i386 target. Examination of source in osfmk/kdp indicates that the kernel expects to see ethernet frames coming in over RS232 preceded with 0xFA and ending with 0xFB with occasional escaping using 0xFE to indicate an escape, two's complement of the escaped bytes, and 0xFB to end the escape. I have succeeded in hand crafting an ARP packet of this format and when I send it down the wire I appear to get the appropriate response from the kernel. So now I am wondering if Apple has provided or can provide a tool similar to the one for Firewire debugging that listens on UDP 41139 and shuttles the data over a /dev/tty device and back. If not, writing one would be reasonably easy but I figured I'd ask before embarking on that. Source is preferable as I'd prefer to run it on a Linux machine with a hardware serial port as the PL2303- based USB I have on the host side seems to have strange issues with the driver from sourceforge. Also, there's an interesting "bug" of sorts in the code. The IP of the KDP will actually be different depending on whether it's big or little endian. The target's IP is set to 0xABADBABE so on little endian targets the IP is 190.186.173.171 but on big endian targets it would presumably be 171.173.186.190. A bad babe indeed. This email sent to ssen(a)apple.com This email sent to site_archiver(a)lists.apple.com

1 0

Internet Block/Unblock on Client
by Rahulkumar Tibdewal 10 Jun '09

10 Jun '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Thread-index: Acnp2+vXAFZl3su8RsmMrzPrYD/qAw== Thread-topic: Internet Block/Unblock on Client _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Host tool for KDP over serial
by Shantonu Sen 10 Jun '09

10 Jun '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Shantonu Sent from my MacBook On Jun 9, 2009, at 9:46 PM, David Elliott wrote: Hi, -Dave _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/ssen%40apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… I don't believe we have any plans to provide a tool for this, since the only Mac with built-in serial is the Xserve, which has both firewire and ethernet. Those are the supported transports for doing GDB interactive debugging and saving kernel core dumps. I'd like to run KDP over RS232 serial on an i386 target. Examination of source in osfmk/kdp indicates that the kernel expects to see ethernet frames coming in over RS232 preceded with 0xFA and ending with 0xFB with occasional escaping using 0xFE to indicate an escape, two's complement of the escaped bytes, and 0xFB to end the escape. I have succeeded in hand crafting an ARP packet of this format and when I send it down the wire I appear to get the appropriate response from the kernel. So now I am wondering if Apple has provided or can provide a tool similar to the one for Firewire debugging that listens on UDP 41139 and shuttles the data over a /dev/tty device and back. If not, writing one would be reasonably easy but I figured I'd ask before embarking on that. Source is preferable as I'd prefer to run it on a Linux machine with a hardware serial port as the PL2303- based USB I have on the host side seems to have strange issues with the driver from sourceforge. Also, there's an interesting "bug" of sorts in the code. The IP of the KDP will actually be different depending on whether it's big or little endian. The target's IP is set to 0xABADBABE so on little endian targets the IP is 190.186.173.171 but on big endian targets it would presumably be 171.173.186.190. A bad babe indeed. This email sent to ssen(a)apple.com This email sent to site_archiver(a)lists.apple.com

1 0

Host tool for KDP over serial
by David Elliott 10 Jun '09

10 Jun '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Hi, -Dave _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… I'd like to run KDP over RS232 serial on an i386 target. Examination of source in osfmk/kdp indicates that the kernel expects to see ethernet frames coming in over RS232 preceded with 0xFA and ending with 0xFB with occasional escaping using 0xFE to indicate an escape, two's complement of the escaped bytes, and 0xFB to end the escape. I have succeeded in hand crafting an ARP packet of this format and when I send it down the wire I appear to get the appropriate response from the kernel. So now I am wondering if Apple has provided or can provide a tool similar to the one for Firewire debugging that listens on UDP 41139 and shuttles the data over a /dev/tty device and back. If not, writing one would be reasonably easy but I figured I'd ask before embarking on that. Source is preferable as I'd prefer to run it on a Linux machine with a hardware serial port as the PL2303-based USB I have on the host side seems to have strange issues with the driver from sourceforge. Also, there's an interesting "bug" of sorts in the code. The IP of the KDP will actually be different depending on whether it's big or little endian. The target's IP is set to 0xABADBABE so on little endian targets the IP is 190.186.173.171 but on big endian targets it would presumably be 171.173.186.190. A bad babe indeed. This email sent to site_archiver(a)lists.apple.com

1 0

Multiple DNS servers
by Platon Fomichev 05 Jun '09

05 Jun '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Dear Friends /* warn lookupd of upcoming change */ notify_post("com.apple.system.dns.delay"); publish_dns_entry(kSCPropNetDNSServerAddresses, ...) that's basically the core of DNS changes. Thanks in advance. Platon Fomichev _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… Recently I've been working on porting of some VPN software to OS X, and although the process was smooth and I am fairly pleased with XCode, OS X, API ;) etc there is a pretty bad issue I don't know how to resolve. I can see this issue is in the pppd daemon as well so I'll use it as example because my work borrows a lot of hints from pppd tool. The issue is simple - let's say I have a pppd based VPN, I receive local/remote address and I can also receive DNS/WINS servers from remote host, if I ask for it. If I do so the following actions take place: and publish_dns_entry operates on SCDynamicStoreKeyCreateNetworkServiceEntity(0, kSCDynamicStoreDomainState, serviceidRef, kSCEntNetDNS); The result is as follows - remote DNS servers become main or primary ones, i.e. all DNS queries are routed through them. In my application this is not true, and I dare say this is also valid for many VPNs as well - I need to have remote DNS servers and current DNS servers simultaneously because VPNs is organized like additional network and not a default gateway for all traffic. So my question is - why does it happen that remote DNS servers completely eliminate current DNS server - is it by design? If yes - what should I do to override such design? I guess adding original DNS servers to the list will work.. Is it possible to fix this for PPPD daemon right from the box? This email sent to site_archiver(a)lists.apple.com

1 0

Re: write(2) fails for large sizes in 64 bit applications
by Robert Homann 04 Jun '09

04 Jun '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On Tue, 2 Jun 2009, Terry Lambert wrote: > On Jun 2, 2009, at 5:37 AM, Quinn <eskimo1(a)apple.com> wrote: Hi! > Our change control process will not allow us to make arbitrary changes > or commit time to doing work without some type of bug report or > project attachment. It's very customer focussed. People can complain > about things on mailing lists all they want, but unless someone writes > up a bug, that's all they've done. OK, this is reasonable. Thank you for your insightful explanations on the bug reporting process. > This particular problem itself, though, is a bad example of something > to argue passionately about, since the cause is understood, it's > easily avoided, the operations in question are practically absurdly > large, and the consequences of making the operations "work" will be > large I/O stalls as the I/O request gets saturated at the disk, whose > firmware doesn't allow the kernel driver to reorder in-flight requests > based on later higher priority requests (i.e. the hardware itself is > uncooperative with background I/O). Well, INT_MAX+1 is absurdly large, but INT_MAX is not? ;-) But I understand what you mean. An application should be playing nice when doing I/O, and not claim the whole bandwidth for itself. Nevertheless, I'd consider this to be an optimization to fulfull an operating system's policy. Alternatively, the kernel could force applications to be nice by writing only as much as the kernel likes to; this might, of course, involve breaking applications that are broken anyway (not checking return values etc.). Anyway, thank you again for all the information. I will keep these in mind when writing/porting software for/to OS X. Best regards, Robert Homann -- Windows is not the answer. Windows is the question. The answer is "No". _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0