February 2010 - Darwin-kernel

IPC Choices with launchd
by Duane Murphy 10 Feb '10

10 Feb '10

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 I'm looking for IPC choices that play well with launchd. We are presently using launchd with Unix sockets. The unfortunate side affect is that accessing the socket causes system activity and prevents the system from sleeping. This happens when we are communicating a lot with our launchd tool. (I'm still confirming that this is the case.) In a previous incantation, we used pipes (standard in/out), this seemed not to cause any system activity and the system would sleep normally. I'm looking for other potential solutions that might play nice with launchd and provide a kind of ipc between the host and the target without preventing the system from sleeping. Thanks for any tips, ...Duane -----BEGIN PGP SIGNATURE----- Version: PGP Desktop 9.10.0 (Build 500) Charset: US-ASCII wsBVAwUBS3IOVUrg9acQ4r2CAQg6Owf/R/mBvMhYd54eavX0WxjwSgkPL9gVAg9L JEhF52ZLn8Tq9Wqi++Vnb2zmldznSkFfNmgRtVpV0LINk5BrY672KJLAfaQGGv3g kRgH+dxTbNmNgY74F/cSm1iIjw9dBLwgovboCc7eDYM5bT1JJUb48giYxIp94RpI MZ01AzcRQycpJcNjsjGGRUljKS8PcyIQSkmNmWxWFOqSic3/EubiBoBuvInJyU8j WfKb8JnQE0Bhd3/TvVckYMwvI6ForD3UmO4ygrepJSsMvu9YPdN484xpJYN6AGYh HYwdAvCAWVsDtkmtbh9i7OmY0bsY33Mhy91rJMtno3jP/AC3mIfCvQ== =XQZn -----END PGP SIGNATURE----- _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: # of bytes read/written by process
by James C 09 Feb '10

09 Feb '10

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type:content-transfer-encoding; bh=26sFbqmE3vfvw64/Ffm1jrvjXWxhnIOz7PJ75pv6yLU=; b=EZfHu9JYCmtoovQR+n7eKg4zXDO8iGxUIR6L/7l/eRpA+1lB4vMJMB5uSlnkv+0m3+ Zw4Zbp86NSXgz4vzhqtMqO8P874zwahAu14elW0RJe0NDevzfAgV5UY5CnvXAwTSxBzz n9OJNnoi6XvmLEhbTCKxS3YHF2z/0jQXjKfJY= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type :content-transfer-encoding; b=OY44WngMWhgReafVzti+sH9FiP94uGyXqojG2LLytE3VqCQaBz+tuVI2El+fvTRWk2 aS+jOCjTutGSmQjerY7gfUMhe+3jkLvGZbEr2U1CEOSvT4M8Et3OI1KXTOp1ojGJZ1Vg Z3gGUgZiH0cZT35MwlWloJEJlxjpnlF3WtVpc= It's not exactly the same, but you could start with: bitesize.d(1m) - analyse disk I/O size by process. Uses DTrace > Message: 1 > Date: Tue, 09 Feb 2010 09:44:35 -0700 > From: Corey Ernst <corey(a)elemtech.com> > Subject: # of bytes read/written by process > To: darwin-kernel(a)lists.apple.com > Message-ID: <4B7190F3.6000202(a)elemtech.com> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > I'm wondering if anyone knows if there is an equivalent capability on > mac os to query the number of bytes that a process reads and writes to > disk. On Linux you can look in /proc/self/io and find the information > there. The documentation in Linux gives the definition for these number > as noted below. I'm wondering if this information can be obtained on > the mac. > thanks, > --Corey > > //----------------------Getting two numbers out of this file > // I/O counter: chars read > //The number of bytes which this task has caused to be read from > storage. This > //is simply the sum of bytes which this process passed to read() and > pread(). > //It includes things like tty IO and it is unaffected by whether or > not actual > //physical disk IO was required (the read might have been satisfied from > //pagecache) > > // I/O counter: chars written > //The number of bytes which this task has caused, or shall cause to be > written > //to disk. Similar caveats apply here as with rchar. > _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

# of bytes read/written by process
by Corey Ernst 09 Feb '10

09 Feb '10

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com User-agent: Thunderbird 2.0.0.23 (X11/20090825) _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… I'm wondering if anyone knows if there is an equivalent capability on mac os to query the number of bytes that a process reads and writes to disk. On Linux you can look in /proc/self/io and find the information there. The documentation in Linux gives the definition for these number as noted below. I'm wondering if this information can be obtained on the mac. thanks, --Corey //----------------------Getting two numbers out of this file // I/O counter: chars read //The number of bytes which this task has caused to be read from storage. This //is simply the sum of bytes which this process passed to read() and pread(). //It includes things like tty IO and it is unaffected by whether or not actual //physical disk IO was required (the read might have been satisfied from //pagecache) // I/O counter: chars written //The number of bytes which this task has caused, or shall cause to be written //to disk. Similar caveats apply here as with rchar. This email sent to site_archiver(a)lists.apple.com

1 0

Re: writev performance
by Quinn 08 Feb '10

08 Feb '10

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Correct. Yes. S+E -- Quinn "The Eskimo!" <http://www.apple.com/developer/> Apple Developer Relations, Developer Technical Support, Core OS/Hardware _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… At 20:35 +1100 7/2/10, Godfrey van der Linden wrote: I'm pretty sure that the 4Gbyte KVA on 32 bit is going to make the uio_copyin() operations relatively expensive but what about the 64bit kernels do they naturally have better access to user space mappings? K64 definitely changes the landscape for this sort of thing, but I don't know whether it's enough to tip a balance like this. This email sent to site_archiver(a)lists.apple.com

1 0

Re: SCTP on mac
by Andreas Fink 08 Feb '10

08 Feb '10

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: writev performance
by Michael Smith 07 Feb '10

07 Feb '10

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

writev performance
by Godfrey van der Linden 07 Feb '10

07 Feb '10

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com G'day, Kernel experts. I've been doing a little bit of coding using UDP protocols recently and tripped over the sendto api that takes two user land addresses: a data address and a sockaddr pointer. I realised that this means that two uio_copyin transfers needs to be made, this then made me wonder about the general efficiency of writev style system calls. I have been using as a rule of thumb that limiting accesses to the vm system is a guaranteed way of increasing performance as the Mach VM system has been slow. It seems then if we have more than one uio interaction with user addresses copying data into the kernel we are multiply the number of vm interactions, and that is bad. Given how fast modern processors can move memory wouldn't it be better to copy the vector of data into a single transfer buffer into the kernel. This means that the processor caches are nice and hot and everything is ready to go when the single transfer into the kernel goes, this seems to me to be probably faster. I haven't run any experiments yet but I'm sort of interested in your ideas. I'm pretty sure that the 4Gbyte KVA on 32 bit is going to make the uio_copyin() operations relatively expensive, but what about the 64bit kernels do they naturally have better access to user space mappings? Thoughts? Godfrey _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: SCTP on mac
by Quinn 04 Feb '10

04 Feb '10

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com For those following along at home these are: o <rdar://problem/3893228> for providing a proper protocol KPI o <radr://problem/3800302> for providing SCTP S+E -- Quinn "The Eskimo!" <http://www.apple.com/developer/> Apple Developer Relations, Developer Technical Support, Core OS/Hardware _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… At 13:21 +0000 4/2/10, Quinn wrote: We have a number of bugs on file requesting a proper KPI for protocol-level plug-ins. This email sent to site_archiver(a)lists.apple.com

1 0

Re: SCTP on mac
by Quinn 04 Feb '10

04 Feb '10

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com [putting on the official Apple Developer Relations hat] S+E -- Quinn "The Eskimo!" <http://www.apple.com/developer/> Apple Developer Relations, Developer Technical Support, Core OS/Hardware _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… At 08:12 -0500 4/2/10, Michael Cashwell wrote: As others have stated before: If you need a kernel symbol that is not available in the KPIs, please file a bug at <https://bugreport.apple.com/> Yeah, Michael is fully aware of this process. We have a number of bugs on file requesting a proper KPI for protocol-level plug-ins. Unfortunately implementing this is hard and, given the current level of engineering resources available, unlikely to happen soon. This email sent to site_archiver(a)lists.apple.com

1 0

Re: SCTP on mac
by Michael Cashwell 04 Feb '10

04 Feb '10

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On Feb 4, 2010, at 5:07 AM, Michael Tüxen wrote: > On Feb 4, 2010, at 12:14 AM, Godfrey van der Linden wrote: >> Also I'm sort of curious, what is the SCTPSupport kext for? Are you working around some kext issue? > > Just exporting symbols that the NKE needs. Unfortunately, it is not possible to implement a transport protocol which makes use of the socket layer by using the API provided by Apple. As others have stated before: If you need a kernel symbol that is not available in the KPIs, please file a bug at <https://bugreport.apple.com/> As I understand it, Apple isn't trying to hide functionality or impede developers via the KPI. The goal is to just define the APIs that are stable from release to release while still allowing innovation and improvement. But if Apple don't know something needed is missing they can't add it and discussion on this list doesn't constitute "knowing". -Mike _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0