June 2010 - Darwin-kernel - lists.apple.com

Re: Doing file io from kernel IOKit driver context, again.
by Chris Suter 10 Jun '10

10 Jun '10

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On Thu, Jun 10, 2010 at 11:49 AM, Michael Smith <drivers(a)mu.org> wrote: > These filesystems will reject F_LOG2PHYS, which is generally a good safety > net. I've come to this thread late, but unless I'm mistaken F_LOG2PHYS is practically useless; it doesn't give you the length of the extent. In any case, it sounds horrendous. There'd be nothing stopping the filesystem from moving your file after you'd got the result back. The best approach is to farm out all I/O requests via a user process. I believe this is what usually happens for Apple's disk images (although it is also possible to have purely in-kernel handling). Kind regards, Chris _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Doing file io from kernel IOKit driver context, again.
by Michael Smith 10 Jun '10

10 Jun '10

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Doing file io from kernel IOKit driver context, again.
by Joe Lowe 09 Jun '10

09 Jun '10

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com User-agent: Thunderbird 2.0.0.16 (Windows/20080708) Joe L. Mike, Wow, that sound like a great idea! I don't like to go from the lower level (block) to the upper (fs) and back to lower myself, i suspect there can be a lot of synchronization issues involved. Again, your idea sounds great, but i wonder how safe it is? How do i lock the file down so that i can guarantee the block map won't go stale for as long as a have the image of that file mounted? Inso. Inso, You don't want to be doing file I/O, and in fact you don't need to. Your userspace helper tool provides everything you need; it can create your backing files, obtain the physical block map for each file (F_LOG2PHYS), and then pass all of this information into your driver. The driver then just looks up the IOMedia that hosts the filesystem and performs block I/O against it while the user process holds the file down. File I/O has a lot of baggage that you just don't want or need for this. Take it from folks that know, eh? = Mike On Jun 9, 2010, at 2:00 AM, Inso Reiges wrote: Hello, After searching darwin mailing lists i got a general impression that doing file io from kernel context is discouraged, especially if it is an IOKit driver context.) People mostly just say "don't do it", ask for bigger picture issues and advise an alternative which works. Me, i want to write a driver similar to Apple's Disk Images just for the sake of writing it. Obviously this loop device has to map block device read/writes to file read/writes which means doing file io in kernel. After messing around a bit with IOHDIXController i figured that hdiutil/diskimages-helper calls createDrive64 through HDIX Controller's user client. The data it sends as a parameter seems like an xml plist with image file name among other things. So if the bigger picture issue is actually creating an IOKit loop device driver, how do i safely read and write files from such driver? How does Apple Disk Images handles that? I experimented a bit with the following approach: 1) Pass an image file name to IOKit driver. 2) Lookup and acquire corresponding vnode through VFS layer. 3) Call VNOP_READ, VNOP_WRITE on vnode. It seems to be working fine for me. Is there anything to look out for with this approach? Are there any other alternatives without using a VFS layer? P.S. I also looked at bsd vn driver which does basically the same, but i want my driver to be an IOService object to have access to IORegistryEntry. Thank you, Inso. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/drivers%40mu.org This email sent to drivers(a)mu.org -- Excellence in any department can be attained only by the labor of a lifetime; it is not to be purchased at a lesser price -- Samuel Johnson _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/joe.lowe%40pismotechni… _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… Bypassing the file system and doing direct block I/O will not work with some file systems, such as encrypting, compressing, log based, and network file systems. Unless your application is inherently tied to HFS then you probably need to support VFS I/O at least as a fallback. Inso Reiges wrote: On Wed, Jun 9, 2010 at 10:23 PM, Michael Smith <drivers(a)mu.org> wrote: This email sent to joe.lowe(a)pismotechnic.com This email sent to site_archiver(a)lists.apple.com

1 0

Re: Doing file io from kernel IOKit driver context, again.
by Inso Reiges 09 Jun '10

09 Jun '10

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=Csr49Jt0xRtzhP1pMtUITfX7COSk84SZd870oAdNmL0=; b=s7gdyxgc46rojV4uuKy4Wb2Pb/u7evd/K2CkLkrZzPmjC+tzbrfX9ZrQgaAxfCAgNR ngFT8pEBNUN9hHeD8iT4MYGr5ZX2gsudX4BOX32q0ted9fJia7guY+OaYpFaNvyK3RHI 7OR43UVpmhEdMf3WmgRhq+/i8h6sIQkk+PNyE= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=IxF1qpzM3v+yAC0Va/KXDmh58gk7a8uM+8SlvXNzrdYMjkM8NUJTMHXsgxlzpYUp0r f5riTYRiIoAlznXjDnQtEzT68UGu/AOG9Zhplt32tzpAscH2ulYJV3WRCzrpoYPSspu1 I6KaFxXFcF5gla/LZx08mpyAPPGYomO07sZHs= Mike, Wow, that sound like a great idea! I don't like to go from the lower level (block) to the upper (fs) and back to lower myself, i suspect there can be a lot of synchronization issues involved. Again, your idea sounds great, but i wonder how safe it is? How do i lock the file down so that i can guarantee the block map won't go stale for as long as a have the image of that file mounted? Inso. On Wed, Jun 9, 2010 at 10:23 PM, Michael Smith <drivers(a)mu.org> wrote: > > Inso, > You don't want to be doing file I/O, and in fact you don't need to. > Your userspace helper tool provides everything you need; it can create your > backing files, obtain the physical block map for each file (F_LOG2PHYS), and > then pass all of this information into your driver. The driver then just > looks up the IOMedia that hosts the filesystem and performs block I/O > against it while the user process holds the file down. > File I/O has a lot of baggage that you just don't want or need for this. > Take it from folks that know, eh? > = Mike > On Jun 9, 2010, at 2:00 AM, Inso Reiges wrote: > > Hello, > > After searching darwin mailing lists i got a general impression that > doing file io from kernel context is discouraged, especially if it is > an IOKit driver context.) People mostly just say "don't do it", ask > for bigger picture issues and advise an alternative which works. > > Me, i want to write a driver similar to Apple's Disk Images just for > the sake of writing it. Obviously this loop device has to map block > device read/writes to file read/writes which means doing file io in > kernel. After messing around a bit with IOHDIXController i figured > that hdiutil/diskimages-helper calls createDrive64 through HDIX > Controller's user client. The data it sends as a parameter seems like > an xml plist with image file name among other things. > > So if the bigger picture issue is actually creating an IOKit loop > device driver, how do i safely read and write files from such driver? > How does Apple Disk Images handles that? > > I experimented a bit with the following approach: > 1) Pass an image file name to IOKit driver. > 2) Lookup and acquire corresponding vnode through VFS layer. > 3) Call VNOP_READ, VNOP_WRITE on vnode. > > It seems to be working fine for me. > Is there anything to look out for with this approach? Are there any > other alternatives without using a VFS layer? > > P.S. I also looked at bsd vn driver which does basically the same, but > i want my driver to be an IOService object to have access to > IORegistryEntry. > > Thank you, > Inso. > _______________________________________________ > Do not post admin requests to the list. They will be ignored. > Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) > Help/Unsubscribe/Update your Subscription: > http://lists.apple.com/mailman/options/darwin-kernel/drivers%40mu.org > > This email sent to drivers(a)mu.org > > -- > Excellence in any department can be attained only by the labor of a > lifetime; it is not to be purchased at a lesser price -- Samuel Johnson > > > > > > > _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Doing file io from kernel IOKit driver context, again.
by Michael Smith 09 Jun '10

09 Jun '10

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Doing file io from kernel IOKit driver context, again.
by Ken Hornstein 09 Jun '10

09 Jun '10

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com >After searching darwin mailing lists i got a general impression that >doing file io from kernel context is discouraged, especially if it is >an IOKit driver context.) People mostly just say "don't do it", ask >for bigger picture issues and advise an alternative which works. Yeah, I ran into that same feedback a few weeks ago. For me, I investigated every other option ... but they all involved writing tons of code. >I experimented a bit with the following approach: >1) Pass an image file name to IOKit driver. >2) Lookup and acquire corresponding vnode through VFS layer. >3) Call VNOP_READ, VNOP_WRITE on vnode. Personally, I think you could simplify things a bit. - vnode_open() (you're not calling VNOP_LOOKUP for each pathname component, are you?) - vn_rdwr() (you don't have to go through the trouble of building a uio structure, for one). >It seems to be working fine for me. >Is there anything to look out for with this approach? Are there any >other alternatives without using a VFS layer? I think the only things you need to worry about is: - Make sure you're holding an iocount on the vnode before you call any vfs layer functons (via vnode_get() or a similar function) - Don't hold the iocount too long. --Ken _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Doing file io from kernel IOKit driver context, again.
by Inso Reiges 09 Jun '10

09 Jun '10

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:content-type; bh=HcbI8YzCK9e+T+CkOM+p3PzwoDHrWCEMTU1O5Q/8Z3o=; b=dPYcgJRj8LX5ow0e/J52GztkwPG6y2gWzQ2nDB4IBzVQiLrnTMJvmnJjJju5DotEDP AxRAltul3soLMjlKy4KBeLgcRTglW0+PKvFlC30ZSxersNyK7u2geNMWXgEhCQv8FJCK hK2SHcImgLsEUOt21InuRfKHskncD3b0+gL2c= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=PDVz5hVNhaPn6btd4Lfr/sMAJWbN6uklYnxvFDTsM9Dd9Lw5JH2iyJk9dVSCMA6I4X NNhuzvruXg+SWywYfdonyOItn6sstLAcfLRqsScaVW4B8ZI+rknx4Q1b2lNGxksYSoxn nZ9HDFIxGoqZfkwNLqezSpsIDKYxoqXxgrCAg= Hello, After searching darwin mailing lists i got a general impression that doing file io from kernel context is discouraged, especially if it is an IOKit driver context.) People mostly just say "don't do it", ask for bigger picture issues and advise an alternative which works. Me, i want to write a driver similar to Apple's Disk Images just for the sake of writing it. Obviously this loop device has to map block device read/writes to file read/writes which means doing file io in kernel. After messing around a bit with IOHDIXController i figured that hdiutil/diskimages-helper calls createDrive64 through HDIX Controller's user client. The data it sends as a parameter seems like an xml plist with image file name among other things. So if the bigger picture issue is actually creating an IOKit loop device driver, how do i safely read and write files from such driver? How does Apple Disk Images handles that? I experimented a bit with the following approach: 1) Pass an image file name to IOKit driver. 2) Lookup and acquire corresponding vnode through VFS layer. 3) Call VNOP_READ, VNOP_WRITE on vnode. It seems to be working fine for me. Is there anything to look out for with this approach? Are there any other alternatives without using a VFS layer? P.S. I also looked at bsd vn driver which does basically the same, but i want my driver to be an IOService object to have access to IORegistryEntry. Thank you, Inso. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Problem setting up ip4 bind address
by Dreamcat Four 01 Jun '10

01 Jun '10

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:mime-version:received:from:date :message-id:subject:to:content-type; bh=G2pQY18RINF5jEFKHEMvKhegLnFmqKNsB2t9VTJZS30=; b=P8rhy+jHN4Nni8J0LWWYl4Olq9pOE7ndEQV1h8FKdX9ACzWI3LzKsc2ohiAc03khWB E3s31yTwbV5EB2uK5bHeiQa3wJ6gBLxmL9cS6o+C05s+Gq63pmko3R1xLzG0/KsbKTO+ u+D9M7VAfgUzOUEyF04WQ33yRrT1Ip5bH42E8= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:from:date:message-id:subject:to:content-type; b=UcD5M3VM/giJZEPJzLmTWlgJQwoGeJJsUoAtZAOdLRJiFoZFwl0ke41GKyB+3XnKG/ myVet+M7hzggemBgygL9Gpi/Rmu5jv1HBNNE/iUvITVLSfj74hCXJE90aBubdTyMNmYS tpaPbpHcKWyXk+5mR49mztibNffAJTOPaYmEc= Having problems with an ifconfig alias for VPN. Its for an ipv4 bind address. To set the alias address im using: $ ifconfig ppp0 alias <bind ip address> <vpn gateway remote addr> netmask 255.255.255.0 Is ifconfig is calling out to pppd daemon to set up the alias address? It seems packets send to the alias address 111 appear on the ppp0 interface. But not to go to the right destination. Perhaps some ipfw / packet-rewriting rule is needed? janitors-Mac-mini:peers id$ sudo tcpdump -i ppp0 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on ppp0, link-type PPP (PPP), capture size 65535 bytes 14:31:55.350430 IP 192.168.0.26.59258 > 192.168.0.111.http: Flags [S], seq 1400391633, win 65535, options [mss 1404,nop,wscale 3,nop,nop,TS val 95271349 ecr 0,sackOK,eol], length 0 14:31:55.350576 IP 192.168.0.26.59259 > 192.168.0.111.https: Flags [S], seq 3103647381, win 65535, options [mss 1404,nop,wscale 3,nop,nop,TS val 95271349 ecr 0,sackOK,eol], length 0 If its any help, I would be happy to post some further logs and info about this particular case. dreamcat4 dreamcat4(a)gmail.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0