July 2009 - Darwin-dev - lists.apple.com

Re: Sending SCSI commands in Mac OS X
by mlists＠ugsoft.de 23 Jul '09

23 Jul '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha1; c=relaxed/relaxed; t=1248357879; l=1674; s=domk; d=ugsoft.de; h=References:Date:Subject:Mime-Version:Content-Transfer-Encoding: Content-Type:In-Reply-To:To:From:Cc:X-RZG-CLASS-ID:X-RZG-AUTH; bh=dDGVFVO2WijJShXqL1N9JUC6D/Q=; b=HMYbZxu8FvaDk9n1IcXSk3a5VoHTQ/76GORdq+uUgcTPLLlDgYdaJYyRcWSeq0edF/Z azXhplA50dQ2OGwPjglODX7knG/fgZGuiCK13akgI9JCAPHodtxU9pAJYj/u0PjJthchU zoNBrvzJnDNCgBY5AVwrvLM3aT1SU/hepF4= Am 23.07.2009 um 13:26 schrieb Terry Lambert: It states: Uwe _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Did you go to <http://developer.apple.com> and read the document "SCSI Architecture Model Device Interface Guide"? "By design, Mac OS X does not allow applications to send SCSI or ATA commands to storage devices unless the application developer also provides an in-kernel device driver that supports the commands." One of the intents if this is to allow the driver to model state for such coomands such that the state if the device is known to the driver at all times. For standard tape devices, you should not need to write your own driver; if you plug the device in, it will show up in /dev, Never seen a tape drive, neither SCSI, FireWire, USB nor FibreChannel connected to Mac OS X 10.2 up to 10.5 which shows up in /dev. What device filename should I look for? and you can at least issue reads, writes, and standard commands, such as causing a rewind, via the device file. For non-standard commands, such as you would use to control loading and unloading tapes in a tape robot, etc., you will need to write an in-kernel driver that knows how to talk to the hardware and subclasses the more generic device (i.e. the IOKit model lets you write only the missing pieces implementing the missing control commands; you don't write a whole driver that implements read/write/etc.). Instead we use SCSITaskDeviceInterface functions from userland as described in Apple's SCSI Architecture Model Device Interface Guide "WorkingWithSAM.pdf" in order to implement a "SCSI pass-thru" functionality. No additional driver required. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Sending SCSI commands in Mac OS X
by rohan a 23 Jul '09

23 Jul '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=I1qr60sJs1D4MLDDqvtz2Enxc9GC0lCbLJr/y1zNQTU=; b=tELW090HNC/wdeW3xt01SIfS7ANZYn8lrH6/Rb3eVgpUdkpm88hG6OScAYn+dpPbk4 LbIE+mzvWD5cps5+AYt6YR9XqGY+5H8uerdhsJfT2tua0+XCJ9DjO2X7QfTPm0ahCXFX M3uTVDuw4Bx0eOQiHqG5R1e8U4VPZyyjIA0fk= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=cWxckY1JFVBjWyhMDe3NGgazfK8SJ7E9usoc8kNhd3biIRMK530LrNj8629+i3dLu3 rC95B8YpTnwrKQmsXYC3DGtBfhRUio1I1DHoeuRPHaomRSzlSupETkZNiiiSU5vf609P pZnkV5BWg9+zcGezOXYvH6M6WXa8PijQgvaGc= Will an ioctl() enable me to do this ? Hence, any SCSI commands on a standalone device can be sent without the need for a seperate driver? On 7/23/09, Terry Lambert <tlambert(a)apple.com> wrote: > Did you go to <http://developer.apple.com> and read the document "SCSI > Architecture Model Device Interface Guide"? > > It states: > > "By design, Mac OS X does not allow applications to send SCSI or ATA > commands to storage devices unless the application developer also provides > an in-kernel device driver that supports the commands." > > One of the intents if this is to allow the driver to model state for such > coomands such that the state if the device is known to the driver at all > times. > > For standard tape devices, you should not need to write your own driver; if > you plug the device in, it will show up in /dev, and you can at least issue > reads, writes, and standard commands, such as causing a rewind, via the > device file. For non-standard commands, such as you would use to control > loading and unloading tapes in a tape robot, etc., you will need to write an > in-kernel driver that knows how to talk to the hardware and subclasses the > more generic device (i.e. the IOKit model lets you write only the missing > pieces implementing the missing control commands; you don't write a whole > driver that implements read/write/etc.). > > -- Terry > > > On Jul 23, 2009, at 3:10 AM, rohan a <info1686(a)gmail.com> wrote: > > > > > Also, can ioctl() be used in any way to send SCSI commands to devices? > > > > On 7/23/09, rohan a <info1686(a)gmail.com> wrote: > > > > > Hello, > > > I am trying to learn SCSI programming on Mac OS X. > > > While reading a document "Working with Sam" I read that only INQUIRY > > > and TEST_UNIT_READY commands can be sent using the IO registry. > > > Does that mean even sending SCSI Read or Write commands is not > > > possible without writing a driver for the device? > > > > > > So if I need to write a backup/restore tool that backups to SCSI tape > > > devices, will I also need to write drivers for the tape? > > > > > > Thanks > > > > > > > > _______________________________________________ > > Do not post admin requests to the list. They will be ignored. > > Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) > > Help/Unsubscribe/Update your Subscription: > > > http://lists.apple.com/mailman/options/darwin-dev/tlambert%40apple.com > > > > This email sent to tlambert(a)apple.com > > > _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Sending SCSI commands in Mac OS X
by Terry Lambert 23 Jul '09

23 Jul '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com It states: -- Terry On Jul 23, 2009, at 3:10 AM, rohan a <info1686(a)gmail.com> wrote: Also, can ioctl() be used in any way to send SCSI commands to devices? Hello, I am trying to learn SCSI programming on Mac OS X. While reading a document "Working with Sam" I read that only INQUIRY and TEST_UNIT_READY commands can be sent using the IO registry. Does that mean even sending SCSI Read or Write commands is not possible without writing a driver for the device? So if I need to write a backup/restore tool that backups to SCSI tape devices, will I also need to write drivers for the tape? Thanks _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/tlambert%40apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Did you go to <http://developer.apple.com> and read the document "SCSI Architecture Model Device Interface Guide"? "By design, Mac OS X does not allow applications to send SCSI or ATA commands to storage devices unless the application developer also provides an in-kernel device driver that supports the commands." One of the intents if this is to allow the driver to model state for such coomands such that the state if the device is known to the driver at all times. For standard tape devices, you should not need to write your own driver; if you plug the device in, it will show up in /dev, and you can at least issue reads, writes, and standard commands, such as causing a rewind, via the device file. For non-standard commands, such as you would use to control loading and unloading tapes in a tape robot, etc., you will need to write an in-kernel driver that knows how to talk to the hardware and subclasses the more generic device (i.e. the IOKit model lets you write only the missing pieces implementing the missing control commands; you don't write a whole driver that implements read/write/etc.). On 7/23/09, rohan a <info1686(a)gmail.com> wrote: This email sent to tlambert(a)apple.com This email sent to site_archiver(a)lists.apple.com

1 0

Re: Sending SCSI commands in Mac OS X
by rohan a 23 Jul '09

23 Jul '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=9LojEHaBtB6wdV6MuRnBPqU8g+6BP1/bO5MGPIqNhW0=; b=K+bRSwcEY5oXKKeOOcvDp55tfeR2NyKYu6HPrdRcAzXPb3Oxs2q2lWjnIkNw208XEz pYQ8gQZQTQGUJpotGy7rDe4Ejy4NvIAHyvtpuIqHDyRSuqWgFAvdJq8KmjKnJBEdTVkV 15rnTswhPq43S2YSKYGqGElnLvQcIpKZBtzvY= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=qNJOJ3BnhXpeK55p0/n+9q0lamiRoa7c4x273fmXvEYwueMseoC2BGaH3USQzPgF9O pOw1uPR4uJSMUd2O2gedpYJvrBqQr/ntU6hbnepunDa3BNeoQ8dZiAlz0xgqf1N3gSNU qLfC3uhpohvar4NaXCIGqT6W8QQD7x8Q0vf80= Also, can ioctl() be used in any way to send SCSI commands to devices? On 7/23/09, rohan a <info1686(a)gmail.com> wrote: > Hello, > I am trying to learn SCSI programming on Mac OS X. > While reading a document "Working with Sam" I read that only INQUIRY > and TEST_UNIT_READY commands can be sent using the IO registry. > Does that mean even sending SCSI Read or Write commands is not > possible without writing a driver for the device? > > So if I need to write a backup/restore tool that backups to SCSI tape > devices, will I also need to write drivers for the tape? > > Thanks > _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Sending SCSI commands in Mac OS X
by rohan a 23 Jul '09

23 Jul '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type:content-transfer-encoding; bh=9Gg02i7SqYEjdxAPa7l/Mt852HID92i0x8JdzA6sJD4=; b=nGMWQJhJZuN1fvT75Kwq9b6YAyhNRM8A7l+xm7DtQZgsH7RvXg+H/n35D98ATZknPx 9ztQ3HhL1jKsd3TC6gWxDDMcETz2f9kX0ep8WMSz2A8EEYOEGsRb3gy1XWF+FBafSpHq isPyDmLS8e66XLC+HXiAT9QVNzg0/2FEeNd/k= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type :content-transfer-encoding; b=ogaG1/Evyeu7/GB1Ai+f9bcO/0IHHhm926j2SwcdesfMZ6BNV7exJhqatuzYxLMSZe BEBFRn+QhDpqOWh4/eTViHnrwICFApSnl8ftFu1HeWWS+8eAezHJJzMCuGhvRg8KlelM xkG8TbX+Qg/Pkgf6RkE5eLDcXXfUMFqsZextg= Hello, I am trying to learn SCSI programming on Mac OS X. While reading a document "Working with Sam" I read that only INQUIRY and TEST_UNIT_READY commands can be sent using the IO registry. Does that mean even sending SCSI Read or Write commands is not possible without writing a driver for the device? So if I need to write a backup/restore tool that backups to SCSI tape devices, will I also need to write drivers for the tape? Thanks _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Algorithm for deciding when to wake a drive?
by David Elliott 22 Jul '09

22 Jul '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Hi, On Jul 21, 2009, at 10:19 PM, Soren Spies wrote: On Jul 21, 2009, at 19:00, Andre-John Mas wrote: -Dave _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… I have a user complaining that one of his hard-disks is spinning up when he is not expecting it. What algorithm does Darwin/MacOS X use to decide when to wake a drive? For example is it when the file system is accessed or a specific volume, or does it vary based on certain conditions? Basically, any I/O to the drive that is not in the cache will cause it to spin up. Unfortunately, much code (including in Apple's frameworks) likes to call statfs() on each and every filesystem any time an Open/Save dialog comes up, an application launches, etc. I can't remember for sure, but in my experience this often seems to spin drives up. One other notable thing to check are the "Places" within the finder. If you have any folder references stashed in there then you might need to remove them. A potential workaround is to make a folder called Links or something like that in your home directory and put that in Places. Inside of that put normal aliases to your oft-used folders located on other drives. This was causing me all kinds of problems when I had a folder from an NFS-mapped drive in there. The eventual workaround was to use the server's FQDN and make sure it would not resolve outside of my home network. But even then I found I had little delays doing just about anything in Finder so I finally took the link out of my Places and things are a lot better now. I believe one of the Finder enhancements with Leopard was that it is now "multithreaded" in the sense that it at least won't hang the UI when trying to get at network drives or spun-down drives. Instead it will show the spinner inside the window while it waits. But it still apparently WILL hang the UI if any folders on these drives are referenced in the Places section. That code path seems to still be waiting on results before updating the UI. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Algorithm for deciding when to wake a drive?
by Soren Spies 22 Jul '09

22 Jul '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On Jul 21, 2009, at 19:00, Andre-John Mas wrote: -- Soren Spies CoreOS::IOTeam _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… I have a user complaining that one of his hard-disks is spinning up when he is not expecting it. What algorithm does Darwin/MacOS X use to decide when to wake a drive? For example is it when the file system is accessed or a specific volume, or does it vary based on certain conditions? Basically, any I/O to the drive that is not in the cache will cause it to spin up. Unfortunately, much code (including in Apple's frameworks) likes to call statfs() on each and every filesystem any time an Open/Save dialog comes up, an application launches, etc. I can't remember for sure, but in my experience this often seems to spin drives up. sudo fs_usage -w -f filesys will show which filesystem calls are occurring and the elapsed time in the third to last column might allow your user to figure out which system call had to wait for the disk to spin up. It may then be possible to use dtrace to figure out the stack trace leading to the I/ O and thus roughly where to assign blame / how to reproduce the problem. Feel free to report a bug to Apple; it's always good for us to hear how users are impacted by the behaviors of our systems. This email sent to site_archiver(a)lists.apple.com

1 0

Algorithm for deciding when to wake a drive?
by Andre-John Mas 22 Jul '09

22 Jul '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Hi, André-John _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… I have a user complaining that one of his hard-disks is spinning up when he is not expecting it. What algorithm does Darwin/MacOS X use to decide when to wake a drive? For example is it when the file system is accessed or a specific volume, or does it vary based on certain conditions? This email sent to site_archiver(a)lists.apple.com

1 0

Re: What is exactly read_only_reloc
by Jean-Daniel Dupas 22 Jul '09

22 Jul '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Le 21 juil. 09 à 20:45, Terry Lambert a écrit : Hello, Regards Jean-Daniel -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On Jul 21, 2009, at 8:35 AM, Jean-Daniel Dupas <devlists(a)shadowlab.org> wrote: I'm not sure this is the most appropriate list to ask (and I know the noise/information ratio is pretty high those days ;-) ) but I'm hope the ABI specialists here may help me. Some unix project (with hand-coded asm) require to pass '- read_only_reloc suppress' to the linker to link as dynamic library. I wonder what this mean exactly, and what are the drawback of this flags. I didn't found any information about it, but the ld man page. And I wonder too if there is a better way to solve this issue (what may be change in the asm to avoid linker error). It won't link as a dynamic library without it because you have relocations that apparently live in a read-only text segment. Normally these, if generated by the compiler rather than coded by hand, would be in a read/write segment mapped copy-on-write, which is why the man page for ld claims the compiler will never generate code like this. Without that, you have to suppress so that the page the relocations are in will get remapped by dyld and modified anyway, as needed to use them. This has security implications from two perspectives: (1) since it's in a text page, if the code in the page has a code signature, you're going to make it invalid by dirtying the page, and (2) since type of mapping is only attributable on page boundaries, you're going to open up whatever also happens to be in the pages with the relocs to also being modified, along with the relocs themselves. Depending on your exact use of memory you could also open yourself to stack execution attacks, etc.. Depending on your code signing enforcement settings, the code might decide to simply not work and abort fowwing the mapping change or the page getting dirtied. The easiest way to deal with relocs is to write your code in C/C++/ ObjC instead of hand-coded assembly. If you must deal with it from an assembly perspective, you can use assembler directives to move the relocations into a separate section. The easiest way to know exactly what to do to do that is to write a short chunk of C code that does something similar and thinks it's going to live in a dynamic library so it also has relocs, and then ask the compiler to generate assembly from the C to see what the compiler does in the assembly it generates so you can know what to do in yours. Also realize the contracts between the compiler, assembler, and linker can and do change over time, so your code will be subject to "bit rot" any time those move out from under you. FWIW, unless you are doing unaligned structure assignments or other cases of degerate code, it's pretty rare these days for a human to be able to out-code a compiler in assembly at higher optimization levels, so I was only half-joking about writing the code in C instead. Thank you very much for this detailed answer. In fact, this is not my code, this is code from the ffmpeg project. This is not the first project with ASM I have to deal with, and each time it brings more trouble than benefit. C is so close to ASM that I often wonder how hand written ASM could be "so much faster" than compiled code. Maybe the answer is just that it's not. In the previous case, the assembly is used to optimize some codec (especially H.264). The asm version is faster mainly because it uses SSE extensions but it can be done in C AFAK. If I have some time, I will try to follow you advice and to rewrite the annoying function in C. If there is too much difference between the two versions, there is something else I would like to try. This is to write a version using llvm assembly instead of x86 or PPC ASM. It may be interesting to compare the generated code to the original version. If I understand correctly, an llvm assembly version can be used to generate either PIC or non relocatable code. And it would be portable :-) This email sent to site_archiver(a)lists.apple.com

1 0

Re: What is exactly read_only_reloc
by Terry Lambert 21 Jul '09

21 Jul '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Hello, Regards Jean-Daniel -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On Jul 21, 2009, at 8:35 AM, Jean-Daniel Dupas <devlists(a)shadowlab.org> wrote: I'm not sure this is the most appropriate list to ask (and I know the noise/information ratio is pretty high those days ;-) ) but I'm hope the ABI specialists here may help me. Some unix project (with hand-coded asm) require to pass '- read_only_reloc suppress' to the linker to link as dynamic library. I wonder what this mean exactly, and what are the drawback of this flags. I didn't found any information about it, but the ld man page. And I wonder too if there is a better way to solve this issue (what may be change in the asm to avoid linker error). It won't link as a dynamic library without it because you have relocations that apparently live in a read-only text segment. Normally these, if generated by the compiler rather than coded by hand, would be in a read/write segment mapped copy-on-write, which is why the man page for ld claims the compiler will never generate code like this. Without that, you have to suppress so that the page the relocations are in will get remapped by dyld and modified anyway, as needed to use them. This has security implications from two perspectives: (1) since it's in a text page, if the code in the page has a code signature, you're going to make it invalid by dirtying the page, and (2) since type of mapping is only attributable on page boundaries, you're going to open up whatever also happens to be in the pages with the relocs to also being modified, along with the relocs themselves. Depending on your exact use of memory you could also open yourself to stack execution attacks, etc.. Depending on your code signing enforcement settings, the code might decide to simply not work and abort fowwing the mapping change or the page getting dirtied. The easiest way to deal with relocs is to write your code in C/C++/ ObjC instead of hand-coded assembly. If you must deal with it from an assembly perspective, you can use assembler directives to move the relocations into a separate section. The easiest way to know exactly what to do to do that is to write a short chunk of C code that does something similar and thinks it's going to live in a dynamic library so it also has relocs, and then ask the compiler to generate assembly from the C to see what the compiler does in the assembly it generates so you can know what to do in yours. Also realize the contracts between the compiler, assembler, and linker can and do change over time, so your code will be subject to "bit rot" any time those move out from under you. FWIW, unless you are doing unaligned structure assignments or other cases of degerate code, it's pretty rare these days for a human to be able to out-code a compiler in assembly at higher optimization levels, so I was only half-joking about writing the code in C instead. This email sent to site_archiver(a)lists.apple.com

1 0