April 2008 - Darwin-kernel - lists.apple.com

Named Semaphorse
by Duane Murphy 24 Apr '08

24 Apr '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Is it possible to use named semaphores to "communicate" from the kernel to user space? We have an application where on another platform (Windows), the kernel module uses a named semaphore to trigger a user space application. Is there anything like that for Mac OS X? Ideas, recommendations? Thanks, ...Duane _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Rosetta and Code Injection
by Bob Murphy 24 Apr '08

24 Apr '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com - Bob _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… When I launch Word, though, both ps -Ax and programmatic diagnostics show only show Word itself, not anything that looks like a separate Rosetta process. There isn't a "separate Rosetta process". There is just an instance of the Rosetta interpreter trying its best to look like Word (in your case). However, attempts to inject x86 code into the process reported as Word result in immediate crashes. So it appears to me there's more going on than just Rosetta being a plain ol' x86 process interpreting some stuff on disk that happens to be a PPC executable, and doing endian swapping and thunking system calls and so on. But I imagine that's part of the Rosetta "secret sauce" and not something that can really be discussed outside of Apple and Transitive. However, I feel I owe it to the people paying me to examine the possibility that there is some official, Apple-supported way to do code-inject a PPC executable running under Rosetta, or that someone else has succeeded in doing this, before rushing off to do what I consider a dubious hack. And I thought there might be a faint chance that Apple might, for some odd reason, support this by some means I hadn't considered. No supported way (even if you weren't running under Rosetta). <sigh> The story of my life. I find that If I pick my challenges a little more carefully, I find fewer disappointments. 8) Nothing ventured, nothing gained. In addition to the disappointments, there is the occasional real triumph. p.s. If you're not patching the code dynamically at runtime, what stops you from patching the PPC executable on-disk? Just a thought... Alas, I am indeed patching the code dynamically at runtime. That's how mach_star achieves under OS X what used to be done via INITs and trap patches on 680x0 Macs. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Rosetta and Code Injection
by Bob Murphy 24 Apr '08

24 Apr '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com You don't. - Bob _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… So how does one distinguish the Rosetta process from the PPC surrogate? Let me put this pretty bluntly... how would you like to be the poor engineer catching bug reports because Joe Blows thread that he ran in your address space happened to overwrite something critical because Joe Blow made a mistake in his code? How much worse would this be if someone were giving out cookbooks to people who otherwise were beginning coders? I know how hazardous this is. Even worse than causing crashes, it could be a security hole a mile wide. Frankly, I wasn't expecting to get answers from Apple engineers, or much of any real discussion on the list. I just posted here because I thought maybe some third party developers who knew about code injection might hang out on this list, and email me privately. Shipping commercial and open-source apps have been using code injection and the underlying kernel routines for years. I'm just trying to fix one of those apps so it works with Rosetta. If there were another, safer way to accomplish this, I'd happily use it. However... 1. Apple has a security model that prevents arbitrary apps from doing code injection. So it's safer than it could be, and the user gets some warning when they have to type in a supervisor password to get the app to work. 2. As a professional Mac developer since 1985, I've been that "poor engineer catching bug reports" many, many times. The last thing I want to do is write buggy injection code that causes any poor engineer grief - whether it's myself or someone else. 3. The app I'm working on is well-established, and its users are quick to report any crashes or odd behaviors. Now ask yourself again why this is unsupported... Oh, I already knew! :-) I sit on your side of the fence in my other life, working on a Linux-based cell phone platform. So I get to tell people why they can't do things like render directly to the hardware frame buffer (the X server owns it exclusively), no matter how important the frame rate in their game is. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Rosetta and Code Injection
by Terry Lambert 24 Apr '08

24 Apr '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com You don't. Now ask yourself again why this is unsupported... -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… On Apr 23, 2008, at 6:37 PM, Bob Murphy wrote: So how does one distinguish the Rosetta process from the PPC surrogate? Let me put this pretty bluntly... how would you like to be the poor engineer catching bug reports because Joe Blows thread that he ran in your address space happened to overwrite something critical because Joe Blow made a mistake in his code? How much worse would this be if someone were giving out cookbooks to people who otherwise were beginning coders? This email sent to site_archiver(a)lists.apple.com

1 0

Re: Rosetta and Code Injection
by Bob Murphy 24 Apr '08

24 Apr '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Your fundamental premise here is wrong. So how does one distinguish the Rosetta process from the PPC surrogate? <sigh> The story of my life. On Apr 23, 2008, at 4:57 PM, Terry Lambert wrote: On Apr 23, 2008, at 11:32 AM, Bob Murphy wrote: However, mach_star uses routines like vm_allocate() and vm_write() and thread_create_running() that are implemented in xnu. I figured this list is a good place to ask about my problem because: a. The problem I'm having is with how the underpinnings of thread_create_running() operate in the kernel. What you are doing is more or less like succeeding sneaking into a bowling alley, even though you have no bowling ball or bowling shoes: whatever else happens, you're not going to be doing any bowling. When I was a kid in Ohio, I just paid for the games, used a ball off the freebie racks, and bowled in my socks. Rosetta should probably be thought of as a virtual machine or interpreter. The process you see in "ps", "lsof", "top", etc., is actually an instance of a Rosetta process, and not really a PPC process at all. The PPC "process" is the data on which the native Rosetta process operates. If I understand this, then in theory, I could take my PPC-only copy of MS Word 2004, and successfully execute an x86 thread inside its Rosetta parent. That would work for me, except it doesn't: Word crashes hard. The crash log has a mix of x86 and PPC information, so I gather that what I think I'm injecting into isn't the Rosetta process. When I launch Word, though, both ps -Ax and programmatic diagnostics show only show Word itself, not anything that looks like a separate Rosetta process. However, I feel I owe it to the people paying me to examine the possibility that there is some official, Apple-supported way to do code-inject a PPC executable running under Rosetta, or that someone else has succeeded in doing this, before rushing off to do what I consider a dubious hack. And I thought there might be a faint chance that Apple might, for some odd reason, support this by some means I hadn't considered. No supported way (even if you weren't running under Rosetta). 1986 exchange with MacDTS: "How can I write floating point code that directly accesses the 68881 FPU in our beta Mac II, rather than incur the overhead of SANE's rather thick wrapper?" "No supported way." (I hand-coded machine language FPU instructions and embedded them as data in the middle of an "asm" block in the C source.) _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Rosetta and Code Injection
by Terry Lambert 23 Apr '08

23 Apr '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Your fundamental premise here is wrong. No supported way (even if you weren't running under Rosetta). -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… On Apr 23, 2008, at 11:32 AM, Bob Murphy wrote: However, mach_star uses routines like vm_allocate() and vm_write() and thread_create_running() that are implemented in xnu. I figured this list is a good place to ask about my problem because: a. The problem I'm having is with how the underpinnings of thread_create_running() operate in the kernel. What you are doing is more or less like succeeding sneaking into a bowling alley, even though you have no bowling ball or bowling shoes: whatever else happens, you're not going to be doing any bowling. Rosetta should probably be thought of as a virtual machine or interpreter. The process you see in "ps", "lsof", "top", etc., is actually an instance of a Rosetta process, and not really a PPC process at all. The PPC "process" is the data on which the native Rosetta process operates. However, I feel I owe it to the people paying me to examine the possibility that there is some official, Apple-supported way to do code-inject a PPC executable running under Rosetta, or that someone else has succeeded in doing this, before rushing off to do what I consider a dubious hack. And I thought there might be a faint chance that Apple might, for some odd reason, support this by some means I hadn't considered. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Rosetta and Code Injection
by Bob Murphy 23 Apr '08

23 Apr '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Why am I not surprised? :-) On Apr 23, 2008, at 2:05 PM, Garth Cummings wrote: Hi Bob, On Apr 23, 2008, at 11:32 AM, Bob Murphy wrote: --gc ____________________________________________________________________ Garth Cummings gcummings(a)apple.com Sr. Software Engineer Apple Developer Technical Support <http://developer.apple.com/technicalsupport> WWDC 2008 • June 9-13 • Moscone West • San Francisco, CA <http://developer.apple.com/wwdc/> _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… official, Apple-supported way to do code-inject "Code inject" and "Apple-supported" are orthogonal. No lifeguard on duty. :-) This email sent to site_archiver(a)lists.apple.com

1 0

Re: Rosetta and Code Injection
by Garth Cummings 23 Apr '08

23 Apr '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Hi Bob, On Apr 23, 2008, at 11:32 AM, Bob Murphy wrote: --gc ____________________________________________________________________ Garth Cummings gcummings(a)apple.com Sr. Software Engineer Apple Developer Technical Support <http://developer.apple.com/technicalsupport> WWDC 2008 • June 9-13 • Moscone West • San Francisco, CA <http://developer.apple.com/wwdc/> _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… official, Apple-supported way to do code-inject "Code inject" and "Apple-supported" are orthogonal. No lifeguard on duty. :-) This email sent to site_archiver(a)lists.apple.com

1 0

Re: Rosetta and Code Injection
by Bob Murphy 23 Apr '08

23 Apr '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Hi Shantonu, Best regards, Bob _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This doesn't seem like a kernel issue, and Rosetta is not part of Darwin. You might have better luck contacting the author or development team of mach_star. It's a question of how to use kernel routines to do things not normally permitted to user-space programs, and what to do if the kernel doesn't contain functionality I need. Among other things, mach_star allows one MacOS X process to take a chunk of code out of bundle on disk, and spawn it as a thread in an arbitrary unrelated process. This can then be used for a variety of purposes, most commonly to do an OS X equivalent of Classic's trap patching for an individual app. As you can imagine, that's a rather dodgy activity, and something Apple doesn't exactly provide sample code for. :-) And with mach_star, there's nobody official you can contact to ask questions. mach_star is a more-or-less open-source project that nobody is actively maintaining, and that evolves when people hack on it and then post their modified sources somewhere. Then if the original author notices and feels like it, he merges those changes into his version. However, mach_star uses routines like vm_allocate() and vm_write() and thread_create_running() that are implemented in xnu. I figured this list is a good place to ask about my problem because: a. The problem I'm having is with how the underpinnings of thread_create_running() operate in the kernel. b. Given the number and types of kernel routines it uses, chances are good that anybody who is actively hacking on mach_star is reading this list. However, I will leave you with this thought experiment: Do you believe that ./osfmk/ppc/status.c is compiled into the Intel I386 kernel running on Intel-based Macintosh? I have thought the chances of this were almost zero. If nothing else, in the absence of something like C preprocessor juju, building both osfmk/ppc/status.c and osfmk/i386/pcb.c into the same executable would lead to multiply-defined symbols at link time. However, I feel I owe it to the people paying me to examine the possibility that there is some official, Apple-supported way to do code-inject a PPC executable running under Rosetta, or that someone else has succeeded in doing this, before rushing off to do what I consider a dubious hack. And I thought there might be a faint chance that Apple might, for some odd reason, support this by some means I hadn't considered. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Rosetta and Code Injection
by Shantonu Sen 23 Apr '08

23 Apr '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Shantonu Sen ssen(a)apple.com Sent from my Mac Pro On Apr 23, 2008, at 10:01 AM, Bob Murphy wrote: Here are a few things I've discovered: And any advice will be gratefully appreciated. - Bob _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/ssen%40apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This doesn't seem like a kernel issue, and Rosetta is not part of Darwin. You might have better luck contacting the author or development team of mach_star. However, I will leave you with this thought experiment: Do you believe that ./osfmk/ppc/status.c is compiled into the Intel I386 kernel running on Intel-based Macintosh? Has anybody gotten Jonathan Rentzsch's mach_star system to work with Rosetta? The latest versions of mach_star, modified by Bertrand Guihéneuf and others, work fine if the target process's executable is a PPC exe running on a PPC chip, or an x86 exe running on an x86 chip. However, code injection doesn't work for a PPC exe running under Rosetta. I've been trying to get this to work, but have run into some problems, and was hoping that either somebody has succeeded at this, or can at least provide some pointers. - A PPC source exe that works fine at injecting particular PPC target exes on a PowerMac G5, fails when running in Rosetta on a MacBook. - Most of the injection process (allocating memory in the target process, etc.) succeeds. Where things fail is inside thread_create_running(). When you feed it a PPC_THREAD_STATE thread state flavor when running on an x86 CPU, it returns an error code of KERN_INVALID_ARGUMENT. This happens no matter whether the calling process is an x86 or PPC exe. - It looks like things are failing in xnu, in machine_thread_set_state(). This is a routine that sets up the thread based on register contents and so on. In recent kernels, there are two versions of this, one for x86 (osfmk/i386/pcb.c) and one for PPC (osfmk/ppc/status.c). Each of these has a switch block that handles the allowable thread state flavors for the hardware. And if you try to use a thread state flavor for PPC in an x86 kernel (or vice versa), the switch block hits the default and returns KERN_INVALID_ARGUMENT. There may be other places code injection on Rosetta will also fail; I haven't determined that yet. The only solution I've been able to think of so far is to abstract the PPC thread setup code from osfmk/ppc/status.c and do it in my injector. But if there's an easier, or proven, way to accomplish the task, that would be better. This email sent to ssen(a)apple.com This email sent to site_archiver(a)lists.apple.com

1 0