February 2006 - Darwin-dev - lists.apple.com

intel source code release
by Peter O'Gorman 17 Feb '06

17 Feb '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com User-agent: Mozilla Thunderbird 1.0.2 (Macintosh/20050317) Thanks for listening, Peter _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Hi, Some of you may recognize me as having started a similarly themed thread <http://lists.apple.com/archives/Darwin-dev/2005/Jun/msg00136.html> a year ago. Well, today, I was looking for the source code to the 10.4.x linker on Intel. So I went to the <http://darwinsource.opendarwin.org> website, fully expecting to look at the cctools sources and see what differences in behavior to expect between Intel and ppc. I expected the linker sources to be available, because they have always been available, and because cctools contains the assembler which is based on GNU as, and thus under the terms of the GPL. I was amazed to find that the gas sources had been split out of cctools, so they could be provided in accordance with the GPL, but no other part of cctools was made available. So I never did get an answer to my question. I assume that the idea is to limit the source code availability to those who are attempting to steal Mac OS X and use it on systems not built or approved by Apple. I can understand and applaud the goal, but not the methods. By limiting published source code to that which is "infected" by the GPL, Apple is, in my honest opinion, scoring an own goal. Let me give some examples. Without cctools source code, Shantonu Sen's wonderful odcctools project will die. Nobody will be able to build a to darwin gcc cross compiler anymore. <http://www.opendarwin.org/projects/odcctools> dlcompat could never have been written, as it uses "internal" APIs from dyld <http://darwinsource.opendarwin.org/10.3.9/cctools-573.1/dyld/dyld_libfuncs.c> <http://www.opendarwin.org/projects/dlcompat> Were it not for Apple publishing their changes to the perl project, perl upstream would never have known about Apple's changes to perl for tiger. <http://www.mail-archive.com/perl5-porters@perl.org/msg86873.html> Now, those are only things that I know about/can remember right now, I'm sure that there are many other examples. I guess that kext developers will now have a significantly harder time when something makes them say "wtf!" there will be no sources to check. It's not only very sad, it is quite unnecessary to have such a blanket ban on publishing source code. Userland sources that come from *BSD should be published, as should projects like perl, python that have relaxed licensing, but in no way affect anyones ability to run Mac OS X on their Dell. If possible, I'd advocate publishing obfuscated kernel sources too. According to <http://developer.apple.com/documentation/MacOSX/Conceptual/OSX_Technology_O…>, consists of Core OS, Core Services, Application Services, Graphics & Multimedia, Application Environments, and, User Experience. Of these, Core OS has previously had the source code published on each release of the operating system. This means that the source code that Apple used to publish and allow people to browse, in the main, comes from opensource projects. While many project's licences do not require that changes be sent back or source code published, I think it only fair play. This email sent to site_archiver(a)lists.apple.com

1 0

Re: mmap alignment.
by Jim Magee 16 Feb '06

16 Feb '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On Feb 14, 2006, at 12:00 AM, Theo Schlossnagle wrote: On Feb 13, 2006, at 11:31 PM, Mike Smith wrote: I'm porting an app that uses mmap() and banks on its alignment behaviour on Solaris and Linux. MAP_ALIGN is not supported on Mac OS X. Thanks everyone for the replies. --Jim _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On Feb 13, 2006, at 3:04 PM, darwin-dev-request(a)lists.apple.com wrote: I'd like to mmap a 64k and have the resulting address be aligned on a 64k boundary. I'm trying with: mmap((void *)65536, 65536, PROT_NONE, MAP_PRIVATE | MAP_ANON | MAP_NORESERVE | MAP_ALIGN, -1, 0); This is almost always guaranteed to fail, as you are trying to map *at* 64KB, and normally you will find your program's text mapped around there. However, the return address is not modulo 64k. How do I accomplish what I want? Well now, what is it, specifically, that you think you want? Why is a virtual address modulo 64K important? (Pointer overloading?) One approach would be to use vm_allocate to allocate an N + 64K sized chunk, round the base address up to the next boundary, free the chunk and then mmap with that address as a hint. Unless you have activity elsewhere (on e.g. another thread) in your app, that's pretty safe). What I am attempting to do it port Solaris' super-scalable slab sllocator (libumem) to Darwin. We have a product that runs ons on a variety of architecture that leverages a custom allocator for performance and uniformity. The Solaris would seems a bit better than ours and we wanted to adopt it while still having a consistent allocator across all platforms. Using the vm_map() directly may be one approach. We already use pthread_mach_thread_np in part of the porting effort. As it will be allocating a tremendous amount of memory and efficient use is a goal, over-allocating and doing alignment ourselves isn't the right approach. Note that over-allocating and aligning only wastes virtual space. Physical memory is not allocated for the "alignment slop." At least not in most normal uses. But if you are already using Mach interfaces, then using vm_map()'s alignment mask parameter is probably a good way to go. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Using too much memory crashes entire machine
by Tim Seufert 16 Feb '06

16 Feb '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On Feb 16, 2006, at 2:50 PM, Brian Bechtel wrote: Now, I would be willing to blame this on bad memory, but the machine runs weeks on end performing other heavy computational tasks (although in 32-bit mode). The same problem also occurs on a separate dual G5 with 8 GB of memory. panic(cpu 2 caller 0x000A8D80): Uncorrectable machine check: pc = FFFFFFFFFFFF8260, msr = 900000000004F030, dsisr = 00200000, dar = 00000000B0011C00 Your RAM is incompatible with your machine, or you have a bad RAM DIMM. Run the Apple Hardware Test which came with the machine. In 32-bit mode, your application can only access 4GB of RAM. In 64-bit mode, it's accessing previously unused RAM; accessing that RAM is causing the machine check panic. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On 2/15/06, Tor Andre Myrvoll <myrvoll(a)iet.ntnu.no> wrote: The problem is that if I let the program use 6-7 GB out of my total of 8 GB, the machine will eventually crash (see an example of a crash- dump below. The cpu causing the panic varies from time to time). Note that the machine is not swapping when these crashes occur. It's not necessarily that simple. The presence of a VM system and buffer cache in the OS which both understand 64-bit physical addresses means there is no such thing as RAM which is only available to a 64-bit process. Especially on Tiger, even if he never ran anything but 32-bit processes, the system could easily use all available physical memory. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Using too much memory crashes entire machine
by Brian Bechtel 16 Feb '06

16 Feb '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=DM6WPxudI8DqNd6sWtY0dsnMMC/gobVvATQllV+gTuSX3C7POmSECqJY66EiMwzW8paWe3Mba6E8jNVeZaSFuPnBeVH1TJPTuBFtmz4IbIgQPa6FS6y48ulZHw4NRHU4Kq+iuHekD0wIKNPc12QXF0ml+BAQf2bmbe1e2YZyN1g= On 2/15/06, Tor Andre Myrvoll <myrvoll(a)iet.ntnu.no> wrote: > The problem is that if I let the program use 6-7 GB out of my total > of 8 GB, the machine will eventually crash (see an example of a crash- > dump below. Having said all that, I found that there is a bug (fixed in 10.4.5) which was reported in MySQL <http://bugs.mysql.com/bug.php?id=13084> which may be relevant. Try upgrading to 10.4.5 and see if the problem goes away. The Apple bug number was 4280976. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Using too much memory crashes entire machine
by Brian Bechtel 16 Feb '06

16 Feb '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=URKRvqgt48FEb9jA65sXGmEZydJ0yCP2KymFq3dfx0hFnJnoMt851phQR/+Ehyab9HC6MT74RnWYmehPZB4v/2o5T5v2hfWoWRYRZKwwcQYCFswhaTUbdlmXeGqVGkd2ZeW5PWiasQo8awI4o5mRt6u8TnDGK+f3pPGP+tDELo8= On 2/15/06, Tor Andre Myrvoll <myrvoll(a)iet.ntnu.no> wrote: > The problem is that if I let the program use 6-7 GB out of my total > of 8 GB, the machine will eventually crash (see an example of a crash- > dump below. The cpu causing the panic varies from time to time). Note > that the machine is not swapping when these crashes occur. > > Now, I would be willing to blame this on bad memory, but the machine > runs weeks on end performing other heavy computational tasks > (although in 32-bit mode). The same problem also occurs on a separate > dual G5 with 8 GB of memory. > panic(cpu 2 caller 0x000A8D80): Uncorrectable machine check: pc = > FFFFFFFFFFFF8260, msr = 900000000004F030, dsisr = 00200000, dar = > 00000000B0011C00 Your RAM is incompatible with your machine, or you have a bad RAM DIMM. Run the Apple Hardware Test which came with the machine. In 32-bit mode, your application can only access 4GB of RAM. In 64-bit mode, it's accessing previously unused RAM; accessing that RAM is causing the machine check panic. RAM specifications for the PowerMac G5 machines can be found at <http://docs.info.apple.com/article.html?artnum=86414> and <http://developer.apple.com/documentation/Hardware/Developer_Notes/Macintosh…> _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Alternative "dead port listener" implementations under intel?
by Timothy J. Wood 16 Feb '06

16 Feb '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On Feb 15, 2006, at 2:39 PM, Jeffrey Johnson wrote: Hello, -tim _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Now that the use of task_for_pid() on intel boxes has been tightened to use by root or processes belonging to procmod: <http://developer.apple.com/documentation/MacOSX/Conceptual/ universal_binary/index.html#//apple_ref/doc/uid/TP40002217> those of us relying on the helpful snippet in TN2050 have been chopped off at the knees. Does anyone have a workaround for getting notified when a process terminates without resorting to polling? (I only care about processes owned by my UID.) In addition to the kqueue suggestion (which is better for this situation, probably), there are bootstrap namespaces. OmniObjectMeter, for example, needs to be able to scrounge though the target process's memory. We do this by setting up a private bootstrap namespace and then having the target process register its task port there-in under a well-known name. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Alternative "dead port listener" implementations under intel?
by Dave Zarzycki 15 Feb '06

15 Feb '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com kqueue. Use EVFILT_PROC. davez On Feb 15, 2006, at 2:39 PM, Jeffrey Johnson wrote: Hello, Thanks Jeffrey Johnson Macintosh Development Wavefunction, Inc. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/zarzycki%40apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Now that the use of task_for_pid() on intel boxes has been tightened to use by root or processes belonging to procmod: <http://developer.apple.com/documentation/MacOSX/Conceptual/universal_binary… > those of us relying on the helpful snippet in TN2050 have been chopped off at the knees. Does anyone have a workaround for getting notified when a process terminates without resorting to polling? (I only care about processes owned by my UID.) This email sent to zarzycki(a)apple.com This email sent to site_archiver(a)lists.apple.com

1 0

Alternative "dead port listener" implementations under intel?
by Jeffrey Johnson 15 Feb '06

15 Feb '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Hello, Now that the use of task_for_pid() on intel boxes has been tightened to use by root or processes belonging to procmod: <http://developer.apple.com/documentation/MacOSX/Conceptual/universal_binary…> those of us relying on the helpful snippet in TN2050 have been chopped off at the knees. Does anyone have a workaround for getting notified when a process terminates without resorting to polling? (I only care about processes owned by my UID.) Thanks Jeffrey Johnson Macintosh Development Wavefunction, Inc. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Protected Memory
by John Davidorff Pell 15 Feb '06

15 Feb '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com JP On 14 Feb 2006, at 14:51, William Riley-Land wrote: So, my main question is where can I find that sort of information? _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This is definitely possible, and relatively easy* to do. However, it is quite dangerous. You have a couple of options: 1) Load the code into the target application at launch (using some sort of plugin), or inject code into another running application at some later time. To do the first, you should ask the APE or SIMBL people how they do it. WARNING: This is a good way to get a huge number of people to hate your software, since it will potentially damage other unrelated applications. i.e. your code will not be loaded into the target application, but rather into *every* application. 2) Use mach_inject/mach_override. This is where Alex's security hole comes in. the microkernel for Mac OS X/Darwin allows for running applications to start execution threads (mach threads, not posix threads) in a task other than its own (another application). WARNING: This is a good way to totally hose the target application, but won't affect anything else. WARNING: This will break with (more or less) every major Darwin/Mac OS X release. WARNING: This may be a security hole and will be patched eventually. Technically speaking, this is not a security hole since it cannot be used to escalate privileges. All it can do is allow a user to alter his/her/its own processes. Since a user owns his/her/its own processes, this is not security related. It does violate "protected memory", but "protected" means that one application cannot unintentionally corrupt another's workspace, it has nothing to do with security. See http://rentzsch.com/mach_inject and http://rentzsch.com/mach_override for more information. P.S. If any of the above is a security risk (which its not), then keeping it secret (but freely available elsewhere) is totally meaningless. P.P.S. *When I say easy, I mean ridiculously obtuse and prohibitively complex. Hi, I haven't posted here before. So, nice to meet you all and hope this isn't off topic :) I was wondering if it is possible at all for one userland application to access another application's memory space. By "userland" I mean non-kernel code... that's the correct term right? I had a program under OS 9 that could be used to do this (mostly to cheat at games :) I understand that under Darwin memory is "protected" and one application can only use its memory space and/or shared memory (which I have no idea about at all). Anyway, my secondary question is: can one application access another's memory? I am dissapointed that I have to ask this question, but I could not find any sort of in-depth documentation of Darwin's memory management on the Apple site or after doing a bit of Googling... -- -william _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/jpell.lists% 40mac.com This email sent to jpell.lists(a)mac.com -- "... was it a dream where you see yourself standing in sort-of Sun- God robes, on a pyramid, with a thousand naked women screaming and throwing little pickles at you? ... Why am I the only one who has that dream?" This email sent to site_archiver(a)lists.apple.com

1 0

Re: Protected Memory
by Peter Bierman 15 Feb '06

15 Feb '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Am 14.02.2006 um 23:51 schrieb William Riley-Land: So, to the community: SHOULD one be able to do that? Alex -pmb _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… At 11:59 AM +0100 2/15/06, Alexander von Below wrote: Hi, I haven't posted here before. So, nice to meet you all and hope this isn't off topic :) I was wondering if it is possible at all for one userland application to access another application's memory space. By "userland" I mean non-kernel code... that's the correct term right? I had a program under OS 9 that could be used to do this (mostly to cheat at games :) Yes it is possible. I am somewhat hesitant to post the code here, because I am not sure if it is a "security leak" . The snippet I have was originally intended for the 21c3 (1) (1) http://www.ccc.de/congress/2004/index.en.html If you think you have discovered a "security leak" or similar flaw with Apple software, please contact <product-security(a)apple.com> with details. Apple engineers will investigate and can work with you to develop a responsible publication schedule if a real issue exists. This email sent to site_archiver(a)lists.apple.com

1 0