April 2008 - Darwin-dev - lists.apple.com

Re: mach-o section question
by Army Research Lab 17 Apr '08

17 Apr '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Thread-index: AcighsmxCD3JNgx6Ed2lgQAdT0T19A== Thread-topic: mach-o section question User-agent: Microsoft-Entourage/11.4.0.080122 Thank you for the explanation and the link! But, as you say in your post below, not all sections are listed in that link. Is there a single global document that lists everything that you can run across in a mach-o file? Failing that, is there an EBNF grammar for what the 'nm' command produces? My scrapper now works for everything I've seen on the command line so far, but it is quite fragile, and there is no guarantee that I won't run across something new that breaks it. Thanks, Cem Karan On 4/16/08 1:30 PM, "David Elliott" <dfe(a)tgwbd.org> wrote: > Hi Cem, > > What you really want is the "Mac OS X Assembler Reference", > specifically "Directives for Designating the Current Section" > > http://devworld.apple.com/documentation/DeveloperTools/Reference/Assembler/… > Directives/chapter_5_section_2.html > > That doesn't explain absolute or common symbols. Absolute are pretty > easy to explain. With the assembler you can define a symbol with an > absolute address which is quite useful when you have some binary > interface that depends on this. > > For example, from a C file you could prototype extern void > _commpage_bcopy(const void *src, void *dst, size_t len); Then from an > assembler file you could specify a __commpage_bcopy symbol at exactly > 0xffff0780. You should not do this because that's not an official ABI > but if you do something like x/10i 0xffff0780 from gdb on an Intel mac > with an Intel program you will clearly see that this is bcopy. Or > rather _bcopy (in C, hence __bcopy from gdb). Whether Apple uses an > absolute symbol to achieve this or not is not the concern here, the > point is that you could do it and this just illustrates a particular > real-world potential use of it. In this case, the contents of memory > at that address are actually mapped into all processes by the kernel, > that's why all Intel processes on OS X can see it there. > > As for common symbols, it's perhaps a bit of a hack. Consider two C > source files that do this: > > int foo; > > In this case if you did not use common symbols the compiler would have > to generate assembler statements that allocate 4 bytes (assuming 32- > bit int) from the __DATA,__bss section. When you then linked the two > resulting object files together you would get a duplicate symbol > error. Granted the correct modern code would be for one file to > declare int foo; and all the others to declare extern int foo;. The > compiler by default uses -fcommon (to turn it off use -fno-common > which is listed in the manpage). That means that any time the > compiler sees an uninitialized variable declaration it treats it > basically as if it was declared extern. > > If you really want to understand it further, perhaps you should use > gcc's -S option so you can get a feel for the assembler statements > that the compiler uses. > > Normally you should only see common in .o files. Once they are linked > into an executable, bundle, or dylib they are allocated by the link > editor (ld) into __DATA,__common. > > Also, otool -lV will show you quite a bit of information on the load > commands. The capital v makes it display in non-numeric verbose mode > so for example you see the flags as names instead of hexadecimal > bitmasks. > > Please don't go off and do something foolish like use bcopy directly > from the commpage in some shipping app just because you can. This > information is merely to satisfy your curiosity, nothing more. > > -Dave > > > On Apr 16, 2008, at 10:36 AM, Army Research Lab wrote: >> Hi all, I'm working on writing a command line scraper for the output >> of the >> 'nm' command written in Python. For right now, the only things I'm >> interested in are the lines output by the command >> 'nm -gmf foo.o | c++filt -_'. This applies to anything that nm can >> parse, >> not just .o files. Although it isn't 100% necessary for what I'm >> doing, I >> am curious as to what the various sections in the mach-o binaries >> are. As a >> brief, but incomplete list, here is what I've seen so far: >> >> __DATA,__common -- Uninitialized imported global symbols >> __DATA,__data -- Initialized mutable variables >> __TEXT,__cstring -- Constant C-strings >> __TEXT,__eh_frame -- ??? >> __TEXT,__literal8 -- ??? >> __TEXT,__text -- Executable machine code >> __TEXT,__textcoal_nt -- ??? >> absolute -- absolute addresses - start up code? >> common -- ??? _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: enabling #ident in Apple gcc
by Andrew Pinski 16 Apr '08

16 Apr '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; bh=fqS/9yOxol1xjKCQmxtrcLTtwW6Fl8KPqTVwoxQkz/E=; b=a1MnDBhkVNnInnhrh1DHLQHUXoJsEAqI1BNkuIWUJdJTyQ5F/EWFyWARR/a1LP9pPvjlRvA5JPuYLjqNezf5WjWTy19a/iJtcAwqCc48SCNue93sl1M6AMQKtg/tyFK0/KdsZkjQDcU/qTLuE92fo2GQeKLL1YUTHGM/vEbkiPI= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=IRr6vATuvjkf8IG6o3zFVQd5dcsRPHemFvIGL2Fe9dByGY2xuzDua1E2AYl6C0iSsgjuMMmCTK160fcnm+Olf3qrxJgfIc9PTA6keY1m87Twtlq6Zq3YbyDtj2O6e0MthblXX7kWtx2Y1rc0FXwu0iji0yxZbosSZzsPYHr0BiM= On Wed, Apr 16, 2008 at 10:53 AM, Terry Lambert <tlambert(a)apple.com> wrote: > > What compiler version, what local patches, and how do the same versions act > compiled for Darwin? This could be a recent change; the #ifdef's were real. > > Also, Mach-I, like ELF or COFF is a container format; it really doesn't care > what people put in it. That is not fully true since the dynamic loader cares if you don't follow the ABI but other than that, really if you need your mach-o loaded, it can be used any which way you want. -- Pinski _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: enabling #ident in Apple gcc
by Terry Lambert 16 Apr '08

16 Apr '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: mach-o section question
by David Elliott 16 Apr '08

16 Apr '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Hi Cem, http://devworld.apple.com/documentation/DeveloperTools/Reference/Assembler/… int foo; -Dave __DATA,__common -- Uninitialized imported global symbols __DATA,__data -- Initialized mutable variables __TEXT,__cstring -- Constant C-strings __TEXT,__eh_frame -- ??? __TEXT,__literal8 -- ??? __TEXT,__text -- Executable machine code __TEXT,__textcoal_nt -- ??? absolute -- absolute addresses - start up code? common -- ??? Thanks, Cem Karan _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/dfe%40tgwbd.org _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… What you really want is the "Mac OS X Assembler Reference", specifically "Directives for Designating the Current Section" That doesn't explain absolute or common symbols. Absolute are pretty easy to explain. With the assembler you can define a symbol with an absolute address which is quite useful when you have some binary interface that depends on this. For example, from a C file you could prototype extern void _commpage_bcopy(const void *src, void *dst, size_t len); Then from an assembler file you could specify a __commpage_bcopy symbol at exactly 0xffff0780. You should not do this because that's not an official ABI but if you do something like x/10i 0xffff0780 from gdb on an Intel mac with an Intel program you will clearly see that this is bcopy. Or rather _bcopy (in C, hence __bcopy from gdb). Whether Apple uses an absolute symbol to achieve this or not is not the concern here, the point is that you could do it and this just illustrates a particular real-world potential use of it. In this case, the contents of memory at that address are actually mapped into all processes by the kernel, that's why all Intel processes on OS X can see it there. As for common symbols, it's perhaps a bit of a hack. Consider two C source files that do this: In this case if you did not use common symbols the compiler would have to generate assembler statements that allocate 4 bytes (assuming 32- bit int) from the __DATA,__bss section. When you then linked the two resulting object files together you would get a duplicate symbol error. Granted the correct modern code would be for one file to declare int foo; and all the others to declare extern int foo;. The compiler by default uses -fcommon (to turn it off use -fno-common which is listed in the manpage). That means that any time the compiler sees an uninitialized variable declaration it treats it basically as if it was declared extern. If you really want to understand it further, perhaps you should use gcc's -S option so you can get a feel for the assembler statements that the compiler uses. Normally you should only see common in .o files. Once they are linked into an executable, bundle, or dylib they are allocated by the link editor (ld) into __DATA,__common. Also, otool -lV will show you quite a bit of information on the load commands. The capital v makes it display in non-numeric verbose mode so for example you see the flags as names instead of hexadecimal bitmasks. Please don't go off and do something foolish like use bcopy directly from the commpage in some shipping app just because you can. This information is merely to satisfy your curiosity, nothing more. On Apr 16, 2008, at 10:36 AM, Army Research Lab wrote: Hi all, I'm working on writing a command line scraper for the output of the 'nm' command written in Python. For right now, the only things I'm interested in are the lines output by the command 'nm -gmf foo.o | c++filt -_'. This applies to anything that nm can parse, not just .o files. Although it isn't 100% necessary for what I'm doing, I am curious as to what the various sections in the mach-o binaries are. As a brief, but incomplete list, here is what I've seen so far: Can anyone fill me in on what those unknown sections are? I briefly looked at the Mac OS X ABI Mach-O File Format Reference (http://developer.apple.com/documentation/DeveloperTools/Conceptual/MachORun time/Reference/reference.html), which is where I got the few definitions I have above, but I'd like to know more. This email sent to dfe(a)tgwbd.org This email sent to site_archiver(a)lists.apple.com

1 0

Re: enabling #ident in Apple gcc
by Chris Inacio 16 Apr '08

16 Apr '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:references; bh=mdcnU7ZTKthd5PKvkQ00IpIMg06SR+sJpikIupTX6fw=; b=qenU32EGb40s50mmLeK/IerzXHodPniUrMaLkzCIuRTSP8mOjI6hOd9y5W9Txs/jdEX/AoW3Qf9gmlkigcM/P5BxnGEvW8yXSGJGVi7Q/nndGMftGl27jNo3/Q1Nnsl6AqliS6yhKhX8VbB7/DVQXl4lruS1UVsXum0ThDhRWNY= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:references; b=b3iW8cC6iYVGAkQVAv+e+CPJKY5kTs1tc4nyVGFWY29GPSRvfSlCxYcX3liu8UWfNYhZXjBxSUuLdfQc+LmgpCECWl7QSA3CawePnh8W2gDkLSn0wzCcCwKNDG5iWcuGJd1zegEb7xYXLRuhxY5Znwt6t0xMQVLz/TC39kt2zUE= _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: mach-o section question
by Jonas Maebe 16 Apr '08

16 Apr '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On 16 Apr 2008, at 16:36, Army Research Lab wrote: __DATA,__common -- Uninitialized imported global symbols __DATA,__data -- Initialized mutable variables __TEXT,__cstring -- Constant C-strings __TEXT,__eh_frame -- ??? exception handling frame information __TEXT,__text -- Executable machine code __TEXT,__textcoal_nt -- ??? absolute -- absolute addresses - start up code? common -- ??? Don't know. Jonas _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… __TEXT,__literal8 -- ??? That's what a 8byte_literals section becomes (i.e., 8-byte-sized constants) code sections with symbols which can be coalesced (i.e., if a symbol in this section occurs in more than one section, fold it together with another instance rather than giving a "duplicate symbol error"). "nt" is probably an abbreviation for "no toc". See http://developer.apple.com/documentation/DeveloperTools/Reference/Assembler… (under "no_toc" for more information) This email sent to site_archiver(a)lists.apple.com

1 0

mach-o section question
by Army Research Lab 16 Apr '08

16 Apr '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Thread-index: Acifz0hLhthtfAvCEd2kCwAdT0T19A== Thread-topic: mach-o section question User-agent: Microsoft-Entourage/11.4.0.080122 Hi all, I'm working on writing a command line scraper for the output of the 'nm' command written in Python. For right now, the only things I'm interested in are the lines output by the command 'nm -gmf foo.o | c++filt -_'. This applies to anything that nm can parse, not just .o files. Although it isn't 100% necessary for what I'm doing, I am curious as to what the various sections in the mach-o binaries are. As a brief, but incomplete list, here is what I've seen so far: __DATA,__common -- Uninitialized imported global symbols __DATA,__data -- Initialized mutable variables __TEXT,__cstring -- Constant C-strings __TEXT,__eh_frame -- ??? __TEXT,__literal8 -- ??? __TEXT,__text -- Executable machine code __TEXT,__textcoal_nt -- ??? absolute -- absolute addresses - start up code? common -- ??? Can anyone fill me in on what those unknown sections are? I briefly looked at the Mac OS X ABI Mach-O File Format Reference (http://developer.apple.com/documentation/DeveloperTools/Conceptual/MachORun time/Reference/reference.html), which is where I got the few definitions I have above, but I'd like to know more. Thanks, Cem Karan _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: enabling #ident in Apple gcc
by Terry Lambert 15 Apr '08

15 Apr '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: enabling #ident in Apple gcc
by Chris Inacio 15 Apr '08

15 Apr '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:references; bh=ve32HlVT7MIzXunl+7PW+rcrsA//QaD15eTwEXB+MBc=; b=V1jezYTxKd4+T2uKZyq9quPfhYM0BAy20bC8I2sYBVYBNYCV4kJDBYg448minq4kCRSofaLr+Y3nJK3IbVRFyFiSBb4ZiN8lPrHFxQux09sCkm7+lqiwSDu9ElciPWCnS+h98+Qc8f3e8Z92h/QNwnClwt1wa3K9kRBiLJawQLQ= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:references; b=R2Cq/L0FAS4sBZdRJ8BEmKQ+mn6RCcXaumBO25SSwJ2c/XZhhlS91EDzD+0nn9+HLyCezRtgt74a9giYnJOu2XL5/wrC83vPLzgf2FPZRhhGa+1IvptKKyYfMXnYGG93LAEOSDaU224/4aRLoCb91NZrJXPSZYtmLNBTqkP1KpE= _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: enabling #ident in Apple gcc
by Shawn Erickson 14 Apr '08

14 Apr '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; bh=9Ua1pN4ua2EeMfe7yxThbL516hY1frfUduINLMEd1lY=; b=N1PDpGhxwh0EWV0Xm4Mcm7b9A04yUsPKGHBo8im7gF2usGzNVPz29qAGLsG+6m6iIF6SYNfJmVJj7V9tASNgaLFHMBf3eL096gsMG+isKd61IrkoLhUWTtF2nL4VOfonDlp9ixEDqHteXxGzTon7zuEFmpqkZNFIN9AX8yDhDUE= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=o74h5uq3mFL3yxN+Y2ggmb5vtaNcqQcKcFuoQlMZk4JTxa8qsd5odA9vp7EKXbelO/rVf/rRciz2raOZeQADED9MgKz5uMVQSzGuEPren4WkDdh3ZDKKXy0ILHtY6JmPMe43U73CTaygeD7bpDJL1Qvt7Y33vLQC7SS9uxBmQuM= On Mon, Apr 14, 2008 at 11:59 AM, Chris Inacio <nacho319(a)gmail.com> wrote: > I work on developing cross platform software, (YAF, libfixbuf,) which runs > on multiple varieties of Unix/POSIX. We have started to put: > > #ident "$Id$ > > into our code to be able to use the ident command to find version numbers. > This works on all platforms I've tested with all the compilers I've tested, > (Linux, Solaris, gcc, Sun Pro, Intel C) except for Mac OS. It doesn't cause > an error on Mac OS, it just doesn't work. <http://developer.apple.com/documentation/Darwin/Reference/ManPages/man1/ide…> --- <http://gcc.gnu.org/onlinedocs/cpp/Other-Directives.html> 8 Other Directives The '#ident' directive takes one argument, a string constant. On some systems, that string constant is copied into a special segment of the ob ject ﬁle. On other systems, the directive is ignored. The '#sccs' directive is a synonym for '#ident'. These directives are not part of the C standard, but they are not oﬃcial GNU extensions either. What historical information we have been able to ﬁnd, suggests they originated with System V. The null directive consists of a '#' followed by a newline, with only whitespace (including comments) in between. A null directive is understood as a preprocessing directive but has no eﬀect on the preprocessor output. The primary signiﬁcance of the existence of the null directive is that an input line consisting of just a '#' will produce no output, rather than a line of output containing just a '#'. Supposedly some old C programs contain such lines. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0