Mailing Lists: Apple Mailing Lists

Image of Mac OS face in stamp
 
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: open stub in libc trashing word in caller's frame?



R. Matthew Emerson,

The callee is not setting up the stack frame correctly. mode_t is a short, but it must be word aligned on the stack. One can observe this fact by reviewing what GCC actually generates by this code:

int foo(char *path, int flags, short mode, int word) {
	return 0;
}
int main(void) {
	return foo("hello", 123, 755, 0xdeadbeef);
}

The above generates the following. Please note the word alignment of the short value:

0x00001fcd <main+0>:	push   %ebp
0x00001fce <main+1>:	mov    %esp,%ebp
0x00001fd0 <main+3>:	sub    $0x18,%esp
0x00001fd3 <main+6>:	movl   $0xdeadbeef,0xc(%esp)
0x00001fdb <main+14>:	movl   $0x2f3,0x8(%esp)
0x00001fe3 <main+22>:	movl   $0x7b,0x4(%esp)
0x00001feb <main+30>:	movl   $0x1ff9,(%esp)
0x00001ff2 <main+37>:	call   0x1fca <foo>
0x00001ff7 <main+42>:	leave
0x00001ff8 <main+43>:	ret


davez




On Nov 19, 2008, at 11:25 AM, R.Matthew Emerson wrote:

I work on a Common Lisp compiler.

We have a way to call C library functions from the lisp. We process the C headers with a custom ffigen and use the resulting data to know the C types, number of arguments, and so forth.

When we call open(2) on 32-bit x86, though, something odd seems to be happening.

From the libc sources (Libc-498/sys/open.c), we see

/*
* open stub: The legacy interface never automatically associated a controlling
* tty, so we always pass O_NOCTTY.
*/
int
open(const char *path, int flags, mode_t mode)
{
return(__open_nocancel(path, flags | O_NOCTTY, mode));
}


and the code for this (on Mac OS X 10.5.5) is

0x90925a44 <open>:	push   %ebp
0x90925a45 <open+1>:	mov    %esp,%ebp
0x90925a47 <open+3>:	mov    0xc(%ebp),%eax
0x90925a4a <open+6>:	movzwl 0x10(%ebp),%edx     ; [1]
0x90925a4e <open+10>:	or     $0x20000,%eax
0x90925a53 <open+15>:	mov    %edx,0x10(%ebp)     ; [2]
0x90925a56 <open+18>:	mov    %eax,0xc(%ebp)
0x90925a59 <open+21>:	leave
0x90925a5a <open+22>:	jmp    0x908a0e88 <open$NOCANCEL$UNIX2003>

The problem here is that if we call this with only two args, this code will trash a word in the caller's frame (see [1] and [2] marked above).

Like I mentioned, we see the

int open(const char *, int, ...)

prototype in fcntl.h, and don't know anything about the __DARWIN_ALIAS_C(open) stuff, which is why we end up calling the "legacy" interface.

Do I need some remedial i386 ABI instruction? Or is something else wrong?


_______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (email@hidden) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/email@hidden

This email sent to email@hidden

_______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (email@hidden) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/email@hidden

This email sent to email@hidden
References: 
 >open stub in libc trashing word in caller's frame? (From: R.Matthew Emerson <email@hidden>)



Visit the Apple Store online or at retail locations.
1-800-MY-APPLE

Contact Apple | Terms of Use | Privacy Policy

Copyright © 2007 Apple Inc. All rights reserved.