April 2005 - Darwin-dev - lists.apple.com

Re: Darwin 7.9 Source Code Available
by Kevin Van Vechten 16 Apr '05

16 Apr '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Darwin 7.9 Source Code Available
by Tsuyoshi MOMOSE / ももせつよし 16 Apr '05

16 Apr '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On 2005/04/16, at 11:48, Kevin Van Vechten wrote: I could not build a kernel by the source code. The error message is as followed: I don't know this is the same problem that Ben pointed out. --- Tsuyoshi MOMOSE _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… The Darwin team is pleased to announce the availability of the source code for Darwin 7.9. These sources correspond to Mac OS X 10.3.9, and are available via the web at <http://www.opensource.apple.com/darwinsource/>. [Apr 15 2004] xnu-517.12.7/osfmk/ppc/chud/chud_cpu.c: In function `chudxnu_set_shadowed_spr': xnu-517.12.7/osfmk/ppc/chud/chud_cpu.c:224: error: `chud_7400_msscr1' undeclared (first use in this function) xnu-517.12.7/osfmk/ppc/chud/chud_cpu.c:224: error: (Each undeclared identifier is reported only once xnu-517.12.7/osfmk/ppc/chud/chud_cpu.c:224: error: for each function it appears in.) xnu-517.12.7/osfmk/ppc/chud/chud_cpu.c: In function `chudxnu_set_shadowed_spr64': xnu-517.12.7/osfmk/ppc/chud/chud_cpu.c:260: error: `chud_970_hid0' undeclared (first use in this function) xnu-517.12.7/osfmk/ppc/chud/chud_cpu.c:272: error: `chud_970_hid1' undeclared (first use in this function) xnu-517.12.7/osfmk/ppc/chud/chud_cpu.c:284: error: `chud_970_hid4' undeclared (first use in this function) xnu-517.12.7/osfmk/ppc/chud/chud_cpu.c:296: error: `chud_970_hid5' undeclared (first use in this function) xnu-517.12.7/osfmk/ppc/chud/chud_cpu.c: In function `chudxnu_clear_cpu_rupt_counters': xnu-517.12.7/osfmk/ppc/chud/chud_cpu.c:444: warning: passing arg 1 of `bzero' from incompatible pointer type This email sent to site_archiver(a)lists.apple.com

1 0

Re: authopen source code and -stdoutpipe usage
by Soren Spies 16 Apr '05

16 Apr '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com User-agent: Mutt/1.5.8i On Sat, Apr 16, 2005 at 01:09:16AM +0300, Dan Bernstein wrote: I'm looking for the authopen source code, if it's available, and for an example of how to use it from within another program with the -stdoutpipe option. The source to authopen is not currently part of Darwin. #include <err.h> #include <errno.h> #include <fcntl.h> #if qDEBUG #include <stdio.h> /* needed for perror */ #endif // qDEBUG #include <sys/types.h> #include <sys/socket.h> #include <unistd.h> #include <sys/wait.h> #include <stdio.h> #if qDEBUG #define dperrret(x) do { perror x; return -1; } while(0) #define dprintf(x) do { printf x; fflush(stdout); } while(0) #define dfflush(x) do { fflush(x); } while(0) #else #define dperrret(x) do { return -1; } while(0) #define dprintf(x) do { } while(0) #define dfflush(x) do { } while(0) #endif /* authopen(3) uses these and they may come in handy */ #define AO_PROGNAME "authopen" #define AO_INSTALL_PATH "/usr/libexec/" #define AO_PATH AO_INSTALL_PATH AO_PROGNAME #define AO_WRITEOPT "-w" #define AO_APPENDOPT "-a" #define AO_PIPEOPT "-stdoutpipe" #define AO_CREATOPT "-c" #define AO_EXCLOPT "-x" #define AO_MODEOPT "-m" #define AO_OFLAGOPT "-o" ssize_t discardfunc(int i, const void* p, size_t l) { return l; } /* ensure authopen's STDOUT is a copy of the pipe */ if(pipe[1] != STDOUT_FILENO) if(dup2(pipe[1], STDOUT_FILENO) != STDOUT_FILENO) err(1, "dup2 failed for STDOUT_FILENO"); /* errors result in child exit */ /* set up arguments */ if(snprintf(oarg, 11, "0x%08x", flags) != 10) dperrret(("snprintf into oarg failed")); if(snprintf(marg, 6, "0%04o", omode) != 5) dperrret(("snprintf into marg failed")); /* now to actually get the fd back */ rval = recv_fd(pipe[0], userfunc); waitpid(pid, &status, 0); return rval; } #include <errno.h> #include <sys/types.h> #include <sys/socket.h> /* struct msghdr */ #include <sys/uio.h> /* struct iovec */ #include <stddef.h> #include <stdlib.h> /* malloc(3) */ #include <unistd.h> static struct cmsghdr *r_cmptr = NULL; /* malloc'ed first time */ #define CONTROLLEN (sizeof(struct cmsghdr) + sizeof(int)) /* size of control buffer to send/recv one file descriptor */ /* Receive a file descriptor from another process (a server). * In addition, any data received from the server is passed * to (*userfunc)(STDERR_FILENO, buf, nbytes). We have a * 2-byte protocol for receiving the fd from send_fd(). */ int recv_fd(int servfd, ssize_t (*userfunc)(int, const void *, size_t)) { int newfd = -1, nread, status; char *ptr, buf[4096]; struct iovec iov[1]; struct msghdr msg; status = -1; for ( ; ; ) { iov[0].iov_base = buf; iov[0].iov_len = sizeof(buf); msg.msg_iov = iov; msg.msg_iovlen = 1; msg.msg_name = NULL; msg.msg_namelen = 0; if (r_cmptr == NULL && (r_cmptr = malloc(CONTROLLEN)) == NULL) return(-1); msg.msg_control = (caddr_t) r_cmptr; msg.msg_controllen = CONTROLLEN; if ((nread = recvmsg(servfd, &msg, 0)) < 0) { if(errno == EINTR) { errno = 0; dprintf(("retrying...\n")); continue; } else dperrret(("recvmsg error")); } else if (nread == 0) { errno = ECONNRESET; dprintf(("connection closed by server\n")); return -1; } /* See if this is the final data with null & status. Null must be next to last byte of buffer, status byte is last byte. Zero status means there must be a file descriptor to receive. */ for (ptr = buf; ptr < &buf[nread]; ) { if (*ptr++ == 0) { if (ptr != &buf[nread-1]) { /* err_dump("message format error"); * this dumped core */ dprintf(("protocol error\n")); errno = EIO; return -1; } status = *ptr & 255; if (status == 0) { if (msg.msg_controllen != CONTROLLEN) { dprintf(("fatal error msg_controllen != CONTROLLEN\n")); errno = EIO; return -1; } /* err_dump("status = 0 but no fd"); */ newfd = *(int *)CMSG_DATA(r_cmptr); /* new descriptor */ } else { newfd = -1; errno = status; } nread -= 2; } } /* if userfunc is NULL, we don't call it */ if (userfunc && nread > 0) if ((*userfunc)(STDERR_FILENO, buf, nread) != nread) return(-1); if (status >= 0) /* final data has arrived */ return(newfd); /* descriptor, or -status */ } } #define CONTROLLEN (sizeof(struct cmsghdr) + sizeof(int)) static struct cmsghdr *s_cmptr = NULL; /* buffer is malloc'ed first time */ /* size of control buffer to send/recv one file descriptor */ /* Pass a file descriptor to another process. * If fd<0, then -fd is sent back instead as the error status. */ int send_fd(int clifd, int fd) { struct iovec iov[1]; struct msghdr msg; char buf[2]; /* send_fd()/recv_fd() 2-byte protocol */ iov[0].iov_base = buf; iov[0].iov_len = 2; msg.msg_iov = iov; msg.msg_iovlen = 1; msg.msg_name = NULL; msg.msg_namelen = 0; if (fd < 0) { dprintf(("sending error %d: %s\n", -fd, strerror(-fd))); msg.msg_control = NULL; msg.msg_controllen = 0; buf[1] = -(char)fd; /* nonzero status means error */ if (buf[1] == 0) buf[1] = 1; /* -256, etc. would screw up protocol */ } else { dprintf(("sending fd\n")); if (s_cmptr==NULL&&(s_cmptr=(struct cmsghdr*)malloc(CONTROLLEN))==NULL) return(-1); s_cmptr->cmsg_level = SOL_SOCKET; s_cmptr->cmsg_type = SCM_RIGHTS; s_cmptr->cmsg_len = CONTROLLEN; msg.msg_control = (caddr_t) s_cmptr; msg.msg_controllen = CONTROLLEN; *(int *)CMSG_DATA(s_cmptr) = fd; /* the fd to pass */ buf[1] = 0; /* zero status means OK */ } buf[0] = 0; /* null byte flag to recv_fd() */ dfflush(stdout); if (sendmsg(clifd, &msg, 0) != 2) return(-1); dprintf(("sendmsg successful\n")); return(0); } -- Soren Spies DiskImages, etc; I/O Kit team Apple Computer, Inc _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… /usr/libexec/authopen is an implementation of an "open server" as described in Stevens' "Advanced Programming in the Unix Environment." The send_fd and recv_fd functions come from that text. Here's an example authopen() function with support (apologies that I'm cribbing this together from various sources; it does compile :): // passfd.h int recv_fd(int servfd, ssize_t (*userfunc)(int, const void *, size_t)); int send_fd(int clifd, int fd); // authopen.h /* * authopen tries to be like open(2) ... returning -1 and letting caller * deal with errno, etc ... now passes everything through to open; needs * to learn to set errno appropriately after getting info from the socket. */ int authopen(const char *path, int flags, mode_t omode) { int pipe[2], pid, rval, status; ssize_t (*userfunc)(int, const void *, size_t); char oarg[11], marg[6]; /* 0x12345678\0 and 01234\0 */ /* could use pipe(2)? */ if(socketpair(AF_UNIX, SOCK_STREAM, 0, pipe) < 0) dperrret(("socketpair failed")); if((pid = fork()) < 0) dperrret(("fork() error")); else if(pid == 0) { /* am child */ close(pipe[0]); /* close the parent's side */ /* fire off authopen itself with flags (execl NULL-termated) */ dprintf((". %s %s %s %s %s %s %s\n", AO_PROGNAME, AO_PIPEOPT, AO_MODEOPT, marg, AO_OFLAGOPT, oarg, path)); if(execl(AO_PATH, AO_PATH, AO_PIPEOPT, AO_MODEOPT, marg, AO_OFLAGOPT, oarg, path, NULL)) err(1, "exec of " AO_PATH " failed"); } /* parent; no 'else' since child exec's or exit's above */ if(close(pipe[1]) < 0) dperrret(("close should not fail")); /* allows recv_fd to spew messages on console or not */ #if qDEBUG #warning enabling output of excess data from authopen userfunc = write; #else userfunc = discardfunc; #endif This email sent to site_archiver(a)lists.apple.com

1 0

Re: Darwin 7.9 Source Code Available
by Benjamin Herrenschmidt 16 Apr '05

16 Apr '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On Fri, 2005-04-15 at 19:48 -0700, Kevin Van Vechten wrote: > The Darwin team is pleased to announce the availability of the source > code for Darwin 7.9. These sources correspond to Mac OS X 10.3.9, and > are available via the web at > <http://www.opensource.apple.com/darwinsource/>. [Apr 15 2004] Hi ! it seems that xnu osfmk/ppc/machine_routines_asm.s has been truncated, exactly like it happened with xnu in 10.3.8 bits. Looks like the automatic tool removing bits for the open source version is having trouble :) (kidding) Not directly related, but is there any chance some of the newer drivers that are definitely 100% Apple IP get open sourced one day ? I'm thinking about things like MacIO GPIO driver (I see no reason to "hide" that one, there is nothing fancy about it), or some of the low level i2c sensor drivers (most those chips are off-the-shelves parts). It would be absolutely fantastic if AppleSMU & friends were opensourced too, and the new USB trackpad & keyboard drivers, but I suppose that will never happen ... Ben. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Darwin 7.9 Source Code Available
by Kevin Van Vechten 16 Apr '05

16 Apr '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… The Darwin team is pleased to announce the availability of the source code for Darwin 7.9. These sources correspond to Mac OS X 10.3.9, and are available via the web at <http://www.opensource.apple.com/darwinsource/>. [Apr 15 2004] -- Kevin Van Vechten <kvv(a)apple.com> Open Source Software Engineer Apple Computer This email sent to site_archiver(a)lists.apple.com

1 0

authopen source code and -stdoutpipe usage
by Dan Bernstein 15 Apr '05

15 Apr '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=D64PcAydJd4SFt7mw9rdApSXLHH2R4UBG1QpYrHYnZw2x4ctdEvoUn6ijx7e6XJaKwfqZ349xh72FdSVMmvXG0moiKs8rPZ1q+Tf9IBB1sdjLrW4Uis2XtIs2FUoeVMpbb9JfAjNqhTH8oCUwEZi64IWrZsHPb9z9vWneuVjIB0= Hi, I'm looking for the authopen source code, if it's available, and for an example of how to use it from within another program with the -stdoutpipe option. Thanks, -- Dan _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: overriding malloc
by Mike Smith 15 Apr '05

15 Apr '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com - force a flat namespace using DYLD_FORCE_FLAT_NAMESPACE - link applications with -multiply_defined -u foo_hack() - add foo_hack() into scalable_malloc.c along with the hooks we need - include the scalable_malloc.o object in our lib. It's really quite simple. = Mike _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On Apr 14, 2005, at 12:05 PM, darwin-dev-request(a)lists.apple.com wrote: Because we need to override malloc, and the only way we can figure out to do it is to As I understand it (I didn't come up with it) this forces the rtld to load our scalable_malloc.o rather than the one from libc, since the first thing it does is to go looking for foo_hack(). mach_override is evil. Really evil. Patching someone else's runtime without using hooks is stupid. It's even more stupid when their runtime has hooks that you can use already. I've already told you in private email how to subvert the default allocator for malloc. Since the topic is so popular, I'll go over it in public again. 8) 1) include <malloc/malloc.h> 2) call malloc_default_zone() and look at the version number in the returned structure. If it's not one you recognise, abort(). This is important; there is no guarantee of compatibility if the version numbers change. 3) save the relevant vectors out of the default zone's malloc_zone_t, overwrite them with pointers to your interpose functions. Presto, you're on the call chain for calls to the default allocator. You can either never call it again, or just mangle things and pass them through. Be aware that the default allocator isn't the only allocator in the system, it's just the one that backs malloc(). Do not mess with the reserved fields, the comments there are not for show. Don't try to cook up another malloc_zone_t and put the default allocator's function vectors in there; it depends on the default malloc_zone_t being embedded at the head of it's internal datastructure. If you decide to never call the default malloc again, you will leak about 9MB, most of which won't be resident. That's not too bad. For the most part, locking is not critical when you do this. You may cause some free() leakage if you race with the (size) and (free) swap out, but that's about it. This email sent to site_archiver(a)lists.apple.com

1 0

Etiquette Reminder!
by Godfrey van der Linden 14 Apr '05

14 Apr '05

site_archiver(a)lists.apple.com Delivered-To: Darwin-dev(a)lists.apple.com G'day All, I'm not sorry for the cross post. Recently my unsolicited email load has become unacceptable. Godfrey van der Linden _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… It looks like I need to remind the lists that although I often respond to questions on Darwin and I try to be as helpful as possible; I really can NOT respond to email questions that are sent directly to me. This email sent to site_archiver(a)lists.apple.com

1 0

Re: How do I prevent a function from being optimized out?
by Andrew Gallatin 13 Apr '05

13 Apr '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Brian Dantes writes: > Read the stuff about mach_override. It's not replacing the symbol - it's > actually patching the actual malloc call in libc *on the fly*. > That's essentially what I meant. If the "different" mallocs all get mapped to the same address, one patch should work. Since you say it works, I guess that must be what happens. Thanks again, Drew _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: How do I prevent a function from being optimized out?
by Brian Dantes 13 Apr '05

13 Apr '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com User-agent: Microsoft-Entourage/10.1.4.030702.0 Read the stuff about mach_override. It's not replacing the symbol - it's actually patching the actual malloc call in libc *on the fly*. -Brian > From: Andrew Gallatin <gallatin(a)cs.duke.edu> > Date: Wed, 13 Apr 2005 18:31:17 -0400 (EDT) > To: Brian Dantes <brian(a)things.org> > Cc: <darwin-dev(a)lists.apple.com> > Subject: Re: How do I prevent a function from being optimized out? > > > Brian Dantes writes: >> No, you do not need to force a flat namespace with mach_override, that's the >> beauty of it. It is a bit scary when you realize what it's doing - but hey >> you'll learn to live with it. ;-> Now, if you want to override something in > > Wait, how can that work? > > Are you saying that even though there are multiple namespaces, the > "different" malloc functions wind up at the same address, or do > you have to do the override in each namespace? > > Drew > _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0