March 2004 - Darwin-kernel - lists.apple.com

Re: Stacked file systems - why not and what instead?
by Marek Kozubal 06 Mar '04

06 Mar '04

On Wed, 3 Mar 2004, Quinn wrote: > At 5:55 +0100 3/3/04, Ragnar Sundblad wrote: > >Apple Q&A 1242 says: > > Marek did a fine job of answering your questions but, seeing as I > wrote the Q&A in question, I feel obliged to follow up with more > information. Thanks! :) > Finally, I'd like to reiterate the comments I made in the Q&A. The > fact that Apple doesn't encourage or support stacking doesn't mean > that you can't do it. It just means that a) you can't get official > support from DTS, and b) you will probably suffer more binary > compatibility problems than if you were developing a supported > product. However, we have no secrets--the entire kernel source is > available via Darwin--and there's a variety of unofficial support > resources (such as this list). Marek is even using this technology > in a commercial product and he hasn't (yet) been abducted by the > Apple Kernel Purity Police (tm) (-: I can vouch for this! (This made my day Quinn :) ) However, in version 2.0 of our product we are very likely going to switch to the style of WebDav's file system for a number of reasons. One of them being that Apple has to keep WebDav working, so anything we steal^H^H^H^H^Hborrow from there will probably mirror into something in future variations of the VFS layer. Also I have a much better understanding of whats going on at this layer than when I started this project, which also helps... One thing that I am worried about is how much of a speed hit one would take having data going User -> Kernel -> User -> Kernel for a lot or most of the file system calls. MIG RPC's clearly have a good amount of overhead, and WebDav which I believe uses UNIX sockets, still seems like a sluggish file system in general, though maybe http has more to blame on that one than the boundary bounces. Any thoughts on this Quinn? :) --- Marek Kozubal marek(a)portents.com _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: LOCAL_PEERCRED and getpeereid
by Etsushi Kato 05 Mar '04

05 Mar '04

On Fri, Mar 05, 2004 at 07:24:50PM +0900, Etsushi Kato <ekato(a)ees.hokudai.ac.jp> wrote: > Can someone check this patch? and is it possible that getpeereid() > is supported in the next release of darwin? Hmm, my patch seems to be deleted. So, I'll resend the patch as a text. P.S. Is it the proper list for sending patches? Cheers, -- Etsushi Kato ekato(a)ees.hokudai.ac.jp --- xnu-517.3.7/bsd/kern/uipc_proto.c.orig Sat Dec 20 07:36:53 2003 +++ xnu-517.3.7/bsd/kern/uipc_proto.c Fri Mar 5 02:02:05 2004 @@ -78,7 +78,7 @@ static struct protosw localsw[] = { { SOCK_STREAM, &localdomain, 0, PR_CONNREQUIRED|PR_WANTRCVD|PR_RIGHTS, - 0, 0, 0, 0, + 0, 0, 0, &uipc_ctloutput, 0, 0, 0, 0, 0, 0, &uipc_usrreqs --- xnu-517.3.7/bsd/kern/uipc_usrreq.c.orig Sat Dec 20 07:36:53 2003 +++ xnu-517.3.7/bsd/kern/uipc_usrreq.c Fri Mar 5 03:08:06 2004 @@ -219,6 +219,8 @@ if (unp == 0 || unp->unp_vnode == 0) return EINVAL; + cru2x(p->p_ucred, &unp->unp_peercred); + unp->unp_flags |= UNP_HAVEPCCACHED; return 0; } @@ -450,6 +452,40 @@ sosend, soreceive, sopoll }; +int +uipc_ctloutput(so, sopt) + struct socket *so; + struct sockopt *sopt; +{ + struct unpcb *unp = sotounpcb(so); + int error; + + switch (sopt->sopt_dir) { + case SOPT_GET: + switch (sopt->sopt_name) { + case LOCAL_PEERCRED: + if (unp->unp_flags & UNP_HAVEPC) + error = sooptcopyout(sopt, &unp->unp_peercred, + sizeof(unp->unp_peercred)); + else { + if (so->so_type == SOCK_STREAM) + error = ENOTCONN; + else + error = EINVAL; + } + break; + default: + error = EOPNOTSUPP; + break; + } + break; + case SOPT_SET: + error = EOPNOTSUPP; + break; + } + return (error); +} + /* * Both send and receive buffers are allocated PIPSIZ bytes of buffering * for stream sockets, although the total for sender and receiver is @@ -627,7 +663,7 @@ register struct sockaddr_un *soun = (struct sockaddr_un *)nam; register struct vnode *vp; register struct socket *so2, *so3; - struct unpcb *unp2, *unp3; + struct unpcb *unp, *unp2, *unp3; int error, len; struct nameidata nd; char buf[SOCK_MAXADDRLEN]; @@ -682,12 +718,33 @@ thread_funnel_switch(NETWORK_FUNNEL, KERNEL_FUNNEL); goto bad; } + unp = sotounpcb(so); unp2 = sotounpcb(so2); unp3 = sotounpcb(so3); if (unp2->unp_addr) unp3->unp_addr = (struct sockaddr_un *) dup_sockaddr((struct sockaddr *) unp2->unp_addr, 1); + /* + * unp_peercred management: + * + * The connecter's (client's) credentials are copied + * from its process structure at the time of connect() + * (which is now). + */ + cru2x(p->p_ucred, &unp3->unp_peercred); + unp3->unp_flags |= UNP_HAVEPC; + /* + * The receiver's (server's) credentials are copied + * from the unp_peercred member of socket on which the + * process's credentials at that time so we can use + * them now. + */ + KASSERT(unp2->unp_flags & UNP_HAVEPCCACHED, + ("unp_connect: listener without cached peercred")); + memcpy(&unp->unp_peercred, &unp2->unp_peercred, + sizeof(unp->unp_peercred)); + unp->unp_flags |= UNP_HAVEPC; so2 = so3; } error = unp_connect2(so, so2); --- xnu-517.3.7/bsd/sys/un.h.orig Sat Dec 20 07:37:23 2003 +++ xnu-517.3.7/bsd/sys/un.h Fri Mar 5 01:55:26 2004 @@ -71,11 +71,15 @@ char sun_path[104]; /* path name (gag) */ }; +/* Socket options. */ +#define LOCAL_PEERCRED 0x001 /* retrieve peer credentails */ + #ifdef KERNEL #ifdef __APPLE_API_PRIVATE struct mbuf; struct socket; +int uipc_ctloutput(struct socket *so, struct sockopt *sopt); int uipc_usrreq __P((struct socket *so, int req, struct mbuf *m, struct mbuf *nam, struct mbuf *control)); int unp_connect2 __P((struct socket *so, struct socket *so2)); --- xnu-517.3.7/bsd/sys/unpcb.h.orig Sat Dec 20 07:37:23 2003 +++ xnu-517.3.7/bsd/sys/unpcb.h Fri Mar 5 02:45:28 2004 @@ -105,7 +105,25 @@ int unp_cc; /* copy of rcv.sb_cc */ int unp_mbcnt; /* copy of rcv.sb_mbcnt */ unp_gen_t unp_gencnt; /* generation count of this instance */ + int unp_flags; /* flags */ + struct xucred unp_peercred; /* peer credentials, if applicable */ }; + +/* + * Flags in unp_flags. + * + * UNP_HAVEPC - indicates that the unp_peercred member is filled in + * and is really the credentials of the connected peer. This is used + * to determine whether the contents should be sent to the user or + * not. + * + * UNP_HAVEPCCACHED - indicates that the unp_peercred member is filled + * in, but does *not* contain the credentials of the connected peer + * (there may not even be a peer). This is set in unp_listen() when + * it fills in unp_peercred for later consumption by unp_connect(). + */ +#define UNP_HAVEPC 0x001 +#define UNP_HAVEPCCACHED 0x002 #define sotounpcb(so) ((struct unpcb *)((so)->so_pcb)) #endif /* __APPLE_API_PRIVATE */ --- Libc-320/gen/Makefile.inc.orig Thu Aug 28 16:06:04 2003 +++ Libc-320/gen/Makefile.inc Fri Mar 5 03:32:32 2004 @@ -8,7 +8,7 @@ SRCS += NSSystemDirectories.c OSSystemInfo.c arc4random.c assert.c cache.c \ confstr.c crypt.c devname.c disklabel.c errlst.c fts.c \ - getloadavg.c getttyent.c getusershell.c getvfsbyname.c isnan.c \ + getloadavg.c getpeereid.c getttyent.c getusershell.c getvfsbyname.c isnan.c \ malloc.c nanosleep.c nlist.c scalable_malloc.c setlogin.c sigsetops.c \ stack_logging.c strtofflags.c sysconf.c syslog.c uname.c zone.c --- Libc-320/include/unistd.h.orig Sat Oct 25 07:36:25 2003 +++ Libc-320/include/unistd.h Fri Mar 5 03:53:07 2004 @@ -208,6 +208,7 @@ long gethostid(void); int gethostname(char *, int); mode_t getmode(const void *, mode_t); +int getpeereid(int, uid_t *, gid_t, *); int getpagesize(void) __pure2; char *getpass(const char *); int getpgid(pid_t _pid); --- /dev/null Fri Mar 5 04:07:53 2004 +++ Libc-320/gen/getpeereid.c Fri Mar 5 03:30:07 2004 @@ -0,0 +1,54 @@ +/* + * Copyright (c) 2001 Dima Dorfman. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include <sys/cdefs.h> +__FBSDID("$FreeBSD: src/lib/libc/gen/getpeereid.c,v 1.6 2002/12/16 13:42:13 maxim Exp $"); + +#include <sys/param.h> +#include <sys/socket.h> +#include <sys/ucred.h> +#include <sys/un.h> + +#include <errno.h> +#include <unistd.h> + +int +getpeereid(int s, uid_t *euid, gid_t *egid) +{ + struct xucred xuc; + socklen_t xuclen; + int error; + + xuclen = sizeof(xuc); + error = getsockopt(s, 0, LOCAL_PEERCRED, &xuc, &xuclen); + if (error != 0) + return (error); + if (xuc.cr_version != XUCRED_VERSION) + return (EINVAL); + *euid = xuc.cr_uid; + *egid = xuc.cr_gid; + return (0); +} _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

LOCAL_PEERCRED and getpeereid
by Etsushi Kato 05 Mar '04

05 Mar '04

Hi, I recently found that Darwin kernel doesn't support getpeereid() for peer unix domain socket credentials. In FreeBSD, it seems getpeereid() has been implemented since certain times before. I think local socket credentials are quite important for Unix applications, such as openssh, openldap, postgresql, and so on. So, I've just ported LOCAL_PEERCRED support from FreeBSD, and attached patch was made against darwin 7.2's xnu and Libc. I tested this patch on Mac OS X 10.3, and seems to work fine. Can someone check this patch? and is it possible that getpeereid() is supported in the next release of darwin? Cheers, -- Etsushi Kato ekato(a)ees.hokudai.ac.jp [demime 0.98b removed an attachment of type application/octet-stream which had a name of local_peercred_darwin.patch] _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

2 machine debugging problem...
by Marek Kozubal 05 Mar '04

05 Mar '04

Hello! I've been doing 2 machine debugging for a while now. One of the machines I am trying to debug however has become a pain. It won't let me do proper back traces or other features in kgm_macros either. Both machines are running 10.2.8 and both machines have the developer tools installed (dec '03 for jaguar). I've also tried it from another machine or 2 with the same results. For example, using the 10.2.8 kernel debug disk image from Apple: [villain:~] marek% gdb /Volumes/KernelDebugKit/mach_kernel GNU gdb 5.3-20021014 (Apple version gdb-250) (Sat Dec 7 02:14:27 GMT 2002) Copyright 2002 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "powerpc-apple-macos10". (gdb) source /Volumes/KernelDebugKit/kgmacros Loading Kernel GDB Macros package. Type "help kgm" for more info. (gdb) target remote-kdp (gdb) attach 10.0.0.3 Connected. [Switching to process 3] (gdb) showpid 434 /* this is the Finder's PID */ task vm_map ipc_space #acts pid proc command Cannot access memory at address 0x154 (gdb) showalltasks task vm_map ipc_space #acts pid proc command Cannot access memory at address 0x0 Is there some setting, or version mis-match that would cause these Macro's not to work? The machine has Panther installed on other partitions, I've done the usual boot-args="debug=0xe" and making the arp entry perminate and all that. However, this and other issues (like bt not working properly, not being able to inspect variables when bt does seem to work) is making things quite hard to track down. So if anyone has any ideas, it'd be most helpful! Thanks in advance! --- Marek Kozubal marek(a)portents.com _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Assembly macros with Gcc
by Paul Russell 04 Mar '04

04 Mar '04

On Feb 4, 2004, at 10:48 AM, Brad Post ((MACBU)) wrote: Anyone have a good reference on how to do macros in assembly? I'm trying to get some macros to work with gcc and Project Builder (not Xcode) and having some troubles, so does anyone know a good reference with decent examples? I'm trying to do something like this: #define SAVE_STATISTICS(stat, reg) lwz reg, stat(r3) addi reg, reg, 1 stw reg, stat(r3) I've tried a bunch of stuff, but gcc doesn't seem to like it and I'm hoping someone here might have a good reference book or website.... There are two kinds of asm macros you can use with gcc - old-fashioned preprocessor macros such as your example above (which, BTW, needs \ line continuation characters at the end of each line) and proper asm macros using the .macro directive. Unless there is a good reason I would go for the proper macros, e.g. .macro SAVE_STATISTICS lwz $2,$1(r3) addi $2,$2,1 stw $2,$1(r3) .endmacro Paul _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: auditing support
by Jim Magee 04 Mar '04

04 Mar '04

On Mar 3, 2004, at 2:09 PM, John C. Daub wrote: > on 3/3/04 12:58 PM, Shawn Erickson at shawn(a)freetimesw.com wrote: >> On Mar 3, 2004, at 8:11 AM, John C. Daub wrote: >> >>> I'm looking at the auditing support that was added to the kernel in >>> Panther. >>> I'm figuring out some things from headers, source, and Google, but >>> it's not >>> enough. Just wondering if anyone knows of any documentation and/or >>> sample >>> code pertaining to Darwin's kernel auditing support. >> >> Can you better define "auditing". It can me slightly different things >> to different folks. > > I'm new to this sort of thing (working with the kernel), so please > forgive > my newbieness. :-) > > I'm looking for information about that which is within > /usr/include/sys/audit.h (from Mac OS X 10.3.2). I see various > routines such > as audit(), auditon(), auditsvc(), and auditctl(). I see data > structures > like au_record_t, auditinfo_addr_t, and auditinfo_t. I see constants > like > AUDIT_CNT, A_GETPOLICY, and AUDIT_RECORD_MAGIC. I'm looking for > sample code > or, preferably, documentation about everything within sys/audit.h... > functions, data structures, constants. You'll get most of these things when audit (as defined in these headers/sources) is fully supported in the Darwin kernel. For now, you are looking at the shell of an implementation - (e.g. a work-in-progress). In particular, the kernel currently constructs audit records of its own, and accepts audit records from user space, but doesn't have any code to "do anything" with those records. You'll have to wait for that (it is being worked on, but no release times can be committed to at this point). The general gist of the audit support can be gleaned by looking at the Solaris BSM design. The goal of Darwin audit is to be (at least loosely) compatible with that approach (working from just public information). --Jim [demime 0.98b removed an attachment of type application/pkcs7-signature which had a name of smime.p7s] _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Stacked file systems - why not and what instead?
by Ragnar Sundblad 04 Mar '04

04 Mar '04

Thank you very much Marek and Quinn for your answers! I have started to take a look at webdavfs, it should be a good reference for me when trying to understand what the problems with Arla and OpenAFS are. Quinn wrote: ... it's not possible to create a stacking VFS plug-in that: 1. works reliably, Darwin inherited a VFS design from 4.4BSD that included stacking. However, Apple made significant changes to the VFS architecture (specifically, UBC and volfs) that affect the viability of stacking. I believe that these changes break stacking in the general case, Do you mean that the UBC design in general, on most OSes, break stacking, or is there something specific in the darwin implementation that does that? volfs I haven't even thought about, and neither do any of those two file system clients. I hope that we can instead rely on the emulation layer in the libraries for years to come. and no one in Apple file system engineering has the resources to create an example stacked file system that proves me wrong. Except for the (semi-stacking) webdavfs then, right? I don't mean to split hairs in a moot discussion, I just want to understand both the issues and the terminology. Regardless of the in-kernel problems there are also serious management problems for stacked file systems. I understand that those issues are important to apple and that it is hard to find a good general solution, but in this case I have the privilige to be able to ignore them! :-) However, we chose to support leaf file systems for the following reasons. ... o Leaf file systems are more important to our market than stacked file systems. As a member of the university computing community, I have an itching feeling that I must take that hat on for a moment and say that many of us universites are using AFS, and I hope we are important to apple. We are not alone, there are a few other, some very large, organizations that use it, but I must admit that AFS isn't very common in the general marketplace (yet! :-). For you engineers on this list, this is just information that I am happy if you have heard, I know we must use other channels to communicate our needs. o When we do have a kernel binary compatibility story (see below), it will likely address leaf file systems as a priority. I'd of course be very happy if you could consider those caching file systems too in your design. Should I contact someone for customer input, please let me know! It is good to know that apple wants webdav to work, the AFS clients could hopefully use the same mechanisms if there will be any. As a side discussion, I know that one of the Arla developers talked to Clark Warner about the possibility of generalising the webdav kext and point developers to that for both leaf and other file systems. That seems to be a good idea. (The kernel module that arla is using (formerly known as xfs, now called nnpfs), is actually meant to be such a general purpose cross platform kernel module, but maintaining it is a lot of work.) Since then we've been working hard to flesh out the KPI story for VFS plug-ins and NKEs. I'm hoping that we'll cover this in depth at WWDC 2004. So do I, see you there! Thanks again for your help! /ragge _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: auditing support
by John C. Daub 03 Mar '04

03 Mar '04

on 3/3/04 12:58 PM, Shawn Erickson at shawn(a)freetimesw.com wrote: > On Mar 3, 2004, at 8:11 AM, John C. Daub wrote: > >> I'm looking at the auditing support that was added to the kernel in Panther. >> I'm figuring out some things from headers, source, and Google, but it's not >> enough. Just wondering if anyone knows of any documentation and/or sample >> code pertaining to Darwin's kernel auditing support. > > Can you better define "auditing". It can me slightly different things > to different folks. I'm new to this sort of thing (working with the kernel), so please forgive my newbieness. :-) I'm looking for information about that which is within /usr/include/sys/audit.h (from Mac OS X 10.3.2). I see various routines such as audit(), auditon(), auditsvc(), and auditctl(). I see data structures like au_record_t, auditinfo_addr_t, and auditinfo_t. I see constants like AUDIT_CNT, A_GETPOLICY, and AUDIT_RECORD_MAGIC. I'm looking for sample code or, preferably, documentation about everything within sys/audit.h... functions, data structures, constants. I have been able to figure out a few things based off Google searches, but it appears that tho such auditing support exists in other *nix flavors it is not standardized. That's why I originally said that I'm looking for code and/or docs pertaining to Darwin's auditing support. I hope that clarifies things. Then if I may ask, what are the slightly different things that "auditing" could mean in this context? Thanx for the help. :-) -- John C. Daub }:-)>= <mailto:hsoi@hsoi.com> <http://www.hsoi.com/> "We're only gonna die from our own arrogance." - Bad Religion _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: auditing support
by Shawn Erickson 03 Mar '04

03 Mar '04

On Mar 3, 2004, at 8:11 AM, John C. Daub wrote: Hello. I'm looking at the auditing support that was added to the kernel in Panther. I'm figuring out some things from headers, source, and Google, but it's not enough. Just wondering if anyone knows of any documentation and/or sample code pertaining to Darwin's kernel auditing support. Can you better define "auditing". It can me slightly different things to different folks. -Shawn _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Determine port holder
by Peter Lovell 03 Mar '04

03 Mar '04

On Mar 3, 2004, at 12:50 PM, Matt Jaffa wrote: I just figure now that I am going to give up on trying to intercept classic traffic, too hard to distinguish which app (Within Classic) made the call. For Classic you should think about using a STREAMS module pushed onto TCP and UDP. A lot easier than NKEs in X. IIRC it's a bit hacky to find which app created the stream but there are some ways -- it's been along time. But for Classic you should definitely start with a module in the Classic environment, and not try to extract that info somehow on X. Regards.....Peter _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0