March 2005 - Darwin-dev - lists.apple.com

cyrus-imapd/sasl on MacOS X 10.3.8: hang vs fail?
by Charles Bailey 17 Mar '05

17 Mar '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:mime-version:content-type:content-transfer-encoding; b=AjC0378hpF8L0FPTRpI6SGetaEfiM/nn0tntZRjbE1AlcGU9qFcPztB/K+Hj07JSHKKJd1btB7L0DnYJsCNEJY9m14N73EtaxP4+Gf8Yh76n9U4+SHwMYfvwDHF0nKqHspZhqD0Gaaw3Dfn1GPjMAz4l/wAdj+DEkW2neRQwPCU= I'm trying to set up a small mail service on a machine running OS X 10.3.8 (not Server; it's a small operation), and have run into several problems. I'm posting a summary to both the Cyrus and Darwin lists in the hope that someone more knowledgeable than I will have been over this ground, or at least be able to point me in the right direction. Briefly: OS: Darwin my.host.name 7.8.0 Darwin Kernel Version 7.8.0: Wed Dec 22 14:26:17 PST 2004; root:xnu/xnu-517.11.1.obj~1/RELEASE_PPC Power Macintosh powerpc Prior services running (for some time without problems): OpenSSH, Apache, MySQL First try: Built cyrus SASL 2.1.20 from source with ./configure --enable-static --enable-shared --with-pam=/usr --with-openssl \ --with-saslauthd=/var/run/saslauthd --with-mysql=/usr/local \ --with-dbpath=/usr/local/etc/sasldb --with-plugindir=/usr/local/lib/sasl2 Built cyrus imapd 2.2.12 from source (hacking config* to treat 'Darwin' like 'Rhapsody' and various C source files to skip sys/msg.h) with ./configure --with-pidfile=/var/run/cyrus/cyrus-master.pid --with-cyrus-prefix=/usr/local/cyrus \ --with-cyrus-user=cyrus --with-cyrus-group=mail --enable-listext --enable-gssapi \ --with-bdb=/usr/local/BerkeleyDB.4.3/ --with-openssl --with-sasl=/usr/local --with-perl=/usr/local/bin/perl --with-libwrap --without-snmp Built sendmail 8.3.13 from source with (among other things): APPENDDEF(`conf_sendmail_ENVDEF', `-DSTARTTLS') APPENDDEF(`conf_sendmail_LIBS', `-lssl -lcrypto') APPENDDEF(`conf_sendmail_ENVDEF', `-DSASL=2') define(`confSHAREDLIB_EXT',`.dylib') APPENDDEF(`conf_sendmail_LIBS', `-L/usr/local/lib -lsasl2') Installed alll of above, with /usr/local/lib/sasl2/(Sendmail|Cyrus).conf reading: pwcheck_method: saslauthd and saslauthd -a pam started, and /etc/pam.d/(smtp|imap) identical to login This starts fine, and smtptest and imtest using PLAIN over TLS authenticates successfully using "regular" password. However, after the third or fourth login attempt, the system "hangs": can't authenticate, can't establish new ssh session or open new terminal window on console, can't execute commands like "top" or "ps uax" (though "ps" fine) in existing terminal session, can display static web page, can do most things in programs already open on console, but can't launch new ones. Probem clears only with hard reboot. No error messages in system.log, auth.log, mail.log, imap.log. This has the feel of a deadlock or resource leak, perhaps around PAM, since ssh and various OS X services are affected, but I'm not sure where to look further. Second try (thinking perhaps having 2 sets of sasl libraries around made for some collisions): Removed installed /usr/local/lib/sasl2/* Built saslauthd only from cyrus SASL 2.1.20 with ./configure --enable-static --enable-shared --with-pam=/usr --with-openssl \ --with-saslauthd=/var/run/saslauthd --with-mysql=/usr/local \ --with-dblib=berkeley --disable-anonymous --without-gdbm --disable-otp at top level, but only building in saslauthd Built cyrus imapd 2.2.12 from source (hacking config* to treat 'Darwin' like 'Rhapsody', various C source files to skip sys/msg.h, and version.c to skip call to sasl_version(), which is missing in libsasl supplied with OS X) with ./configure --with-pidfile=/var/run/cyrus/cyrus-master.pid --with-cyrus-prefix=/usr/local/cyrus \ --with-cyrus-user=cyrus --with-cyrus-group=mail --enable-listext --enable-gssapi \ --with-bdb=/usr/local/BerkeleyDB.4.3/ --with-openssl --with-sasl=/usr --with-perl=/usr/local/bin/perl --with-libwrap --without-snmp Built sendmail 8.3.13 with same config as above Installed new binaries, moved (Sendmail|Cyrus).conf to /usr/lib/sasl2, started saslauthd then sendmail or cyrus-master, but 'imaptest -t "" -m plain -u bailey' fails with Mar 17 09:27:59 localhost saslauthd[448]: get_accept_lock : acquired accept lock Mar 17 09:29:29 localhost imap[468]: auxpropfunc error -1 Mar 17 09:29:29 localhost imap[468]: _sasl_plugin_load failed on sasl_auxprop_plug_init for plugin: uxprop Mar 17 09:29:35 localhost imap[468]: unknown password verifier Mar 17 09:29:35 localhost imap[468]: Password verification failed I'm guessing this latter error has to do with behavior of the Apple-supplied libsasl2 and plugins, but looking over the source from opendarwin hasn't given me much of a clue yet. Does this look familiar to anyone? Pointers to TFM/TFS/TFWeb welcome. I'm also happy to supply more info if it'd be useful;I didn't want to litter the lists with yet more log droppings. Thanks for any help you can offer. -- Regards, Charles Bailey Lists: bailey _dot_ charles _at_ gmail _dot_ com Other: bailey _at_ newman _dot_ upenn _dot_ edu _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Real Time timers API
by Nicolas.Castagne＠imag.fr 17 Mar '05

17 Mar '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Importance: Normal User-agent: SquirrelMail/1.4.2 Hi all, thx for answering, Shawn. Indeed, I am developing the core for a user GUI. The core will be a library (or a framework, dunno), and the Library will be built with wxWidget, which uses Cocoa AFAIK. Well... And I have indeed a second question, to add to the question on timers. Within X.3, the POSIX semaphores of the RT lib (sem_init, sem_post, etc.) are not available... (well, to be honnest, the strange thing is that they are on the system, with the doc and all the stuff, but they do not do anything...). I red somewhere that they should be added to X.4. Is that correct ? Thx Nicolas > On Mar 16, 2005, at 8:00 AM, Nicolas.Castagne(a)imag.fr wrote: > >> Hi there, >> >> I am porting quite a big software that was developped under Unix and >> Linux. >> >> This software really needs various "real time timers". >> >> Under Linux, these timer are provided in the real time timer API (-lrt >> library) : >> timer_create >> timer_delete >> timer_settime >> etc. >> >> Under Mac Os X, so far, I only founded the (setitimer getitimer) API... >> But it does not allow to set various timer per process/thread. >> >> Is there on Mac OS X an API that correspond with the RT library under >> Linux (and Irix, and...) ? >> Where may I find documentation ? > > What type of product are you porting? > > Is it a driver/kernel level thing or a user mode application? > > -Shawn > _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Communication between userspace apps and KEXT
by Stephane Sudre 17 Mar '05

17 Mar '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On 17 mars 05, at 05:06, Jeff Laing wrote: [...] Thanks in advance for any help whatsoever, You could use socket NKE otherwise. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… If your minimum requirement is 10.3, you could use kern_control socket. This is basically socket communication from the kernel to userspace and vice-versa. kern_control is documented and there's some kind of sample code in the documentation on Kernel Programming. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Sending notification from a time-constraint thread
by Stéphane Letz 17 Mar '05

17 Mar '05

site_archiver(a)lists.apple.com Delivered-To: Darwin-dev(a)lists.apple.com Le 16 mars 05, à 22:10, Mike Smith a écrit : Ok. To be sure I've understood correctly: when using RPC: Is is correct? OK. Thanks Stephane Letz _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On Mar 16, 2005, at 12:57 PM, Stéphane LETZ wrote: In a server/client system we are developing, we need a time-constraint thread in the server to send a notification to another thread in the server. We use MIG generated RPC calls to communicate between threads. Why do you do this? Because the server "non time-constraint" thread already "listen" to RPC calls coming from clients. So the idea was to use the same RPC system for server inter-thread communication also. Ok. But this then puts your time-constraint thread at the mercy of the other thread's workload, as you have discovered. - an asynchronous call (simpleroutine) *without* timeout will return immediately if the receiver queue is not full - an asynchronous call (simpleroutine) *without* timeout will block if the received queue is full - an asynchronous call (simpleroutine) *with* timeout will return immediately if the receiver queue is not full - an asynchronous call (simpleroutine) *with* timeout will return an "error (ipc/send) timed out " message if the receiver queue is full Unless you mean "between client and server", you would generally be much better off implementing a trivial messaging system using posix condition variables. Is is possible to have the server thread answer to RPC calls and handle condition variables at the same time? No; if it's blocked waiting for an incoming RPC, it can't also block waiting for a cvar. Your best bet is probably to have a helper thread blocked on the cvar, which will turn around and message the server thread as needed. This decouples your TC thread from the RPC; depending on your architecture you can have this helper thread handle fan-in for multiple TC threads. It's a bit miserable needing yet another thread, but the realities of queueing (RPC) vs. asynchronous messaging (cvars) mean that you need a transition interface of some sort. This email sent to site_archiver(a)lists.apple.com

1 0

Communication between userspace apps and KEXT
by Jeff Laing 16 Mar '05

16 Mar '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com (Apologies in advance if this shows that I don't understand the first thing I'm talking about) I have a toy project running which involves a KEXT thats monitoring certain kernel operations. I have a sysctl interface working fine, which allows me to control the KEXT to my satisfaction. However, I don't have a mechanism to collect the output from the KEXT. At the moment, its just doing printf()s which send results to the console.log, thus proving its all working, but making the results as good as inaccessible. I believe that what I want to do is have my userspace program create a Mach port, use sysctl() to let the KEXT know about the port and connect to it (somehow), then have the KEXT stream its output to the port with my userspace program slurping up the results. Presumably if the program dies, the KEXT can see that the port goes invalid and can stop talking to it, without crashing my machine. Sadly, I can't find any simple samples that show how to do this sort of stuff. Can anyone give me a pointer? Or am I dreaming and this sort of thing would never work anyway? (The alternative that occurred to me was to map a page of shared memory between userspace app and kext and have them manage a ring-buffer between themselves, but I don't think I know anywhere near enough about messing with kernel memory) Thanks in advance for any help whatsoever, Jeff Laing <jeffl(a)spatialinfo.com> ---------------------------------------------------------------------------- A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over, beginning with a working simple system. -- J Gall. "Systemantics: How systems work and how they fail" _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: OpenDirectory, DirectoryServices, OpenLDAP architecture questions
by Jason Townsend 16 Mar '05

16 Mar '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com DirectoryService is a daemon which provides various directory data to the rest of the operating system from a variety of sources. These are provided by plug-ins which use a common API. Current plug-ins include LDAPv3, NetInfo, BSD flat files and NIS, and others. Essentially DirectoryService acts like a combination of pam and the name-switching service on linux? lookupd services many of the POSIX style APIs for looking up information on users, groups, mounts, hosts, services, etc. For example: getpwnam(), getpwuid(), getaddrinfo(), gethostbyname() and so on. There is a plug-in to lookupd called DSAgent which calls the Directory Services API and provides information from the DirectoryService daemon to clients of lookupd. As described above, LDAPv3 is used in addition to the Password Server protocol. The Password Server protocol is based on SASL for any challenge response authentications, and there are some additional commands specific to it. Currently there is no documentation but the Password Server plug-in source is available from the DSPasswordServerPlugin project. Password Server uses either port 3659 or 106. 3659 is the preferred port for current implementations. For more on SASL, check out CMU's list of RFCs here: There are at least two cases for password changes: - a user changing his own password - an administrator changing another user's password One of my goals is to somehow get a pam module created for linux to interface with OpenDirectory. LDAP is adequate for many things, but talking to the SASL repository would be even better. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com DirectoryService provides an abstraction for reading directory data, as well as authentication (including secure authentications). It also handles write operations like creating and modifying records as well as changing passwords. So I think your statement is correct, but I believe this is a superset of what PAM and the name-switching service on Linux are providing. I have observed that on occasion when the ldap server goes weird, things slowly start to not work. Eventually, if left long enough, even DirectoryService freezes. If I kill it, it won't restart, but just freezes immediately. At this point, any application that makes any libc call that goes to lookupd or any DirectoryService call blocks indefinitely. This almost seems like a deadlock. IE the DirectoryService system cannot respond because LDAP is not responding (still accepting connections but blocks indefinitely). And at this point I cannot get lookupd or DirectoryService to restart because they both need to do some call that depends on LDAP. So it is kind of circular. The only way I can break the cycle is if I kill lookupd, DirectoryService, PasswordService, and slapd (whatever is still running) all simultaneously. Only then does the whole system unstick so that I can get slapd and all the other parts of OpenDirectory running again. In this case PasswordService won't start automatically so I have to start it manually. Have you tried sampling the various daemons when you get into this state? In 10.3.x, DirectoryService and lookupd are started by mach_init and should restart automatically. PasswordService should be restarted by watchdog. slapd is started by a startup item so you would need to restart it manually if you killed it. http://asg.web.cmu.edu/sasl/sasl-ietf-docs.html So essentially all an opendirectory client (say an iMac) uses is straight LDAP and SASL protocol? How are things like authenticating to change a password handled (the auth flag thing you were talking about)? Changing passwords is actually a fairly complicated topic once you get into the details... for the specific case of Password Server, it is necessary to first authenticate using the DHX authentication method to negotiate a session key and establish an encrypted channel, then the new password is sent over that encrypted channel. This falls more under the "additional commands" I referred to above as there is nothing in SASL for password changes. For the first case, there is a kDSStdAuthChangePasswd authentication method in Directory Services which allows the client to specify the old and new passwords (which is probably what you'd want to do from a PAM module I suppose). This does not require the authOnly flag to be set to false since it is a "one-shot" authentication method that includes the necessary credentials to make the change. In the case that you are an administrator using Workgroup Manager for example, the Password Server node reference is held onto by the LDAPv3 plug-in when authOnly is false. This allows that existing encrypted channel which was established when the administrator authenticated to be reused for set password operations that happen later on. This is handled internally to the LDAPv3 plug-in, so a DS client only needs to call dsDoDirNodeAuth using the kDSStdAuthSetPasswordAsRoot method (essentially, set the password with current credentials) in this case. Michael Bartosh's suggestion of using Kerberos works great if you have Kerberos set up. You should be able to change passwords and authenticate through Kerberos. It would be possible in theory to create a PAM module based on the DSPasswordServerPlugIn project that uses the Password Server to do the same thing as well. -Jason smime.p7s

1 0

Re: Sending notification from a time-constraint thread
by Mike Smith 16 Mar '05

16 Mar '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com In a server/client system we are developing, we need a time-constraint thread in the server to send a notification to another thread in the server. We use MIG generated RPC calls to communicate between threads. To be sure the time-constraint thread will not have to "wait" in any case, our first try was to use a "simpleroutine" type of RPC thinking that since return values were not expected, a simpleroutine would simply send the Mach message and return immediately. But it seems not to be the case, thus we tried to use a simpleroutine type of RPC with a timeout value of 0. But now the simpleroutine calls sometimes fails with a "error (ipc/send) timed out " message... You have probably filled the receiver's queue. Can this idea of notification be implemented using RPC calls with the desired behaviour (never blocking the time-constraint thread)? Or another strategy would be better? The thread wakeup implicit in the use of cvars is less likely to block, and since the queueing system for your notifications will be your own design, you can tweak it to have whatever behaviour you desire. = Mike _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Why do you do this? Unless you mean "between client and server", you would generally be much better off implementing a trivial messaging system using posix condition variables. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Real Time timers API
by Shawn Erickson 16 Mar '05

16 Mar '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On Mar 16, 2005, at 8:00 AM, Nicolas.Castagne(a)imag.fr wrote: Hi there, This software really needs various "real time timers". Under Linux, these timer are provided in the real time timer API (-lrt library) : timer_create timer_delete timer_settime etc. Under Mac Os X, so far, I only founded the (setitimer getitimer) API... But it does not allow to set various timer per process/thread. Is there on Mac OS X an API that correspond with the RT library under Linux (and Irix, and...) ? Where may I find documentation ? What type of product are you porting? Is it a driver/kernel level thing or a user mode application? -Shawn _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… I am porting quite a big software that was developped under Unix and Linux. This email sent to site_archiver(a)lists.apple.com

1 0

Real Time timers API
by Nicolas.Castagne＠imag.fr 16 Mar '05

16 Mar '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Importance: Normal User-agent: SquirrelMail/1.4.2 Hi there, I am porting quite a big software that was developped under Unix and Linux. This software really needs various "real time timers". Under Linux, these timer are provided in the real time timer API (-lrt library) : timer_create timer_delete timer_settime etc. Under Mac Os X, so far, I only founded the (setitimer getitimer) API... But it does not allow to set various timer per process/thread. Is there on Mac OS X an API that correspond with the RT library under Linux (and Irix, and...) ? Where may I find documentation ? Thx very much Nicolas _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Sending notification from a time-constraint thread
by Stéphane Letz 16 Mar '05

16 Mar '05

site_archiver(a)lists.apple.com Delivered-To: Darwin-dev(a)lists.apple.com Hi, Thanks Stephane Letz _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… In a server/client system we are developing, we need a time-constraint thread in the server to send a notification to another thread in the server. We use MIG generated RPC calls to communicate between threads. To be sure the time-constraint thread will not have to "wait" in any case, our first try was to use a "simpleroutine" type of RPC thinking that since return values were not expected, a simpleroutine would simply send the Mach message and return immediately. But it seems not to be the case, thus we tried to use a simpleroutine type of RPC with a timeout value of 0. But now the simpleroutine calls sometimes fails with a "error (ipc/send) timed out " message... Can this idea of notification be implemented using RPC calls with the desired behaviour (never blocking the time-constraint thread)? Or another strategy would be better? This email sent to site_archiver(a)lists.apple.com

1 0