January 2006 - Darwin-dev - lists.apple.com

Re: How to list all files opened by a process on the Mac OS X?
by Jonas Maebe 24 Jan '06

24 Jan '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On 24 jan 2006, at 11:06, Alexander von Below wrote: char * kvm_getfiles(kd, op, arg, cnt) kvm_t *kd; int op, arg; int *cnt; { return (0); } Jonas _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On the subject, I have just reviewed the latest Libkvm code as of OS X 10.4.4, and it is indeed scary: Anyone else here know a way to get a list of open files? Since lsof can do that, you can look at its source. I don't know which Darwin project it's part of though. This email sent to site_archiver(a)lists.apple.com

1 0

Re: How to list all files opened by a process on the Mac OS X?
by Alexander von Below 24 Jan '06

24 Jan '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com char * kvm_getfiles(kd, op, arg, cnt) kvm_t *kd; int op, arg; int *cnt; { return (0); } Anyone else here know a way to get a list of open files? Alex Am 24.01.2006 um 10:48 schrieb Alexander von Below: Hi all! How can I list all open files of one process? I have used sysctl to list many process infomations, but I cannot find a way to get something like the vnode or file descripters of one process. What should I do? _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On the subject, I have just reviewed the latest Libkvm code as of OS X 10.4.4, and it is indeed scary: Thanks -- Ben Chen, Apple Technology Lab of Software Engineering School, Tongji University _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/below%40mac.com This email sent to below(a)mac.com This email sent to site_archiver(a)lists.apple.com

1 0

Re: How to list all files opened by a process on the Mac OS X?
by Alexander von Below 24 Jan '06

24 Jan '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Hello, You might want to look at this example: http://gobsd.com/code/freebsd/lib/libkvm/kvm_file.c Bye Alex Am 25.12.2005 um 17:32 schrieb Chen Ben: Hi all! How can I list all open files of one process? I have used sysctl to list many process infomations, but I cannot find a way to get something like the vnode or file descripters of one process. What should I do? _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… I am currently working on this, too, and sysctl sounds to be a good way to got with CTL_KERN and KERN_FILE. Something that scared me was this post: http://lists.apple.com/ archives/darwin-kernel/2004/Mar/msg00034.html but I will try if that still is the case Thanks -- Ben Chen, Apple Technology Lab of Software Engineering School, Tongji University _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/below%40mac.com This email sent to below(a)mac.com This email sent to site_archiver(a)lists.apple.com

1 0

Re: assembly instrcutions asm/io.h alternative
by Matt Watson 24 Jan '06

24 Jan '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com What does the compiler say if you don't #include <asm/io.h> ? matt. On Jan 23, 2006, at 11:36 PM, rohit dhamija wrote: Hi, Working on Mac OS 10.4 Any suggestions/comments are welcome. Regards, -- _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/mwatson%40apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… I am porting a win32 based C file to Mac platform and using gcc to compile, the file includes some assembly level instructions and includes #include <asm/io.h> file. But there is not such asm folder in /usr/include Is there any alternative to io.h file in Mac OS ? Although the same file exists in Linux OS. This email sent to mwatson(a)apple.com This email sent to site_archiver(a)lists.apple.com

1 0

assembly instrcutions asm/io.h alternative
by rohit dhamija 24 Jan '06

24 Jan '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:mime-version:content-type; b=Gt+c8jyM7V9bQ8vrrGP/hF9HSZXvyBghiqDBQ0KF5Ko881AkVgYP3+5RF/lKiVKtSIx2jzrk6T9Sxx7WVEZ9usHcX39Ac5OnxvNHRLd4tLibsTVKXitJkA+QgCgCqyAMeqRAtV3AEcrd5WZrjq9nVrMAFO9rmd5exrC+U+CfDxg= _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: intercepting system calls?
by Andrew White 23 Jan '06

23 Jan '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com User-agent: Mozilla Thunderbird 0.9 (Macintosh/20041103) Erik Paulson wrote: One process really shouldn't be hacking the syscall API of a different process. -- Andrew White -------------------------------------------------------------------------- This email and any attachments may be confidential. They may contain legally privileged information or copyright material. You should not read, copy, use or disclose them without authorisation. If you are not an intended recipient, please contact us at once by return email and then delete both messages. We do not accept liability in connection with computer virus, data corruption, delay, interruption, unauthorised access or unauthorised amendment. This notice should not be removed. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… No, I'm looking for an unprivileged interface. PTRACE_SYSCALL means that I can run two processes as user 'nobody' - the first one is the actual process that I want to run on idle computers at the UofI, and the second process, also running as 'nobody', can trace the first process, and if need be do an RPC back to a server running at the UofW. I don't want to run privileged code on the UofI machines. Such an unprivileged interface doesn't appear to exist, short of some sort of binary rewriting or single stepping. The clean way to do this in user space is to create a library shim that you send your filesystem calls through. You can then attach to this shim and redirect the appropriate FS calls or let them pass through to the underlying system libraries (frameworks). Yes, this means you need to recompile your applications. That's life if you want user-space interaction. If you can't recompile the application (because it's someone else's code) then it's pretty dangerous to be trying to snarf calls at the framework / kernel interface since there's no actual guarantee which calls are relevant to you or that you have them all, nor is there any version control. The other place to intercept is at a recognised extension API, such as the filesystem API. As you noticed, this is typically a privileged operation. This email sent to site_archiver(a)lists.apple.com

1 0

Re: intercepting system calls?
by Mike Smith 23 Jan '06

23 Jan '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On Jan 23, 2006, at 1:43 PM, Erik Paulson wrote: This sounds like an administrative misconception revolving around traditional use of network filesystems. Hrm, without going way off-topic I don't think I can convince you that my motivation is real - my simple attempt at it is: If I want to use idle computers at the University of Illinois and I'm at the University of Wisconsin, I either have to get the UofI to mount my file systems, or copy the files down to the UofI, or interpose and make the processes running at the UofI believe that they're running at the UofW. You're already talking about running privileged code on the client system in order to intercept system calls; Such an unprivileged interface doesn't appear to exist, short of some sort of binary rewriting or single stepping. = Mike _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… A shared filesystem is great, if the administrators of both computers agree to use the same shared filesystems. That usually doesn't happen between institutions, for example. I don't think this is off-topic at all; the issue here is how to go about doing what you want to do, and how that relates to the way that Darwin works. That sounds a lot more ambitious than just files. Does "running at the UoW" mean "they see a filesystem at UoW", or does it include everything else that they would see at UoW as well? Network interfaces? Context in other parts of the system? No, I'm looking for an unprivileged interface. PTRACE_SYSCALL means that I can run two processes as user 'nobody' - the first one is the actual process that I want to run on idle computers at the UofI, and the second process, also running as 'nobody', can trace the first process, and if need be do an RPC back to a server running at the UofW. I don't want to run privileged code on the UofI machines. Ok; here's where we got a bit of a disconnect. The runtime environment on Mac OS (which is primarily what Darwin lives to support) is a lot more complicated than that on something like Linux (unless you bring KDE into the picture, and even then...). It's not sufficient in the general sense to redirect a single application; there are quite a large number of situations in which work is done for an application by proxy, and to have that happen correctly the proxy has to have relevant context. Now, arguably you can redirect the proxy requests, but now you're talking about handling distributed shared memory as well, and don't forget the heavy use of Mach-style messaging. A general mechanism needs to take all these things into account. One built on top of intercepting just BSD system calls is going to be a partial solution at best, and as such better left undone. You can achieve this with the dynamic linker interfaces already mentioned. This email sent to site_archiver(a)lists.apple.com

1 0

Re: intercepting system calls?
by Erik Paulson 23 Jan '06

23 Jan '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com User-agent: Mutt/1.4.2.1i On Mon, Jan 23, 2006 at 01:30:31PM -0800, Mike Smith wrote: > > On Jan 23, 2006, at 12:02 PM, darwin-dev-request(a)lists.apple.com wrote: > > >On Mon, Jan 23, 2006 at 01:00:26AM -0800, Mike Smith wrote: > >>On Jan 22, 2006, at 12:02 PM, darwin-dev-request(a)lists.apple.com > >>wrote: > >> > >>>I want to be able to capture the I/O calls of an application and > >>>then > >>>ship them back over the network to be handled by another machine. > >> > >>There are plenty of examples of this in the system already; > >>they're called > >>"network filesystems". One of them is even called "the Network > >>File System". > > > >A shared filesystem is great, if the administrators of both computers > >agree to use the same shared filesystems. That usually doesn't happen > >between institutions, for example. > > This sounds like an administrative misconception revolving around > traditional > use of network filesystems. > Hrm, without going way off-topic I don't think I can convince you that my motivation is real - my simple attempt at it is: If I want to use idle computers at the University of Illinois and I'm at the University of Wisconsin, I either have to get the UofI to mount my file systems, or copy the files down to the UofI, or interpose and make the processes running at the UofI believe that they're running at the UofW. > You're already talking about running privileged code on the client > system > in order to intercept system calls; No, I'm looking for an unprivileged interface. PTRACE_SYSCALL means that I can run two processes as user 'nobody' - the first one is the actual process that I want to run on idle computers at the UofI, and the second process, also running as 'nobody', can trace the first process, and if need be do an RPC back to a server running at the UofW. I don't want to run privileged code on the UofI machines. Such an unprivileged interface doesn't appear to exist, short of some sort of binary rewriting or single stepping. -Erik _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: intercepting system calls?
by Mike Smith 23 Jan '06

23 Jan '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On Jan 23, 2006, at 12:02 PM, darwin-dev-request(a)lists.apple.com wrote: A shared filesystem is great, if the administrators of both computers agree to use the same shared filesystems. That usually doesn't happen between institutions, for example. I'd point you to our papers for more motivating examples, but there are times when if we want to access data remotely we can't count on a shared filesystem, and have to use some sort of interposition: http://www.cs.wisc.edu/condor/publications.html (or a few other conferences that have papers in the same space: IEEE High Performance Distributed Computing: http://www.hpdc.org/ USENIX WORLDS: http://www.usenix.org/events/bytopic/worlds.html = Mike _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On Mon, Jan 23, 2006 at 01:00:26AM -0800, Mike Smith wrote: On Jan 22, 2006, at 12:02 PM, darwin-dev-request(a)lists.apple.com wrote: I want to be able to capture the I/O calls of an application and then ship them back over the network to be handled by another machine. There are plenty of examples of this in the system already; they're called "network filesystems". One of them is even called "the Network File System". This sounds like an administrative misconception revolving around traditional use of network filesystems. You're already talking about running privileged code on the client system in order to intercept system calls; a network filesystem implementation on the client is no different in degree. There is no need for the server side of the equation to require administrative intervention; if you are currently intercepting client-side file I/O and handling it on the server, you already have a network fileservice protocol. The only argument I can see for requiring interposition over simply having a portion of the filesystem namespace dedicated to remote access is when you want to subvert the local namespace. From an architectural perspective that's pretty undesirable. It also seems like Apple could keep the system call interface stable between versions - the great thing about system calls is if you need to change the meaning of one, you can just create another system call with a different name. You can leave the old one stable, and just change the framework to call the new one - if someone has found a way to call the actual system call, the can keep using it. This is a naive assumption. It is much easier to version at the shared library interface; locking down the system call layer introduces a lot of problems, especially if you guarantee the interface as ABI. If you've got time, could you expand on this? Keep in mind I'm assuming that the main interface layer is the shared library, so most applications use whatever system calls the library uses, either a new version of the old system call, or a new system call. My claim is that Apple can leave the old system call in place and "version" by just introducing a new system call. The *only* interface permitted to the kernel is through the system frameworks. If the system call interface were exposed as ABI, it would be impossible to refactor the kernel/framework relationship without incurring substantial overhead in the form of "legacy support" code. This isn't one-time overhead either; as the internals of the kernel evolve, it has to be kept up. Applications that (at their own peril) bypass the shared library will keep working, and any application that uses the shared library gets the new interface. (I'll admit my evidence is "this approach works for Linux" :) Having watched Linux for over a decade now, I would contest your definition of "works". Far too much effort has been wasted, and far too many users have suffered over the relative instability and lack of coherent behaviour in that area. 8( This email sent to site_archiver(a)lists.apple.com

1 0

Re: intercepting system calls?
by Erik Paulson 23 Jan '06

23 Jan '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com User-agent: Mutt/1.4.2.1i On Mon, Jan 23, 2006 at 01:00:26AM -0800, Mike Smith wrote: > > On Jan 22, 2006, at 12:02 PM, darwin-dev-request(a)lists.apple.com wrote: > > >I want to be able to capture the I/O calls of an application and then > >ship them back over the network to be handled by another machine. > > There are plenty of examples of this in the system already; they're > called > "network filesystems". One of them is even called "the Network File > System". > A shared filesystem is great, if the administrators of both computers agree to use the same shared filesystems. That usually doesn't happen between institutions, for example. I'd point you to our papers for more motivating examples, but there are times when if we want to access data remotely we can't count on a shared filesystem, and have to use some sort of interposition: http://www.cs.wisc.edu/condor/publications.html (or a few other conferences that have papers in the same space: IEEE High Performance Distributed Computing: http://www.hpdc.org/ USENIX WORLDS: http://www.usenix.org/events/bytopic/worlds.html > >It also seems like Apple could keep the system call interface stable > >between versions - the great thing about system calls is if you > >need to > >change the meaning of one, you can just create another system call > >with > >a different name. You can leave the old one stable, and just change > >the > >framework to call the new one - if someone has found a way to call the > >actual system call, the can keep using it. > > This is a naive assumption. It is much easier to version at the shared > library interface; locking down the system call layer introduces a > lot of > problems, especially if you guarantee the interface as ABI. > If you've got time, could you expand on this? Keep in mind I'm assuming that the main interface layer is the shared library, so most applications use whatever system calls the library uses, either a new version of the old system call, or a new system call. My claim is that Apple can leave the old system call in place and "version" by just introducing a new system call. Applications that (at their own peril) bypass the shared library will keep working, and any application that uses the shared library gets the new interface. (I'll admit my evidence is "this approach works for Linux" :) At any rate, our experience is that it's easier to interpose at the system call interface instead of the library interface to get the sort of virtualization we want. It sounds like the interface we want is not available in Darwin, and there's not really something equivalent. Thanks to everyone who answered! -Erik _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0