June 2004 - Darwin-kernel - lists.apple.com

Re: Monitoring File Access (Read / Write)
by Justin Walker 17 Jun '04

17 Jun '04

On Jun 17, 2004, at 15:24, Brian Bergstrand wrote: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jun 17, 2004, at 4:42 PM, Justin Walker wrote: On Jun 17, 2004, at 14:15, Krishna Monian wrote: [snip] The only way to do this is with some kind of kernel assistance. Hacking into dispatch tables won't, in all likelihood, survive new releases of the kernel, though. This approach is frowned on. I'd suggest, as someone already has, looking at ktrace, or perhaps, fs_usage. The source for both is available. This is a polling type of interface, though, so you need to apply it with some care and forethought. There is no notification mechanism for file access. That's not quite true. If the OP can require Darwin7+, then he can use the kqueue interface to get everything he needs except for reads -- and w/o polling or being in the kernel. See <sys/event.h>. That's right; I'd forgotten about kqueue (one of many new tricks I need to learn). My point about kernel assistance was that some sort of kernel mechanism, like those used by ktrace/fs_usage, is needed. There is no good reason to implement something new inside the kernel, for the oft-mentioned variety of reasons that can be found in the archives. For tracking reads, I think you are SOL. Thanks for the correction. Regards, Justin -- Justin C. Walker, Curmudgeon-At-Large * Institute for General Semantics | "Weaseling out of things is what | separates us from the animals. | Well, except the weasel." | - Homer J Simpson *--------------------------------------*-------------------------------* _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Monitoring File Access (Read / Write)
by Krishna Monian 17 Jun '04

17 Jun '04

I think fs_usage might just do the trick, although I am not yet sure. I will have to test it out more thoroughly. 1 thing I noticed about fs_usage was that when u read a file say using more from the shell, it doesn't display the entire path of the file. Is there any way to make it do this? Thanks Krishna --- Justin Walker <justin(a)mac.com> wrote: > On Jun 17, 2004, at 14:15, Krishna Monian wrote: > > > Hi All, > > > > Is it possible to write some sort of kernel mode > app > > that hooks on to all file system calls / IO calls > and > > logs all of these ? > > No really. Since the source is available, you can > always hack away to > your heart's content, but this kind of approach > really isn't going to > work in the long run. In addition, it only works on > your system (or on > those belonging to people you can con into running > your kernel :-}). > > > I took a look at the File Alteration Monitor (FAM) > but > > that doesn't achieve this completely. It doesn't > log > > file reads. > > > > I am basically trying to write a utility that will > log > > every file that is accessed (read / write) in a > > particular directory in the system. > > > > What would be the best way to go about doing this? > Is > > a kernel hook the only way? Is there anything that > has > > already been done along these lines (quite sure it > > has)? > > The only way to do this is with some kind of kernel > assistance. > Hacking into dispatch tables won't, in all > likelihood, survive new > releases of the kernel, though. This approach is > frowned on. > > I'd suggest, as someone already has, looking at > ktrace, or perhaps, > fs_usage. The source for both is available. This > is a polling type of > interface, though, so you need to apply it with some > care and > forethought. There is no notification mechanism for > file access. > > Regards, > > Justin > > -- > Justin C. Walker, Curmudgeon-At-Large * > Institute for General Semantics | Men are > from Earth. > | Women > are from Earth. > | Deal > with it. > *--------------------------------------*-------------------------------* > _______________________________________________ > darwin-kernel mailing list | > darwin-kernel(a)lists.apple.com > Help/Unsubscribe/Archives: > http://www.lists.apple.com/mailman/listinfo/darwin-kernel > Do not post admin requests to the list. They will be > ignored. > > __________________________________ Do you Yahoo!? Read only the mail you want - Yahoo! Mail SpamGuard. http://promotions.yahoo.com/new_mail _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Monitoring File Access (Read / Write)
by Brian Bergstrand 17 Jun '04

17 Jun '04

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jun 17, 2004, at 4:42 PM, Justin Walker wrote: > On Jun 17, 2004, at 14:15, Krishna Monian wrote: >> >> I am basically trying to write a utility that will log >> every file that is accessed (read / write) in a >> particular directory in the system. >> >> What would be the best way to go about doing this? Is >> a kernel hook the only way? Is there anything that has >> already been done along these lines (quite sure it >> has)? > > The only way to do this is with some kind of kernel assistance. > Hacking into dispatch tables won't, in all likelihood, survive new > releases of the kernel, though. This approach is frowned on. > > I'd suggest, as someone already has, looking at ktrace, or perhaps, > fs_usage. The source for both is available. This is a polling type > of interface, though, so you need to apply it with some care and > forethought. There is no notification mechanism for file access. > That's not quite true. If the OP can require Darwin7+, then he can use the kqueue interface to get everything he needs except for reads -- and w/o polling or being in the kernel. See <sys/event.h>. For tracking reads, I think you are SOL. HTH. > Regards, > > Justin > Brian Bergstrand <http://www.bergstrand.org/brian/>, AIM: triryche206 PGP Key: <http://www.bergstrand.org/brian/misc/public_key.txt> Home computers are being called upon to perform many new functions, including the consumption of homework formerly eaten by the dog. - Doug Larson As of 05:01:41 PM, iTunes is playing "Think About You" from "Appetite For Destruction" by "Guns N' Roses" -----BEGIN PGP SIGNATURE----- Version: PGP 8.1 iQA/AwUBQNIL+XnR2Fu2x7aiEQKzTACg4gZX1oCFMx+Qd45Ngyn4o9eiuHUAnA5p svxMFMx2M2myRoYLgMaAn3wB =8ykt -----END PGP SIGNATURE----- _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Timers in my NKE
by Matt Jaffa 17 Jun '04

17 Jun '04

So if my thread is not scheduled to be called 5 minutes down the road, what if step 3 is called, So what if thread_call_free(tc) is called before the timeout on the thread, will that cause problems? Matt On Jun 17, 2004, at 3:58 PM, Godfrey van der Linden wrote: There is some pretty simple sample code in xnu/iokit/Kernel/IOTimerEventSource.cpp However in general it is easy to use the thread_call APIs Step 1> tc = thread_call_allocate(...); Step 2> While needed call thread_call_enter(tc, sometimeout); Step 3> thread_call_free(tc) Godfrey On Jun 17, , at 14:15, Matt Jaffa wrote: The reason why I was weary about the thread_call stuff was cause I don't want to have to keep calling thread_call_allocate everytime the thread needs to be called, or do I just allocate it once, say as a global, and allocate it in my Start method and also I guess do the same with the thread_call_function and then in the actual send function that needs to call the thread, then use those global parameters and create the call to the thread, this just seems like so much more work. Matt On Jun 17, 2004, at 3:03 PM, Godfrey van der Linden wrote: G'day Matt, The problem with IOCreateThread/IOSleep is that you are tying up a lot of kernel resource to little effect. <kern/thread_call.h> and BSD's <sys/systm.h> both use a more efficient model which is to use a shared pool of threads. 'thread_call' has the advantage that it is efficient to rearm and re-schedule. But timeout in <sys/systm.h> is better known and fairly easy to use. Don't worry about either of these APIs going anytime soon. Godfrey _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Timers in my NKE
by Godfrey van der Linden 17 Jun '04

17 Jun '04

There is some pretty simple sample code in xnu/iokit/Kernel/IOTimerEventSource.cpp However in general it is easy to use the thread_call APIs Step 1> tc = thread_call_allocate(...); Step 2> While needed call thread_call_enter(tc, sometimeout); Step 3> thread_call_free(tc) Godfrey On Jun 17, , at 14:15, Matt Jaffa wrote: The reason why I was weary about the thread_call stuff was cause I don't want to have to keep calling thread_call_allocate everytime the thread needs to be called, or do I just allocate it once, say as a global, and allocate it in my Start method and also I guess do the same with the thread_call_function and then in the actual send function that needs to call the thread, then use those global parameters and create the call to the thread, this just seems like so much more work. Matt On Jun 17, 2004, at 3:03 PM, Godfrey van der Linden wrote: G'day Matt, The problem with IOCreateThread/IOSleep is that you are tying up a lot of kernel resource to little effect. <kern/thread_call.h> and BSD's <sys/systm.h> both use a more efficient model which is to use a shared pool of threads. 'thread_call' has the advantage that it is efficient to rearm and re-schedule. But timeout in <sys/systm.h> is better known and fairly easy to use. Don't worry about either of these APIs going anytime soon. Godfrey _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Monitoring File Access (Read / Write)
by Justin Walker 17 Jun '04

17 Jun '04

On Jun 17, 2004, at 14:15, Krishna Monian wrote: Hi All, Is it possible to write some sort of kernel mode app that hooks on to all file system calls / IO calls and logs all of these ? No really. Since the source is available, you can always hack away to your heart's content, but this kind of approach really isn't going to work in the long run. In addition, it only works on your system (or on those belonging to people you can con into running your kernel :-}). I took a look at the File Alteration Monitor (FAM) but that doesn't achieve this completely. It doesn't log file reads. I am basically trying to write a utility that will log every file that is accessed (read / write) in a particular directory in the system. What would be the best way to go about doing this? Is a kernel hook the only way? Is there anything that has already been done along these lines (quite sure it has)? The only way to do this is with some kind of kernel assistance. Hacking into dispatch tables won't, in all likelihood, survive new releases of the kernel, though. This approach is frowned on. I'd suggest, as someone already has, looking at ktrace, or perhaps, fs_usage. The source for both is available. This is a polling type of interface, though, so you need to apply it with some care and forethought. There is no notification mechanism for file access. Regards, Justin -- Justin C. Walker, Curmudgeon-At-Large * Institute for General Semantics | Men are from Earth. | Women are from Earth. | Deal with it. *--------------------------------------*-------------------------------* _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Monitoring File Access (Read / Write)
by Krishna Monian 17 Jun '04

17 Jun '04

Hi All, Is it possible to write some sort of kernel mode app that hooks on to all file system calls / IO calls and logs all of these ? I took a look at the File Alteration Monitor (FAM) but that doesn't achieve this completely. It doesn't log file reads. I am basically trying to write a utility that will log every file that is accessed (read / write) in a particular directory in the system. What would be the best way to go about doing this? Is a kernel hook the only way? Is there anything that has already been done along these lines (quite sure it has)? Any help on this will be greatly appreciated. Thanks Krishna Monian __________________________________ Do you Yahoo!? Yahoo! Mail - Helps protect you from nasty viruses. http://promotions.yahoo.com/new_mail _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Timers in my NKE
by Matt Jaffa 17 Jun '04

17 Jun '04

The reason why I was weary about the thread_call stuff was cause I don't want to have to keep calling thread_call_allocate everytime the thread needs to be called, or do I just allocate it once, say as a global, and allocate it in my Start method and also I guess do the same with the thread_call_function and then in the actual send function that needs to call the thread, then use those global parameters and create the call to the thread, this just seems like so much more work. Matt On Jun 17, 2004, at 3:03 PM, Godfrey van der Linden wrote: G'day Matt, The problem with IOCreateThread/IOSleep is that you are tying up a lot of kernel resource to little effect. <kern/thread_call.h> and BSD's <sys/systm.h> both use a more efficient model which is to use a shared pool of threads. 'thread_call' has the advantage that it is efficient to rearm and re-schedule. But timeout in <sys/systm.h> is better known and fairly easy to use. Don't worry about either of these APIs going anytime soon. Godfrey On Jun 17, , at 13:26, Matt Jaffa wrote: hmmm I found a pretty good method for accomplishing my purposes, unless it isn't a good way. use IOCreateThread to create the thread and run it, then inside the thread function I call IOSleep(timetosleep); then so the work after the thread wakes up. If I read correctly these are just wrappers for the mach_threads, and I guess IOKit knows threads better to me, so I will let that API take care of it, as long as this API is going to be staying along time. I don't really care if the thread wakes up anywhere from 1 to 10000 milliseconds off from what I told it to sleep, so this method should work for me, unless there is a suggested and recommended way. Matt On Jun 17, 2004, at 12:30 PM, Godfrey van der Linden wrote: G'day Justin, You are right it is #include <kern/thread_call.h> // And can be found in xnu/osfmk/kern It sort of depends on what you want to do with the timeou()t. But for regularly scheduled timeouts I prefer using thread_call_enter directly as it is lighter weight and closer to my requirements. Godfrey On Jun 17, , at 10:38, Justin Walker wrote: On Jun 17, 2004, at 10:11, Godfrey van der Linden wrote: Check out <xnu/kern/thread_call.h> Huh? I don't find such a file. What about a 'timeout()' call? _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Timers in my NKE
by Matt Jaffa 17 Jun '04

17 Jun '04

ok, I got pretty far with the stuff in <kern/thread_call.h> except I am not familiar with the thread_call stuff, like the function and everything actually. Any examples you can provide it you have used this before, in my NKE the send function that gets attached to each socket will call this thread under certain conditions, I would like to use the way that uses the less resources. Matt On Jun 17, 2004, at 3:03 PM, Godfrey van der Linden wrote: G'day Matt, The problem with IOCreateThread/IOSleep is that you are tying up a lot of kernel resource to little effect. <kern/thread_call.h> and BSD's <sys/systm.h> both use a more efficient model which is to use a shared pool of threads. 'thread_call' has the advantage that it is efficient to rearm and re-schedule. But timeout in <sys/systm.h> is better known and fairly easy to use. Don't worry about either of these APIs going anytime soon. Godfrey On Jun 17, , at 13:26, Matt Jaffa wrote: hmmm I found a pretty good method for accomplishing my purposes, unless it isn't a good way. use IOCreateThread to create the thread and run it, then inside the thread function I call IOSleep(timetosleep); then so the work after the thread wakes up. If I read correctly these are just wrappers for the mach_threads, and I guess IOKit knows threads better to me, so I will let that API take care of it, as long as this API is going to be staying along time. I don't really care if the thread wakes up anywhere from 1 to 10000 milliseconds off from what I told it to sleep, so this method should work for me, unless there is a suggested and recommended way. Matt On Jun 17, 2004, at 12:30 PM, Godfrey van der Linden wrote: G'day Justin, You are right it is #include <kern/thread_call.h> // And can be found in xnu/osfmk/kern It sort of depends on what you want to do with the timeou()t. But for regularly scheduled timeouts I prefer using thread_call_enter directly as it is lighter weight and closer to my requirements. Godfrey On Jun 17, , at 10:38, Justin Walker wrote: On Jun 17, 2004, at 10:11, Godfrey van der Linden wrote: Check out <xnu/kern/thread_call.h> Huh? I don't find such a file. What about a 'timeout()' call? _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Timers in my NKE
by Godfrey van der Linden 17 Jun '04

17 Jun '04

G'day Matt, The problem with IOCreateThread/IOSleep is that you are tying up a lot of kernel resource to little effect. <kern/thread_call.h> and BSD's <sys/systm.h> both use a more efficient model which is to use a shared pool of threads. 'thread_call' has the advantage that it is efficient to rearm and re-schedule. But timeout in <sys/systm.h> is better known and fairly easy to use. Don't worry about either of these APIs going anytime soon. Godfrey On Jun 17, , at 13:26, Matt Jaffa wrote: hmmm I found a pretty good method for accomplishing my purposes, unless it isn't a good way. use IOCreateThread to create the thread and run it, then inside the thread function I call IOSleep(timetosleep); then so the work after the thread wakes up. If I read correctly these are just wrappers for the mach_threads, and I guess IOKit knows threads better to me, so I will let that API take care of it, as long as this API is going to be staying along time. I don't really care if the thread wakes up anywhere from 1 to 10000 milliseconds off from what I told it to sleep, so this method should work for me, unless there is a suggested and recommended way. Matt On Jun 17, 2004, at 12:30 PM, Godfrey van der Linden wrote: G'day Justin, You are right it is #include <kern/thread_call.h> // And can be found in xnu/osfmk/kern It sort of depends on what you want to do with the timeou()t. But for regularly scheduled timeouts I prefer using thread_call_enter directly as it is lighter weight and closer to my requirements. Godfrey On Jun 17, , at 10:38, Justin Walker wrote: On Jun 17, 2004, at 10:11, Godfrey van der Linden wrote: Check out <xnu/kern/thread_call.h> Huh? I don't find such a file. What about a 'timeout()' call? _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0