April 2007 - Darwin-kernel - lists.apple.com

Re: Info needed for File System Development
by John Davidorff Pell 30 Apr '07

30 Apr '07

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Dkim-signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:in-reply-to:references:mime-version:content-type:message-id:cc:from:subject:date:to:x-mailer; b=kY7dMktBm/+uOD5DsOpKsWCmyjA672oTzl2o6A4Bsxj+uUuupH0ecSSRNwAwHEou8V6OKN4wXCUO56TIc5CPsvl1kcjOCnM9U6yZ16x2hM7EQcW49UgfetbTwHaK+WybcexfOth+K37yMyjNI5hMahECNLKrWB8V4S5fa90h/hA= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:in-reply-to:references:mime-version:content-type:message-id:cc:from:subject:date:to:x-mailer; b=c45OpxVJL36UJKS0FRtNASVxEGfOftXWzO9bwpBYr+1Sfcjue8vbdq1+b4R5UdVy+xArYnDb8xUoWVzdFl6A5GBJFLPOoxtLE5inHoG/3dCf7+LUMW6K3hdF06MQN1RTbSgHR1qX6OyTEJrobYLsfSbMQUVVFWrxSiFpbrkzNew= Hi, JP On Apr 30, 2007, at 5:41 AM, Shailesh Parulekar wrote: Firstly,Thanks all for extending your help. Thanks again, --shailesh parulekar From: Michael Smith <drivers(a)mu.org> To: Shailesh Parulekar <smileyshailoo(a)hotmail.com> CC: darwin-kernel(a)lists.apple.com Subject: Re: Info needed for File System Development Date: Sun, 29 Apr 2007 14:43:39 +0930 On Apr 26, 2007, at 6:13 PM, Shailesh Parulekar wrote: What are the "needs of your product"? = Mike _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com This sounds much like you're trying to add full journaling to HFS+. Is this a fair characterisation? About, need of our product:: At a very high level, I wish to intercept the I/Os (essentially Writes) made to a set of files and store the changes in my secondary storage so that in the event of machine crash user should be able to reconstruct his/her files with the help of the product. In general terminology it is Continuous Data Protection (C.D.P.) My intention is to track the block changes that happen to the files and have my meta data pointing to the old (changed) blocks so that I get to construct various versions of the files from my meta data. As and when plans are crystallized I will have more concrete questions to chalk down. For now I just need to get started with the Mac OS X Internals and think of possible avenues to meet my requirements. I need to develop a Filter driver to plug-in into the MAC File System in order to 'filter' the read/writes to the files and process the same as per the need of our product. _________________________________________________________________ Mortgage refinance is Hot. *Terms. Get a 5.375%* fix rate. Check savings https://www2.nextag.com/goto.jsp?product=100000035&url=% 2fst.jsp&tm=y&search=mortgage_text_links_88_h2bbb&disc=y&vers=925&s=40 56&p=5117 _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/johnpell% 40gmail.com This email sent to johnpell(a)gmail.com -- <Beeth> Girls are like internet domain names, the ones I like are already taken. <honx> well, you can stil get one from a strange country :-P -- geekissues.org quote #369 smime.p7s

1 0

buildin kernel 8.9.0
by cristian pancià 30 Apr '07

30 Apr '07

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Dkim-signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:mime-version:content-type; b=lqhDR6Nck3AJvLymKciu0MSU01k4L1IOtJvyXSQ70g9tF/atPiFQtqVRJvnhdYjyvNFk/1z44LAjHGFBxU0M0OunPF8ngRnYcmF8w0mEFxD4J51i7z67N52d//dmFxy5ebzM7ieQVu7CjxPZptiUuTIx6Mp4MXhrflC54X7+FUA= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:mime-version:content-type; b=hIudyaM707PNrKIs2CGeq3rftxX9mveuvCwmswwCp6b9Vz62t+PNKZDORXRXp7RnLmMIfyYr9lqsdh1AVol3IR1dYs2E5pA/35eEfup2dVhwhawOHvPTOqcOcn2yPv72G0/t8TT+uQusmpicRVKszSmKvWIVX04DinQUbzv5TDY= sudo mkdir -p /System/Library/Frameworks/IOKit.framework/Versions/A/PrivateHeaders/kext cd /System/Library/Frameworks/IOKit.framework/ sudo ln -s Versions/A/PrivateHeaders PrivateHeaders sudo cp PATH_TO_IOKITUSER/IOKitUser-version/kext.subproj/*.h PrivateHeaders/kext cd PATH_TO_KEXT_TOOLS/kext_tools-version gcc kextsymboltool.c -o kextsymboltool _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Info needed for File System Development
by Michael Smith 30 Apr '07

30 Apr '07

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On Apr 30, 2007, at 10:41 PM, Shailesh Parulekar wrote: = Mike _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… About, need of our product:: At a very high level, I wish to intercept the I/Os (essentially Writes) made to a set of files and store the changes in my secondary storage so that in the event of machine crash user should be able to reconstruct his/her files with the help of the product. In general terminology it is Continuous Data Protection (C.D.P.) My intention is to track the block changes that happen to the files and have my meta data pointing to the old (changed) blocks so that I get to construct various versions of the files from my meta data. Are these files specifically managed by applications that are aware of the presence of your product, or is it your intention that the product work with data files maintained by any application? If the former, this sounds like something best handled with a framework or plugin in user space. Otherwise, how do you intend to determine when a file is in a consistent state? If you were to make the simplistic assumption that all file write operations are transactions bracketed by open/close pairs, then the easiest approach is just to catch the last close on a file using the kauth FILEOP scope and replicate it wholesale. If you are trying to provide CDP in the case of files that are open for long periods of time by applications that are not transactional and assume a reliable filesystem, then you are already in an impossible situation; there is no way for you to determine when a file is in a consistent state. Data is not necessarily retired to files in program order. Sensible applications that hold files open for long periods of time tend to enforce some degree of transactionality by the use of the F_FULLFSYNC fcntl. You may have to do some work to detect this at the vnode layer. As and when plans are crystallized I will have more concrete questions to chalk down. For now I just need to get started with the Mac OS X Internals and think of possible avenues to meet my requirements. Definitely start with Amit's book. There is not a supported mechanism for filtering vnode operations to filesystems, so your approach may have to vary depending on which filesystem(s) you plan to support. Patching the vnode table is not viable. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Info needed for File System Development
by Shailesh Parulekar 30 Apr '07

30 Apr '07

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Firstly,Thanks all for extending your help. Thanks again, --shailesh parulekar From: Michael Smith <drivers(a)mu.org> To: Shailesh Parulekar <smileyshailoo(a)hotmail.com> CC: darwin-kernel(a)lists.apple.com Subject: Re: Info needed for File System Development Date: Sun, 29 Apr 2007 14:43:39 +0930 On Apr 26, 2007, at 6:13 PM, Shailesh Parulekar wrote: What are the "needs of your product"? = Mike _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… About, need of our product:: At a very high level, I wish to intercept the I/Os (essentially Writes) made to a set of files and store the changes in my secondary storage so that in the event of machine crash user should be able to reconstruct his/her files with the help of the product. In general terminology it is Continuous Data Protection (C.D.P.) My intention is to track the block changes that happen to the files and have my meta data pointing to the old (changed) blocks so that I get to construct various versions of the files from my meta data. As and when plans are crystallized I will have more concrete questions to chalk down. For now I just need to get started with the Mac OS X Internals and think of possible avenues to meet my requirements. I need to develop a Filter driver to plug-in into the MAC File System in order to 'filter' the read/writes to the files and process the same as per the need of our product. _________________________________________________________________ Mortgage refinance is Hot. *Terms. Get a 5.375%* fix rate. Check savings https://www2.nextag.com/goto.jsp?product=100000035&url=%2fst.jsp&tm=y&searc… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Info needed for File System Development
by Michael Smith 29 Apr '07

29 Apr '07

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On Apr 26, 2007, at 6:13 PM, Shailesh Parulekar wrote: What are the "needs of your product"? = Mike _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… I need to develop a Filter driver to plug-in into the MAC File System in order to 'filter' the read/writes to the files and process the same as per the need of our product. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Darwin-kernel Digest, Vol 4, Issue 100
by Mustapha "Sodhom" Abiola 27 Apr '07

27 Apr '07

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Dkim-signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:mime-version:in-reply-to:references:content-type:message-id:from:subject:date:to:x-mailer; b=H78Rw2J8UMfZGl2iKiUFcp7Q8/vcwYRrJ65ePzoyAoOwe0mM/rxlrxNlj8hExUObTuEgDCM5zxuiOOIHP0Mvce48vr6NZI/qMvUpCTVvao54u3VNmt/3yA/Tx6LSFzhSFNvVVLQ4jDnnvX/89YLFUpE/35AX+PobMB1Yq4C+osU= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:mime-version:in-reply-to:references:content-type:message-id:from:subject:date:to:x-mailer; b=WzYbvqabXMEupQXZMqmwVQ8gg6S3X+ffLTpU+/3tABv1bHYwD0ZOKXZ6Ss1slAj4AG/LT7QG/9AQSqKYSWNzBI7cnOWn5HPYOvtgKW6mH4wocSryOEZhZ03Gpj/C8WVsupHB/kwI9YEasHjCGiNMTUz9XvPSYvLtQ+noH5xEftM= _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Darwin-kernel Digest, Vol 4, Issue 100
by Mustapha "Sodhom" Abiola 27 Apr '07

27 Apr '07

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Dkim-signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:mime-version:in-reply-to:references:content-type:message-id:from:subject:date:to:x-mailer; b=bvbgPM23v410Ot6FqRwp5++tcRiy736VsPQAgCL8b62QiumBBctiJjhrlsR+FD0Wr3Na5ZG6YVc//FEH2goB/psOnNhJz6TRxoIIke1YF7r3pt/D8HSOo8l0W8o46OP4l1+1TCIw2dKTsjZ3H3HpgYngnNNuhd4gjEbkiu/RieY= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:mime-version:in-reply-to:references:content-type:message-id:from:subject:date:to:x-mailer; b=PvLwPAQdwQghrYWfFVvfe9cKCAPa7j+tBCmLsBlM6fbL8c1EMk+Z3iPI8Qzaa+hR6TH6YRzAqnV8kRmMziSpxd9Bk7YsvHHHweQWfxNieLn7fpctjHzNpuXy2k6ilD7qPktzC0T+tPZeJZiliThFPe+XRGFLOIB7Op0ONdnPpxI= _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Info needed for File System Development
by Dan Shoop 27 Apr '07

27 Apr '07

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Hello, Perhaps you should read Amit Singh's excellent "Mac OS X Internals". You should also note that a "MAC" is not a Mac or a Macintosh. ;) -- -dhan ------------------------------------------------------------------------ Dan Shoop AIM: iWiring Systems & Networks Architect http://www.ustsvs.com/ shoop(a)iwiring.net http://www.iwiring.net/ 1-714-363-1174 "The wise man doesn't give the right answers, he poses the right questions." -- Claude Levi-Strauss ------------------------------------------------------------------------ iWiring provides systems and networks support for Mac OS X, unix, and Open Source application technologies at affordable rates. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… At 8:43 AM +0000 4/26/07, Shailesh Parulekar wrote: I need to develop a Filter driver to plug-in into the MAC File System in order to 'filter' the read/writes to the files and process the same as per the need of our product. I have been doing kernel programming since long but I have worked only on Unix/ Linux. I am completely new to MAC OS. Also I have never worked in the File System Area. Can anyone suggest me links/ documents/ white-papers which explain: 1. kernel programming on MAC (essentially for File System Filter Driver) and 2. File system Internals in detail (BSD File System) This email sent to site_archiver(a)lists.apple.com

1 0

Re: Info needed for File System Development
by Terry Lambert 27 Apr '07

27 Apr '07

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com This is probably the best place to start: <http://developer.apple.com/qa/qa2001/qa1242.html> Or perhaps not: <http://developer.apple.com/technotes/tn2005/tn2127.html#SECANTIVIRUS> -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… On Apr 26, 2007, at 3:33 AM, Quinn wrote: At 8:43 +0000 26/4/07, Shailesh Parulekar wrote: Can anyone suggest me links/ documents/ white-papers which explain: 1. kernel programming on MAC (essentially for File System Filter Driver) and 2. File system Internals in detail (BSD File System) Alas, you're probably not going to be happy with the answer )-: It depends on how invasive he intends to be, and why he wants a "filter driver". This email sent to site_archiver(a)lists.apple.com

1 0

Re: Info needed for File System Development
by Quinn 26 Apr '07

26 Apr '07

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Can anyone suggest me links/ documents/ white-papers which explain: 1. kernel programming on MAC (essentially for File System Filter Driver) and 2. File system Internals in detail (BSD File System) This is probably the best place to start: <http://developer.apple.com/qa/qa2001/qa1242.html> Alas, you're probably not going to be happy with the answer )-: S+E -- Quinn "The Eskimo!" <http://www.apple.com/developer/> Apple Developer Relations, Developer Technical Support, Core OS/Hardware _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… At 8:43 +0000 26/4/07, Shailesh Parulekar wrote: This email sent to site_archiver(a)lists.apple.com

1 0