May 2005 - Darwin-kernel - lists.apple.com

Re: NKE sample code
by Quinn 09 May '05

09 May '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com S+E -- Quinn "The Eskimo!" <http://www.apple.com/developer/> Apple Developer Technical Support * Networking, Communications, Hardware _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… At 11:32 -0500 2/5/05, Steven Palm wrote: And I probably already know the answer, but is this example code available to NON-ADC members? (Well, I'm an ADC Online member, but you know that gets you... :-) Yes. We'll get it posted soon (I think I'm on the hook for reviewing it :-) but until then you can just mail <dts(a)apple.com> to request a pre-release copy. Sorry about the delay but, as you can imagine, we're kinda busy at the moment. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Mac OS X VFS
by Quinn 09 May '05

09 May '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com <http://developer.apple.com/qa/qa2001/qa1242.html> It hasn't yet been updated for Tiger. I will get to that shortly. Indeed. This is still true on Tiger. Share and Enjoy -- Quinn "The Eskimo!" <http://www.apple.com/developer/> Apple Developer Technical Support * Networking, Communications, Hardware _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… First up, the Q&A that describes the current status of VFS development on Mac OS X is DTS Q&A 1242 "Developing for VFS". At 13:59 -0400 30/4/05, Bijoy Thomas wrote: I read that currently the kernel allows only leaf filesystems and not stackable ones. What is the difference between them? A stacking file system is one that's implemented in terms of another. The classic example is an encryption file system, where the underlying data is managed by a real file system and the encryption file system is stack on top of that to do the encryption. This is cool because the base file system doesn't need to know anything about encryption, and the encryption file system doesn't need to know anything about how data is stored on disk. The Mac OS X stacking architecture, such as it is, is derived from 4.4 BSD. To learn more about it, read "The Design and Implementation of the 4.4 BSD Operating System". Stacking isn't supported on Mac OS X, for a variety of reasons. Many of these persist in Tiger. Apple does, however, support a specific flavour of stacking, as demonstrated by the WebDAV file system. WebDAV uses HFS as a cache for the WebDAV data. This sort of stacking is supported and very useful for a certain class of problems. See the WebDAV source for details. This email sent to site_archiver(a)lists.apple.com

1 0

Re: thread_funnel_set?
by Quinn 09 May '05

09 May '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Hmmm, the way I've understood it is that when you set the non- reentrant flag, the VFS creates a mutex on the FS's behalf and then automatically acquires/releases it before any VFS entry point. S+E -- Quinn "The Eskimo!" <http://www.apple.com/developer/> Apple Developer Technical Support * Networking, Communications, Hardware _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… At 23:34 -0500 30/4/05, Brian Bergstrand wrote: I'm surprised that they are still in Tiger, as that is supposed to be one of the big changes in the Tiger kernel: no more funnels. I can only surmise that they are there for some kind of backward compatibility, but since all NKE and FS kext's have to be updated to the kpi's I don't know who that compatibility would be for. There are various parts of the kernel that continue to use the funnel for concurrency control. These are typically parts that aren't performance sensitive. A great example is the console driver. At 9:55 -0500 2/5/05, Brian Bergstrand wrote: That 'mutex' is the kernel funnel. It has to be a funnel rather than a normal mutex so that it's automatically dropped regardless of how you block. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Problems building xnu-792
by Markus Hitter 09 May '05

09 May '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com William, first, thanks for the in-depth description of you findings. Am 09.05.2005 um 05:49 schrieb William Kucharski: On May 8, 2005, at 8:37 AM, Markus Hitter wrote: The offending line is always similar: This specific line translates to this line in osfmk/ppc/hw_vm.s: which SHOULD assemble to: GV_HPAGE_SHIFT is defined in osfmk/ppc/mappings.h as: #define GV_HPAGE_SHIFT (GV_PGIDX_SZ_LG2 - GV_GRPS_PPG_LG2) which according to: Cheers, Markus - - - - - - - - - - - - - - - - - - - Dipl. Ing. Markus Hitter http://www.jump-ing.de/ _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… 1362: rlwinm r12,r11,-1,504 There's a bug remaining in the source somewhere, as the "-1" is an illegal third parameter to rlwinm. rlwinm r12,r11,GV_HPAGE_SHIFT,GV_HPAGE_MASK Yes, all 17 occurences of this error use the same "rlwinm" and the same two macros at the end. Luckily, one can easily look up these lines by the unique comment later in these lines. rlwinm r12,r11,31,23,28 Looking at the kernel coming with the Darwin 8.0.1 binary release, I can confirm this for the first occurence. But 5 parameters instead of 4? I tried to look up the mapping in Google as I'm currently resisting to take an in-depth course in PPC assembly ... #define GV_PGIDX_SZ_LG2 3 /* 64-bit Hash-table-page physical-addrress index entry size */ #define GV_SLOT_SZ_LG2 5 /* 32 bytes per slot (mapping size) */ #define GV_PGIDX_SZ_LG2 3 /* 64-bit Hash-table-page physical-addrress index entry size */ #define GV_PAGE_SZ_LG2 12 /* 4k-byte hash-table-page size */ #define GV_GRPS_PPG_LG2 (GV_PAGE_SZ_LG2 - (GV_SLOT_SZ_LG2 + GV_SLOTS_LG2)) calculates out as (3 - (12 - 5 + 3)))), or -1. Oooh. I just saw the 10.4 kernel debug kit is already released. I'll take a look there, next. This email sent to site_archiver(a)lists.apple.com

1 0

Implementing new protocols (was Re: Question on KPI / NKE : sys/select.h : struct selinfo ?)
by Pekka Nikander 09 May '05

09 May '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com There are no plans at this time for net_add_domain and net_add_proto. We are interested in hearing about what you might need such functionality for. lets say that i needed to provide a new protocol family for something like ATM? Besides, what if you don't want to create a new domain but, e.g., just a new version of an existing protocol? For example, I've been trying to extend PF_KEY to support BEET (draft-nikander-esp-beet-mode-01.txt). Then you are likely to just use the same protocol etc. numbers. There are new protocols above IP. SCTP was already mentioned. HIP (Host Identity Protocol) is another example. Integrating protocols such as those into the current kernel (as kexts) is quite painful. If you are interested in hearing about the exact issues, I'd be more than happy to help. --Pekka Nikander _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… We are aware that the socket layer provides a nice framework for interaction between a protocol implemented in the kernel and an application in user space. .... We did look at the problem a number of times and ran in to some fundamental issues with third parties extending the stack to support additional protocols. There are a number of problems with extending the stack to support a new protocol family. One problem is that there are only 255 possible address families in a sockaddr. These leaves us with 255 possible protocol families that have a corresponding address family. If you pick one of the approximately 220 unused protocol families, there is no guarantee that it won't be used by someone else. One possibility would be to assign the PF/AF number dynamically via an API. Maybe there could be a KPI similar to net_add_domain but that would dynamically allocate the AF_ number? The kext could then communicate this to a library that could then use it internally? The apps would use the library and not be exposed to the fact that the AF_ number might be different? In addition to the limited number of useful protocol families, there's an issue of definitions for the various socket options, address structure, and protocols. If two third parties attempt to implement the same protocol, they may actually layout their data structures differently. This means that a developer attempting to use a third party protocol will have to compile their code for a specific implementation of another third party's protocol (such as ATM). If the two independent implementations are assigned different AF_ numbers, that should not really be a problem. There is also the issue of third party drivers and third party protocol families. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Communication between kext
by Stephane Sudre 09 May '05

09 May '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com You need to declare a dependency from one kernel extension to the other. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: sflt_register in 10.4
by Stephane Sudre 09 May '05

09 May '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On 6 mai 05, at 03:43, Peter Lovell wrote: On May 5, 2005, at 7:55 PM, Dean Reece wrote: Eric, Cheers, - Dean The 8.0.0 was working on the latest seeds (425 for instance). _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… You'll need to bump all these up to 8.0.0 (n0 'b1') for Tiger GM. The 8.0.0b* values were only valid for Tiger seeds, so they won't work on Tiger GM. On a related note - this is why it would be great to have GM versions available from the seed site. I've asked for this before and was turned away, but I'll ask again. It's strange that 8.0.0b1 is working because IIRC it was not working anymore at one point... This email sent to site_archiver(a)lists.apple.com

1 0

Re: Problems building xnu-792
by William Kucharski 09 May '05

09 May '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On May 8, 2005, at 8:37 AM, Markus Hitter wrote: The offending line is always similar: This specific line translates to this line in osfmk/ppc/hw_vm.s: rlwinm r12,r11,GV_HPAGE_SHIFT,GV_HPAGE_MASK which SHOULD assemble to: rlwinm r12,r11,31,23,28 according to the 10.4 mach_kernel. The "504" is correct, but the "-1" obviously is not. GV_HPAGE_SHIFT is defined in osfmk/ppc/mappings.h as: #define GV_HPAGE_SHIFT (GV_PGIDX_SZ_LG2 - GV_GRPS_PPG_LG2) which according to: calculates out as (3 - (12 - 5 + 3)))), or -1. Oooh. William Kucharski kucharsk(a)mac.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… 1362: rlwinm r12,r11,-1,504 There's a bug remaining in the source somewhere, as the "-1" is an illegal third parameter to rlwinm. #define GV_PGIDX_SZ_LG2 3 /* 64-bit Hash-table-page physical-addrress index entry size */ #define GV_SLOT_SZ_LG2 5 /* 32 bytes per slot (mapping size) */ #define GV_PGIDX_SZ_LG2 3 /* 64-bit Hash-table-page physical-addrress index entry size */ #define GV_PAGE_SZ_LG2 12 /* 4k-byte hash-table-page size */ #define GV_GRPS_PPG_LG2 (GV_PAGE_SZ_LG2 - (GV_SLOT_SZ_LG2 + GV_SLOTS_LG2)) I dug through the source and found the only place GV_PGIDX_SZ_LG2 is used directly, and it IS supposed to be 3, so there's obviously something wonky somewhere here. :-) This email sent to site_archiver(a)lists.apple.com

1 0

Re: Problems building xnu-792
by Markus Hitter 08 May '05

08 May '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Hello all, Am 07.05.2005 um 07:25 schrieb Jake Luck: This one compiled here, but with warnings: However, I've found another showstopper: The offending line is always similar: 1362: rlwinm r12,r11,-1,504 [...] 4375: rlwinm r4,r3,-1,504 Any clues? Thanks, Markus - - - - - - - - - - - - - - - - - - - Dipl. Ing. Markus Hitter http://www.jump-ing.de/ _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… to join the party, I'm trying to compile xnu-792 from Mac OS X 10.3.9 with Xcode 1.5 installed. The default compiler is gcc 3.3 (gcc version 3.3 20030304 (Apple Computer, Inc. build 1671)). http://10k.org/jake/xnu/xnu-compiletools-20050506.tar.bz2 I'm using the set of compile tools (relpath, seg_hack, libkld.a) I'm using for ages now. Appears to work so far. make[6]: *** [IOWorkLoop.cpo] Error 1 /Network/Users/mah/Development/xnu/iokit/Kernel/IOWorkLoop.cpp: In member function `virtual bool IOWorkLoop::init()': /Network/Users/mah/Development/xnu/iokit/Kernel/IOWorkLoop.cpp:73: warning: converting from `IOReturn (IOWorkLoop::*)(void*, void*, void*, void*)' to `IOReturn (*)(OSObject*, void*, void*, void*, void*)' /Network/Users/mah/Development/xnu/iokit/Kernel/IOWorkLoop.cpp:87: warning: invalid conversion from `void (*)(void*, int)' to `void (*)(void*)' /usr/bin/cc -E -MD -static -D__ASSEMBLER__ -force_cpusubtype_ALL -arch ppc -Dppc -DPPC -D__PPC__ -DPAGE_SIZE_FIXED -DAPPLE -DNeXT -DKERNEL -DKERNEL_PRIVATE -DXNU_KERNEL_PRIVATE -DPRIVATE -D__MACHO__=1 -Dvolatile=__volatile -DMACH -DMACH_KERNEL -DMACH_LOAD -DMACH_BSD -DIOKIT -DMACH_PE -DPOWERMAC -DASSEMBLER -I. -I/Network/Users/mah/Development/xnu/osfmk -I/Network/Users/mah/Development/xnu/BUILD/obj/EXPORT_HDRS/osfmk -I/Network/Users/mah/Development/xnu/BUILD/obj/EXPORT_HDRS/bsd -I/Network/Users/mah/Development/xnu/BUILD/obj/EXPORT_HDRS/iokit -I/Network/Users/mah/Development/xnu/BUILD/obj/EXPORT_HDRS/pexpert -I/Network/Users/mah/Development/xnu/BUILD/obj/EXPORT_HDRS/libkern -I/Network/Users/mah/Development/xnu/BUILD/obj/EXPORT_HDRS/libsa -I/Network/Users/mah/Development/xnu/BUILD/obj/EXTERN_HDRS -I/Network/Users/mah/Development/xnu/EXTERNAL_HEADERS -I/Network/Users/mah/Development/xnu/EXTERNAL_HEADERS/bsd -I/Network/Users/mah/Development/xnu/osfmk/libsa /Network/Users/mah/Development/xnu/osfmk/ppc/hw_vm.s > hw_vm.pp; sed '/^#/d' hw_vm.pp > hw_vm.s; /usr/bin/cc -static -D__ASSEMBLER__ -force_cpusubtype_ALL -arch ppc -Dppc -DPPC -D__PPC__ -DPAGE_SIZE_FIXED -DAPPLE -DNeXT -DKERNEL -DKERNEL_PRIVATE -DXNU_KERNEL_PRIVATE -DPRIVATE -D__MACHO__=1 -Dvolatile=__volatile -DMACH -DMACH_KERNEL -DMACH_LOAD -DMACH_BSD -DIOKIT -DMACH_PE -DPOWERMAC -c hw_vm.s; /bin/rm -f hw_vm.pp hw_vm.s hw_vm.s:1362:Parameter error: expression out of range (parameter 3) hw_vm.s:4375:Parameter error: expression out of range (parameter 3) [repeated about 25 times] This email sent to site_archiver(a)lists.apple.com

1 0

Re: Problems building xnu-792
by Jake Luck 08 May '05

08 May '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Got further with the 792 build. It turns out that xnu doesn't like the default gcc 4.0, you need to switch to 3.3 using "gcc_select". It is nearly working with the compiler binary tool posted here yesterday. Good luck. jake /usr/bin/gcc-3.3 -c -static -g -nostdinc -nostdlib -no-cpp-precomp -fno-builtin -finline -fno-keep-inline-functions -msoft-float -fsigned-bitfields -arch ppc -Dppc -DPPC -D__PPC__ -DPAGE_SIZE_FIXED -O2 -mcpu=750 -mmultiple -fschedule-insns -DAPPLE -DNeXT -DKERNEL -DKERNEL_PRIVATE -DXNU_KERNEL_PRIVATE -DPRIVATE -D__MACHO__=1 -Dvolatile=__volatile -I. -I/Src/xnu-792/. -I/Src/xnu-792/BUILD/obj/EXPORT_HDRS/. -I/Src/xnu-792/BUILD/obj/EXPORT_HDRS/osfmk -I/Src/xnu-792/BUILD/obj/EXPORT_HDRS/bsd -I/Src/xnu-792/BUILD/obj/EXPORT_HDRS/iokit -I/Src/xnu-792/BUILD/obj/EXPORT_HDRS/pexpert -I/Src/xnu-792/BUILD/obj/EXPORT_HDRS/libkern -I/Src/xnu-792/BUILD/obj/EXPORT_HDRS/libsa -I/Src/xnu-792/BUILD/obj/EXTERN_HDRS -I/Src/xnu-792/EXTERNAL_HEADERS -I/Src/xnu-792/EXTERNAL_HEADERS/bsd /Src/xnu-792/BUILD/obj/RELEASE_PPC/version.c -o /Src/xnu-792/BUILD/obj/RELEASE_PPC/version.o /usr/bin/g++-3.3 -nostdlib -static -force_cpusubtype_ALL -Wl,-e,__start -Wl,-segalign,0x1000 -Wl,-sectalign,__TEXT,__text,0x1000 -Wl,-sectalign,__DATA,__common,0x1000 -Wl,-sectalign,__DATA,__bss,0x1000 -Wl,-sectcreate,__PRELINK,__text,/dev/null -Wl,-sectcreate,__PRELINK,__symtab,/dev/null -Wl,-sectcreate,__PRELINK,__info,/dev/null -arch ppc -Wl,-segaddr,__VECTORS,0x0 -Wl,-segaddr,__HIB,0x7000 -Wl,-segaddr,__TEXT,0xe000 /Src/xnu-792/BUILD/obj/RELEASE_PPC/./osfmk/RELEASE/osfmk.o /Src/xnu-792/BUILD/obj/RELEASE_PPC/./bsd/RELEASE/bsd.o /Src/xnu-792/BUILD/obj/RELEASE_PPC/./iokit/RELEASE/iokit.o /Src/xnu-792/BUILD/obj/RELEASE_PPC/./pexpert/RELEASE/pexpert.o /Src/xnu-792/BUILD/obj/RELEASE_PPC/./libkern/RELEASE/libkern.o /Src/xnu-792/BUILD/obj/RELEASE_PPC/./libsa/RELEASE/libsa.o /Src/xnu-792/BUILD/obj/RELEASE_PPC/version.o -o /Src/xnu-792/BUILD/obj/RELEASE_PPC/./mach_kernel.sys -lcc_kext; \ /usr/bin/strip -S -x /Src/xnu-792/BUILD/obj/RELEASE_PPC/./mach_kernel.sys -o /Src/xnu-792/BUILD/obj/RELEASE_PPC/./mach_kernel; ld: Undefined symbols: _OSCompareAndSwap _OSDequeueAtomic _OSEnqueueAtomic make[2]: *** [do_build_mach_kernel] Error 1 make[1]: *** [build_all] Error 2 make: *** [all] Error 2 _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0