June 2005 - Darwin-kernel - lists.apple.com

Re: about MINCLSIZE
by Andrew Gallatin 09 Jun '05

09 Jun '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com chas williams - CONTRACTOR writes: > is this definition for MINCLSIZE right? > > from sys/mbuf.h: > > /* > * Mbufs are of a single size, MSIZE (machine/param.h), which > * includes overhead. An mbuf may add a single "mbuf cluster" of size > * MCLBYTES (also in machine/param.h), which has no additional overhead > * and is used instead of the internal data area; this is done when > * at least MINCLSIZE of data must be stored. > */ > > #define MLEN (MSIZE - sizeof(struct m_hdr)) /* normal data len */ > #define MHLEN (MLEN - sizeof(struct pkthdr)) /* data len w/pkthdr */ > > #define MINCLSIZE (MHLEN + MLEN) /* smallest amount to put in cluster */ > > based on the usage (and description) of MINCLSIZE i would guess it should > be closer to MHLEN + 1. MHLEN + MLEN is essentially 2 * MSIZE which > will not fit inside a single mbuf? Mbufs may also be chained. You can easily encounter an mbuf chain of multiple mbufs of any type. For MINCLSIZE, I think the rationale is that it is cheaper, and wastes less space to allocate another mbuf and chain them than it does to allocate a cluster. Drew _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

about MINCLSIZE
by chas williams - CONTRACTOR 09 Jun '05

09 Jun '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com is this definition for MINCLSIZE right? from sys/mbuf.h: /* * Mbufs are of a single size, MSIZE (machine/param.h), which * includes overhead. An mbuf may add a single "mbuf cluster" of size * MCLBYTES (also in machine/param.h), which has no additional overhead * and is used instead of the internal data area; this is done when * at least MINCLSIZE of data must be stored. */ #define MLEN (MSIZE - sizeof(struct m_hdr)) /* normal data len */ #define MHLEN (MLEN - sizeof(struct pkthdr)) /* data len w/pkthdr */ #define MINCLSIZE (MHLEN + MLEN) /* smallest amount to put in cluster */ based on the usage (and description) of MINCLSIZE i would guess it should be closer to MHLEN + 1. MHLEN + MLEN is essentially 2 * MSIZE which will not fit inside a single mbuf? _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Problem for read routine in filter scheme driver
by Yogesh P 09 Jun '05

09 Jun '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Hi folks, I have written a filter scheme driver. By using this driver i am able to write on the disk.But when i call the read API of driver it is returning me some garbage characters.When i "dd" on the disk i get the correct buffer that i write on the disk using write routine of driver. The code snippet for the my read routine is as follows: void com_My_driver::read(IOService* provider,UInt64 byteStart, IOMemoryDescriptor* buffer, IOStorageCompletion completion) { IOLog("\n com_CoreSnap_driver --- read function Called ...\n"); int i; IOByteCount readBytes = 0; IOByteCount bufLength = buffer->getLength(); IOLog("\n bufLength = %lu\n", bufLength); char *myBuff1 = new char[bufLength]; IOLog("\n before read function Called ...\n"); getProvider()->read(this, byteStart, buffer, completion); IOLog("\n After read function Called ...\n"); IOLog("\n before readbytes function Called ...\n"); readBytes = buffer->readBytes(0, myBuff1, bufLength); IOLog("\n before readbytes function Called ...\n"); IOLog("\n Calling for loop After read .....\n"); for (i = 0; i < 8192; i++) { IOLog("%c C", *myBuff1); } IOLog("\n For loop overs After read.....\n"); } My for loop IOLog shows the following output: Jun 9 21:44:32 localhost kernel: Jun 9 21:44:32 localhost kernel: com_My_driver --- handleIsOpen function Called Jun 9 21:44:32 localhost kernel: com_My_driver[0x11be680]::handleIsOpen(0x191e200) Jun 9 21:44:32 localhost kernel: Jun 9 21:44:32 localhost kernel: com_My_driver --- handleClose function Called Jun 9 21:44:32 localhost kernel: com_My_driver[0x11be680]::handleClose(0x191e200, 0) Jun 9 21:44:39 localhost kernel: C CCCCCCCCCCCCCCCCCCCCCCCCCCCCCC CCCCCCCCCCCCCCCCC CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC CCCCCCCCCCCCCCCCC CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC CCCCCCCCCCCCCCCCC CCCCCCCCCCCCC Jun 9 21:44:39 localhost kernel: For loop overs After read..... It shows the garbage values.I think it will have to show the buffer which gets write on the disk. Am i missing something in my read? Please suggest. Thanks in advance, Yogesh ~ _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: 64-bit problem with VM statistics
by Jim Magee 08 Jun '05

08 Jun '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Sorry for the delay... I found your problem, and it's an interesting one. vm_size_t mac_pagesize; [ ... ] host_page_size(host_port, &mac_pagesize); ... host_page_size(host_port, &mac_pagesize); the Darwin kernel does this: kern_return_t host_page_size( host_t host, vm_size_t *out_page_size) { if (host == HOST_NULL) return(KERN_INVALID_ARGUMENT); *out_page_size = PAGE_SIZE; return(KERN_SUCCESS); } --Jim _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… On Jun 4, 2005, at 11:27 AM, Tony Scaminaci wrote: On Jun 2, 2005, at 1:43 AM, William Kucharski wrote: The problem lies here in your code and the fact that you have your project set to be compiled in 64-bit mode for G5s: What this means is if __ppc__ is defined, vm_size_t will be an unsigned int (size 4 bytes); if __ppc64__ is defined, vm_size_t will be a unintptr_t (size 4 bytes on PPC32, 8 bytes on PPC64.) The problem is that the kernel is NOT a 64-bit executable, and as such is not compiled with __ppc64__ set, so when you make this call from a 64-bit executable: I think you probably see the issue here; Yep, this is a bug with host_page_size(). Since it is a message to the Mach kernel, it doesn't have the information about what kind of address space the caller has, and therefore can only fill in the legacy size. The usual approach we took in these situations was to add a second RPC for 64-bit clients (and then aliased the name in the build of the 64-bit Libc). But this one slipped through the cracks. Why wouldn't it fill in the lower four bytes instead? I ask this because when I print out mac_pagesize, I do get 4096 as expected so the lower bytes must have the correct value. As someone else explained, the printf directive you used triggers the same "looking at the wrong bytes" issue as the host_page_size results output did. If you look in the debugger (on big-endian PowerPC), you'll see the big number. The question is, what SHOULD happen in this case, and should there be an ADC tech note warning of this issue? According to the Darwin 64-bit conversion guide (<http:// developer.apple.com/documentation/Darwin/Conceptual/64bitPorting/ index.html>), "Only libSystem and the Accelerate framework are available to 64-bit developers. Higher level frameworks are 32-bit only." But is a call to host_page_size() really considered a "higher level framework?" I'd consider host_page_size() to be part of libSystem if anything. If we can't address more than 4 GB of memory, what's the point of having a 64-bit OS and a PPC G5? At the very least, Tiger should be able to get at all of the user's installed memory assuming the user runs a 64-bit app. How can the kernel handle more than 4 GB of address space if it's not 64-bit? The host_page_size() returning the wrong thing in 64-bit land doesn't make it so you "can't address more than 4GB of memory." Yep, it's a bug. How could this slip through? Because Libc, through mach_init (), provides a global "vm_page_size" for this value. It is initialized differently for 64-bit processes (as you guessed it - hard coded for now). The intent was to move to use of a value in the commpage (memory shared between the kernel and user-space containing such configuration values). That value would be used to initialize both the global and the return value for host_page_size(). We didn't quite get there - and you see the results. There's got to be a simple workaround to this problem. I can't be the only person trying to use more than 4GB of memory at one time. Any application that bills itself as 64-bit has to address these issues. So, my suggestion is to use the Libc-provided vm_page_size global instead of calling host_page_size() yourself. Heck it's even more efficient. And you don't have to worry about releasing the host port reference returned from mach_host_self() (which hardly anyone does correctly). This email sent to site_archiver(a)lists.apple.com

1 0

Re: Increasing MAXSYMLINKS from 32
by Terry Lambert 07 Jun '05

07 Jun '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Hi all, Answering relative to Tiger... {SYMLOOP_MAX} Maximum number of symbolic links that can be reliably traversed in the resolution of a pathname in the absence of a loop. Minimum Acceptable Value: {_POSIX_SYMLOOP_MAX} {_POSIX_SYMLOOP_MAX} The number of symbolic links that can be traversed in the resolution of a pathname in the absence of a loop. Value: 8 long symloop_max; symloop_max = sysconf(_SC_SYMLOOP_MAX); Hope that helps. -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… On Jun 4, 2005, at 12:35 AM, Andre Pang wrote: I have a need for changing the value of MAXSYMLINKS in sys/param.h (the maximum depth for traversing symbolic links) from 32 to something higher. POSIX & limits.h apparently defines the maximum level of symbolic links to be 255. So, I have a couple of questions about this: * Is there any harm done in simply editing sys/param.h from 32 to something higher such as 64, 128 or 256? * Given that this maximum-symlink-depth value appears to be a constant defined in an included kernel-related header file, is there anyway to increase this symlink limit to something higher in already-compiled programs? (Perhaps even via a kernel extension.) * If there is no way to increase the symlink limit for already- compiled programs, would it be possible to raise the limit to 64/128/256 in future versions of Darwin and Mac OS X? Making this a sysctl would be even nicer ... (Sorry if this post is to the wrong list; I thought darwin-kernel was probably the best choice. Please feel free to move to the post to another list if you feel it's more appropriate.) Thanks! Actually, POSIX states that the minimum acceptable value is 8, and doesn't place a cap on the maximum value: http://www.opengroup.org/onlinepubs/009695399/basedefs/ limits.h.html The MacOS X default value for this is 32, which is 4 times the POSIX minimum. As long as you recompile both the Libc/LibSystem and the kernel, you should be able to change MAXSYMLINKS. In general, MAXSYMLINKS exists to ensure that you don't spend too much time (e.g. "forever" is too much time) going around in a symbolic link loop. Expansion of link targets teds to be in place, unless there are relative paths involved, in which case, you are also limited by the value of MAXPATHLEN -- don't expect to be able to change MAXPATHLEN safely: you are stuck at 1024. So if you are using relative links and hitting your head on MAXPATHLEN, changing the value of MAXSYMLINKS isn't going to help you any. As Justin noted, there can also be compiled code that would use MAXSYMLINKS directly. Any code that does this is non-POSIX code; the correct way for a POSIX compliant program to obtain this value is: This value is obtained out of Libc, rather than from the kernel, which is why you will need to recompile Libc as well as the kernel, so that this value matches the kernel value. In general, the kernel interface is at the top of Libc/LibSystem. Practically, there is not that much you can do with the value in user space, except make decisions not to create link trees deeper than the allowed value, so for most applications, just changing the kernel value will be enough. If you have an already compiled program that depends on the value for MAXSYMLINKS, rather than using the sysconf() interface as it should, then that program is out of luck (sorry). Future programs that need this value should use sysconf() to obtain it instead, so that you can make the change you want to make here, and they will continue to function correctly. This email sent to site_archiver(a)lists.apple.com

1 0

Diff in X-server and Desktop G5
by Imran Kazi 07 Jun '05

07 Jun '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Thread-index: AcVrYr9OsPeQ/lC4SGKuHKL4rOPMZwAAEq3A Hi All, We have written a filter scheme driver to read and write on the disk. Through our code we are able to perform write on disk on X-Serve machine but our problem is that if we try to perform write using the same code on G5 (desktop machine) then it does not write on the disk. Please see the configurations of our machine and please let us know if this is due to difference in system configuration or firmware. We guess both this machine should be same for writing the driver code form developers point of view please correct me if I am wrong. INFORMATION OF DESKTOP MACHINE: Hardware Overview: Machine Model: Power Mac G5 CPU Type: PowerPC 970 (2.2) Number Of CPUs: 1 CPU Speed: 1.6 GHz L2 Cache (per CPU): 512 KB Memory: 1 GB Bus Speed: 800 MHz Boot ROM Version: 5.01f1 Serial Number: YM335GS8NVQ System Software Overview: System Version: Mac OS X 10.3.3 (7F44) Kernel Version: Darwin 7.3.0 INFORMATION OF XSERVE MACHINE: Hardware Overview: Processor Dual 1.33 GHz PowerPC G4 2 MB L3 cache per processor Memory 512 MB DDR SDRAM Machine Model: X-serve CPU Type: PowerPC G4 (3.3) Number Of CPUs: 2 CPU Speed: 1.33 GHz L2 Cache (per CPU): 256 KB L3 Cache (per CPU): 2 MB Memory: 512 MB Bus Speed: 167 MHz Boot ROM Version: 4.65f3 Serial Number: XB32601SN9C System Software Overview: System Version: Mac OS X 10.3.3 (7F44) Kernel Version: Darwin 7.3.0 Thanks Imran _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Diff in X-server and Desktop G5
by Imran Kazi 07 Jun '05

07 Jun '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Thread-index: AcVrYr9OsPeQ/lC4SGKuHKL4rOPMZw== _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

an issue with KPI implementation
by alexklg 07 Jun '05

07 Jun '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com We developed a small NKE and a sequence of steps that may make this NKE to crash even under Tiger 10.4.1 (it crashes in more stable manner under 10.4). However, even if this NKE doesn't crash (doesn't cause the system to panic), the number of the sockets it is attached to grows uncontrollably and this seems to be a Darwin issue. The problem of sockets concerns with the uncontrollable grow of the number os sockets that were created in response to incoming connections but that were not accept'ed. Problem description =================== The very basic socket-filter NKE crashes on unload (kextunload). If it doesn't crash, the number of sockets the socket-filter is attached to grows uncontrollably. Prerequisites ============= 1. Mac OS 10.4.1 2. test_nke NKE (see the attach) test_nke is a basic socket-filter global NKE that simply counts the number of sockets it is attached to. test_nke outputs this number to the system log (sock_count). 3. isockcnt utility. The utility does the following - creates a listening TCP socket (server-side) in a separate process, The second argument of listen() is some large number (256). The listened port is some arbitrary port_number_1 - from a separate process (client-side) connects to the listening socket 100 times, - at the server side accepts the first connection from the client side and then does sleep(2), but the server-side process is killed in one second by alarm. - closes the connection from the client-side (exit(0)). All these steps are repeated several times for the port_number_1 = port_number_1 + 1 with the interval of 3 seconds. This results in a large number of sockets that the test_nke is attached to but is not detached from during reasonable amount of time (about 2 minutes). Remark. From our point of view, the sockets that the filter is not detached from are the sockets from the accept queue. These sockets are created in response to incoming connections. Apparently, for them the kernel structures are allocated but these sockets are not initialized completely. We believe that this happens because accept() was not called for these sockets and meanwhile the client closed the connection (e.g. was terminated) for some reason. How to reproduce ================ 1. Login with administrative account. 3. Load the test_nke NKE (sudo kextload test_nke.kext). 4. Start the insockcnt utility (sudo ./insockcnt). 5. Wait about 30 seconds. 6. Stop the insockcnt utility by pressing 'Control+C'. 7. The number of sockets the test_nke is attached is dumped to the system log. This number may be considerably more than 0, but it should be equal to 0, because all the sockets the NKE had been attached to were closed. If the dumped socket_count is not equal 0, then the subsequent kextunload may result in a system crash. Remark. Even if the insockcnt utility is started without sudo (becoming super-user), and is terminated in about 2-3 minutes (instead of 30 seconds), the kernel starts lacking the resources and goes into panic. Thanks, alexklg _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com testnke-20050606-03.tgz

1 0

vnode iocount vs usecount
by Chaskiel M Grundman 06 Jun '05

06 Jun '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Is there documentation somewhere describing the difference between an "iocount" reference and a "usecount" reference, and when vnode_get/vnode_put should be used vs vnode_ref/vnode_rele? The situation is somewhat confusing since the names of the functions vnode_get, vnode_ref, vnode_put, and vnode_rele are similar to the names of the classic bsd functions vget, vref, vput, and vrele even though the relationship between put and rele does not appear to be the same. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: 64-bit problem with VM statistics
by Andrew Gallatin 06 Jun '05

06 Jun '05

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com William Kucharski writes: > On Monday, June 06, 2005, at 06:50AM, Andrew Gallatin <gallatin(a)cs.duke.edu> wrote: > > >Now, with all my whining and complaining aside, the IOMMU code in > >tiger has gotten *much* better. With tiger, pinning memory is > >nearly as fast as it is on a DAC machine. However, the address space > >is still limited. There's nothing they can do about that. > > You don't have to constantly pin/unpin memory if your OS is sophisticated enough > to be able to manage the process better. While Tiger can't currently do this, > Solaris for one can. Solaris is actually unable to usefully pin any memory for HPC due to quirks in their sysv based VM system. Solaris is probably the worst OS out there for this. > Though given today's Intel announcement, our discussion is largely moot at this > point, as the entire Mac universe, and its performance characteristics, is about to > change drastically. Agreed. Drew _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0