February 2006 - Darwin-kernel

Re: kmem_alloc replacement in Tiger
by Andrew Gallatin 01 Feb '06

01 Feb '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Terry Lambert writes: > > Even if you *are* talking to a device driver, though, you should use > IOKit routines to do it. In the case of a G5 or other large physical > memory system, physical memory above 4G will either need to be > remapped via moving the DART window in the memory controller, or by > bouncing, if you didn't use an IOKit routine to allocate the memory in > the DART window or below 4G to avoid needing the bounce. Here is my rational for using kmem_alloc(). My driver is cross platform (linux, windows, solaris, freebsd, macosx) and all allocations are done from platform independent code, which calls a platform dependent allocator. I'm using kmem_alloc() (via IOMallocAligned()) inside my macosx allocator. The allocator has no idea how the memory will be used. Practically, all large allocations are subsequently wired for DMA and mapped into user space via platform dependent code (IOKit's memory descriptor routines on macosx). There are some allocations which are much smaller, which do not need to be wired. Your comment about bouncing concerns me, especially because we wire userspace memory for DMA so changes to our driver's memory allocator would not guarantee all memory we need to deal with comes from the lower 4GB. The 32-bit nature of IOKit is frustrating, because our device is DAC capable and our older products have been for as long as I can remember. Please tell me that there is some way to coax a full, 64-bit DMA address out of IOKit on a DARTless system. OS-bypass doesn't make any sense when the DMA involves bounce buffers. It looks like getPhysicalSegment64() will only really return a 64-bit address on a DARTful system (where it is useless for DMA), but perhaps I'm reading the code wrong. Drew _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: kmem_alloc replacement in Tiger
by Nikita Danilov 01 Feb '06

01 Feb '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Jim Magee writes: > > On Feb 1, 2006, at 12:28 AM, Nikita Danilov wrote: [...] > > > > Or anything in a file-system. Or in a networking (when paging across > > NFS)... basically almost anything. > > I'm not sure I agree with the NFS comment either. The real > requirement is that you not deal with pageable memory while on the > default-pager paging path. We rarely allocate swap files out on > NFS. ;-) Do you mean this configuration (swap file over NFS) is not supported at all? [...] > --Jim > Nikita. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Re(5): kernel dump server
by Derek Kumar 01 Feb '06

01 Feb '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Hi Derek, the ARP request routine will allow the control transfer, in the same way that takes place during the dump itself. The "flag_panic_dump_in_progress" flag is already set and the routine kdp_arp_request (line 1277) calls kdp_poll which will cause the transfer (test around line 678) if a packet is received. Hello, kdp_poll() doesn't actually perform a control transfer if a debugger packet arrives in the course of a crashdump - it merely clears some globals via abort_panic_transfer() and returns to the caller, which would be your ARP request routine in this case. It is the caller's responsibility to then cause an exit from kdp_panic_dump(), which will then switch to the debugger, i.e. kdp_connection_wait(), if certain conditions are met. In your implementation, you'd address this by checking for a return of -2 from your arp request routine, and then branching to the panic_dump_exit label. But it's a minor point. An issue with the way this is done is that the original debugger packet is discarded, so it will only be the next debugger connection retry that makes it through. The other testing etc I deliberately left alone as much as possible - my thought way to make the minimal set of changes necessary, rather than do a more extensive change which would be harder to integrate (or even accept). Also, some changes might have ramifications elsewhere and I wanted that risk to be as small as possible. For example, I considered and then rejected the temptation to allow a port-number specification as part of the _panicd_ip address, the reason being that it would require a change to the parser for boot-args (16 character limit, this arg could be 22). So, even though there are some redundancies, they were deliberate in order to make the changes easier to review and evaluate. A port number for the crashdump server could certainly be a separate boot-arg (or the parser could be changed to allow IP:port notation, if that's what you meant); I'd have to check if the server allows you to specify the port it would bind to, but if not, that can be changed. It's probable that we will utilize only a portion of your patch in this instance (since the lack of ARP resolution will be addressed in a future update, along with the issue that can cause the polled mode packet send/receive routines to bind to the wrong interface in some cases), but it's great to see contributions - keep them coming! Something else that I wanted to note was that a feature that was asked for last WWDC, the ability to examine the live, running kernel on the same machine, has been in use internally for quite some time, and should be made available externally in a forthcoming tools release. Also, it's possible to do both crashdumps and kernel debugging over a firewire transport, which can be pretty useful in some cases. I'll check to see if those tools can/have been made available externally. Derek _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: kmem_alloc replacement in Tiger
by Quinn 01 Feb '06

01 Feb '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com But we give you so many other memory allocators to choose from (-: S+E -- Quinn "The Eskimo!" <http://www.apple.com/developer/> Apple Developer Technical Support * Networking, Communications, Hardware Mac OS X 10.4 +++++++++++++ MALLOC(<lvar> space, <type> cast, size_t size, int type, int flags) macro include: <sys/malloc.h> declared: xnu/bsd/sys/malloc.h implemented: xnu/bsd/sys/malloc.h zero size: non-NULL summary: just calls through to _MALLOC FREE(void *addr, int type) macro include: <sys/malloc.h> declared: xnu/bsd/sys/malloc.h implemented: xnu/bsd/sys/malloc.h NULL addr: NOP summary: just calls through to _FREE MALLOC_ZONE(<lvar> space, <type> cast, size_t size, int type, int flags) macro include: <sys/malloc.h> declared: xnu/bsd/sys/malloc.h implemented: xnu/bsd/sys/malloc.h zero size: non-NULL? summary: just calls through to _MALLOC_ZONE FREE_ZONE(void *addr, size_t size, int type) macro include: <sys/malloc.h> declared: xnu/bsd/sys/malloc.h implemented: xnu/bsd/sys/malloc.h NULL addr: NOP? summary: just calls through to _FREE_ZONE void * _MALLOC(size_t size, int type, int flags); function include: <sys/malloc.h> declared: xnu/bsd/sys/malloc.h implemented: xnu/bsd/kern/kern_malloc.c zero size: non-NULL summary: calls kalloc_noblock, if M_NOWAIT calls kalloc, otherwise keeps track of size, so that you don't have to provide it on _FREE void _FREE(void *addr, int type); function include: <sys/malloc.h> declared: xnu/bsd/sys/malloc.h implemented: xnu/bsd/kern/kern_malloc.c NULL addr: NOP summary: calls kfree uses header attached by _MALLOC to get size void * _MALLOC_ZONE(size_t size, int type, int flags); function include: <sys/malloc.h> declared: xnu/bsd/sys/malloc.h implemented: xnu/bsd/kern/kern_malloc.c zero size: non-NULL? summary: maintains zones for common BSD element types, eg M_MBUF kmzones array KMZ_CREATEZONE means create the zone for BSD KMZ_LOOKUPZONE means lookup Mach's zone of equivalent size calls zalloc_noblock, if element matches zone element size, and M_NOWAIT calls zalloc, if element matches zone element size, otherwise like _MALLOC, otherwise void _FREE_ZONE(void *elem, size_t size, int type) function include: <sys/malloc.h> declared: xnu/bsd/sys/malloc.h implemented: xnu/bsd/kern/kern_malloc.c NULL addr: NOP? summary: calls zfree, if size matches zone size calls kfree, otherwise * * * I/O Kit ======= void * IOMalloc(vm_size_t size); function include: <IOKit/IOLib.h> declared: xnu/iokit/IOKit/IOLib.h implemented: xnu/iokit/Kernel/IOLib.c zero size: non-NULL summary: calls kalloc void IOFree(void * address, vm_size_t size); function include: <IOKit/IOLib.h> declared: xnu/iokit/IOKit/IOLib.h implemented: xnu/iokit/Kernel/IOLib.c NULL addr: NOP summary: calls kfree void * IOMallocAligned(vm_size_t size, vm_size_t alignment); function include: <IOKit/IOLib.h> declared: xnu/iokit/IOKit/IOLib.h implemented: xnu/iokit/Kernel/IOLib.c zero size: NULL summary: kalloc, if size (plus tracking info) less than a page kernel_memory_allocate from kernel_map, otherwise void IOFreeAligned(void * address, vm_size_t size); function include: <IOKit/IOLib.h> declared: xnu/iokit/IOKit/IOLib.h implemented: xnu/iokit/Kernel/IOLib.c NULL addr: NOP summary: calls kfree, if size (plus tracking info) less than a page calls kmem_free, otherwise void IOFreeContiguous(void * address, vm_size_t size); function include: <IOKit/IOLib.h> declared: xnu/iokit/IOKit/IOLib.h implemented: xnu/iokit/Kernel/IOLib.c NULL addr: NOP summary: calls kfree, if size (plus tracking info) less than a page calls kmem_free, otherwise * * * libkern, OSMalloc ================= void * OSMalloc(uint32_t size, OSMallocTag tag); function include: <libkern/OSMalloc.h> declared: xnu/libkern/libkern/OSMalloc.h implemented: xnu/osfmk/kern/kalloc.c zero size: ??? (in the pageable case) summary: calls kmem_alloc_pageable, if tag is pageable and size a page or more calls kalloc, otherwise extern void * OSMalloc_nowait(uint32_t size, OSMallocTag tag); function include: <libkern/OSMalloc.h> declared: xnu/libkern/libkern/OSMalloc.h implemented: xnu/osfmk/kern/kalloc.c zero size: non-NULL summary: calls kalloc_noblock extern void * OSMalloc_noblock(uint32_t size, OSMallocTag tag); function include: <libkern/OSMalloc.h> declared: xnu/libkern/libkern/OSMalloc.h implemented: xnu/osfmk/kern/kalloc.c zero size: non-NULL summary: calls kalloc_noblock extern void OSFree(void * addr, uint32_t size, OSMallocTag tag); function include: <libkern/OSMalloc.h> declared: xnu/libkern/libkern/OSMalloc.h implemented: xnu/osfmk/kern/kalloc.c NULL addr: crash summary: calls kmem_free, if tag is pageable and size a page or more calls kfree, otherwise libkern, C++ ============ static void * OSObject::operator new(size_t size); function include: <libkern/c++/OSObject.h> declared: xnu/libkern/libkern/c++/OSObject.h implemented: xnu/libkern/c++/OSObject.cpp zero size: non-NULL summary: calls kalloc static void OSObject::operator delete(void *mem, size_t size); function include: <libkern/c++/OSObject.h> declared: xnu/libkern/libkern/c++/OSObject.h implemented: xnu/libkern/c++/OSObject.cpp NULL addr: crash? (protected by C++ runtime?) summary: calls kfree * * * Mach, kalloc ============ does all large allocations within kalloc_map, 16 MB submap of the kernel map void *kalloc(vm_size_t size); function include: n/a declared: xnu/osfmk/kern/kalloc.h implemented: xnu/osfmk/kern/kalloc.c zero size: non-NULL summary: calls kalloc_canblock with canblock set void *kalloc_noblock(vm_size_t size); function include: n/a declared: xnu/osfmk/kern/kalloc.h implemented: xnu/osfmk/kern/kalloc.c zero size: non-NULL summary: calls kalloc_canblock with canblock clear void *kalloc_canblock(vm_size_t size, boolean_t canblock); function include: n/a declared: none implemented: xnu/osfmk/kern/kalloc.c zero size: non-NULL summary: maintains zones of power of two sizes up to 8 KB k_zone array calls kmem_alloc in kalloc_map, if size large (kalloc_max_prerounded, 8 KB) calls zalloc_canblock(k_zone[zindex], ...), otherwise void *kget(vm_size_t size); function include: n/a declared: xnu/osfmk/kern/kalloc.h implemented: xnu/osfmk/kern/kalloc.c zero size: non-NULL summary: uses same same k_zone array as kalloc calls zget(k_zone[zindex]) void kfree(void *data, vm_size_t size); function include: n/a declared: xnu/osfmk/kern/kalloc.h implemented: xnu/osfmk/kern/kalloc.c NULL addr: crash summary: calls kmem_free to kalloc_map, if size large (kalloc_max_prerounded, 8 KB) calls zfree(k_zone[zindex], ...), otherwise * * * void * zalloc(zone_t zone); function include: n/a declared: xnu/osfmk/kern/zalloc.h implemented: xnu/osfmk/kern/zalloc.c zero size: n/a summary: calls zalloc_canblock with canblock set void * zalloc_noblock(zone_t zone); function include: n/a declared: xnu/osfmk/kern/zalloc.h implemented: xnu/osfmk/kern/zalloc.c zero size: n/a summary: calls zalloc_canblock with canblock clear void * zalloc_canblock(zone_t zone, boolean_t canblock); function include: n/a declared: xnu/osfmk/kern/zalloc.h implemented: xnu/osfmk/kern/zalloc.c zero size: n/a summary: tries to dequeue an element from the zone's free list kmem_alloc_wired(kernel_map, ...), zfill for initial space kernel_memory_allocate(zone_map, ...), expansion? void * zget(zone_t zone); function include: n/a declared: xnu/osfmk/kern/zalloc.h implemented: xnu/osfmk/kern/zalloc.c zero size: n/a summary: just gets a block from a zone's free list will never try to expand the zone void zfree(zone_t zone, void *elem); function include: n/a declared: xnu/osfmk/kern/zalloc.h implemented: xnu/osfmk/kern/zalloc.c NULL addr: crash summary: queues the element on the zone's free list kicks off the garbage collector if appropriate * * * Mach, kmem ========== kern_return_t kmem_alloc(vm_map_t map, vm_offset_t *addrp, vm_size_t size); function include: n/a declared: xnu/osfmk/vm/vm_kern.h implemented: xnu/osfmk/vm/vm_kern.c zero size: error summary: calls kernel_memory_allocate(map, addrp, size, 0, 0) void kmem_free( vm_map_t map, vm_offset_t addr, vm_size_t size) function include: n/a declared: xnu/osfmk/vm/vm_kern.h implemented: xnu/osfmk/vm/vm_kern.c NULL addr: not sure, but it looks like a really bad idea summary: vm_map_remove _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… At 20:06 -0800 30/1/06, Herbert wrote: P.S. Bad Apple, for moving kmem_alloc to a private place, and removing it's symbol!!! I've did a bunch of research into kernel memory allocation over the last couple of years. I took copious notes, which I haven't had time to productise yet. I've pasted my notes into the end of this email. I hope folks find them useful. btw I highly recommend using Graphviz to render the diagram. The resulting graph really illuminates the kernel memory allocation Big Picture (tm). Layering ======== If you paste the following into a Graphviz <http://www.pixelglow.com/graphviz/> document, you'll get a nice graph of the dependencies between each layer. ------------------------------------------------------------------------ digraph "Kernel Memory Allocators" { label = "Kernel Memory Allocators"; labelloc = t; labeljust = c; fontsize = 24; ranksep = 2; // BSD subgraph cluster0 { label = "BSD\n<sys/malloc.h>\nKPI"; fontsize = 14; style = bold; MALLOC; _MALLOC; MALLOC_ZONE; _MALLOC_ZONE; } MALLOC -> _MALLOC; _MALLOC -> kalloc; _MALLOC -> kalloc_noblock [label="M_NOWAIT",fontsize=8]; MALLOC_ZONE -> _MALLOC_ZONE; _MALLOC_ZONE -> zalloc_noblock [label="M_NOWAIT and\nzone size match",fontsize=8]; _MALLOC_ZONE -> zalloc [label="zone size match",fontsize=8]; _MALLOC_ZONE -> kalloc; _MALLOC_ZONE -> kalloc_noblock [label="M_NOWAIT",fontsize=8]; // I/O Kit subgraph cluster1 { label = "I/O Kit\n<IOKit/IOLib.h>\nKPI"; fontsize = 14; style = bold; IOMalloc; IOMallocAligned; IOMallocContiguous; } IOMalloc -> kalloc; IOMallocAligned -> kalloc [label="less than\npage size",fontsize=8]; IOMallocAligned -> kernel_memory_allocate [label="greater than\npage size",fontsize=8]; IOMallocContiguous -> kalloc [label="less than\npage size",fontsize=8]; IOMallocContiguous -> kernel_memory_allocate [label="exactly\npage size",fontsize=8]; IOMallocContiguous -> kmem_alloc_contig [label="greater than\npage size",fontsize=8]; // libkern subgraph cluster2 { label = "libkern\n<libkern/OSMalloc.h>\nKPI"; fontsize = 14; style = bold; OSMalloc; OSMalloc_nowait; OSMalloc_noblock; "OSObject::operator new"; } OSMalloc -> kalloc; OSMalloc -> kmem_alloc_pageable [label="greater than page size\nand OSMT_PAGEABLE",fontsize=8]; OSMalloc_nowait -> kalloc_noblock; OSMalloc_noblock -> kalloc_noblock; "OSObject::operator new" -> kalloc; // Mach, kalloc subgraph cluster4 { label = "Mach, kalloc"; fontsize = 14; kalloc; kalloc_canblock; kalloc_noblock; } kalloc -> kalloc_canblock; kalloc_noblock -> kalloc_canblock; kalloc_canblock -> zalloc_canblock [label="fits in zone;\nthat is, less than 8 KB",fontsize=8]; kalloc_canblock -> kmem_alloc [label="bigger than largest zone;\nin kalloc_map submap (16 MB)",fontsize=8]; // Mach, Zone subgraph cluster5 { label = "Mach, zalloc"; fontsize = 14; zalloc; zalloc_canblock; zalloc_noblock; } zalloc -> zalloc_canblock; zalloc_noblock -> zalloc_canblock; zalloc_canblock -> kmem_alloc_wired; zalloc_canblock -> kernel_memory_allocate; // Mach, kmem subgraph cluster6 { label = "Mach, kmem"; fontsize = 14; kmem_alloc; kmem_alloc_aligned; kmem_alloc_wired; kmem_alloc_contig; kmem_alloc_pageable; kernel_memory_allocate; } kmem_alloc -> kernel_memory_allocate; kmem_alloc_aligned -> kernel_memory_allocate; kmem_alloc_wired -> kernel_memory_allocate; } ------------------------------------------------------------------------ BSD === in the BSD allocators, type is one of the M_XXX constants from <sys/malloc.h> (use M_TEMP for miscellaneous allocations) and flags is a bit field made up of bits M_NOWAIT and M_ZERO (only supported by [_]MALLOC; not supported by [_]MALLOC_ZONE) void * IOMallocContiguous(vm_size_t size, vm_size_t alignment, IOPhysicalAddress * physicalAddress); function include: <IOKit/IOLib.h> declared: xnu/iokit/IOKit/IOLib.h implemented: xnu/iokit/Kernel/IOLib.c zero size: NULL summary: calls kalloc, if size (plus tracking info) less than a page calls kernel_memory_allocate from kernel_map, if size exactly a page calls kmem_alloc_contig from kernel_map, otherwise Mach, zalloc ============ uses kmem_suballoc to create a contiguous area (zone_map) in kernel_map for all zone use; set set by zsize boot arg (in MB), otherwise 1/4 of physical memory lots of complex stuff (garbage collect, pageable zones, etc) that I didn't investigate kern_return_t kmem_alloc_aligned(vm_map_t map, vm_offset_t *addrp, vm_size_t size); function include: n/a declared: xnu/osfmk/vm/vm_kern.h implemented: xnu/osfmk/vm/vm_kern.c zero size: error summary: calls kernel_memory_allocate(map, addrp, size, size - 1, KMA_KOBJECT) kern_return_t kmem_alloc_wired(vm_map_t map, vm_offset_t *addrp, vm_size_t size); function include: n/a declared: xnu/osfmk/vm/vm_kern.h implemented: xnu/osfmk/vm/vm_kern.c zero size: error summary: calls kernel_memory_allocate(map, addrp, size, 0, KMA_KOBJECT) kern_return_t kmem_alloc_contig(vm_map_t map, vm_offset_t *addrp, vm_size_t size, vm_offset_t mask, int flags); function include: n/a declared: xnu/osfmk/vm/vm_kern.h implemented: xnu/osfmk/vm/vm_kern.c zero size: error summary: if KMA_KOBJECT use kernel_object backing object else use new backing object vm_map_find_space to find space in map cpm_allocate to allocate pages vm_page_insert to insert pages into object vm_map_wire release reference to backing object kern_return_t kernel_memory_allocate(vm_map_t map, vm_offset_t *addrp, vm_size_t size, vm_offset_t mask, int flags); function include: n/a declared: xnu/osfmk/vm/vm_kern.h implemented: xnu/osfmk/vm/vm_kern.c zero size: error summary: if KMA_KOBJECT use kernel_object backing object else use new backing object vm_map_find_space to find space in map vm_page_alloc to populate pages vm_map_wire release reference to backing object (sneaky) kern_return_t kmem_kmem_alloc_pageable(vm_map_t map, vm_offset_t *addrp, vm_size_t size); function include: n/a declared: xnu/osfmk/vm/vm_kern.h implemented: xnu/osfmk/vm/vm_kern.c zero size: ??? summary: vm_map_enter This email sent to site_archiver(a)lists.apple.com

1 0

Re: Is kernel aware of pthreads ?
by Andrew Gallatin 01 Feb '06

01 Feb '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Sudarshan S writes: > A multi-threaded user process makes a system call and control comes to kernel extension ( I mean proc_self( ) gives me the pid of the user process). > If the kernel extension blocks using msleep( ), does it block the entire process or does it block only a specific thread of that process that is making a call. > I am trying to understand the threading model implemented in Tiger. It may depend somehow on how the control gets to the kernel extension, but I can confirm that a sleep (IOLockSleepDeadline()) in an IOKit driver reached via a BSD ioctl blocks only the calling thread, and not the entire process. Drew _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: kmem_alloc replacement in Tiger
by Jim Magee 01 Feb '06

01 Feb '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On Jan 31, 2006, at 3:39 PM, Terry Lambert wrote: Since Herb was talking about using the memory for a large array, I recommended the API that I'd normally tell people to use for that usage. If this array was actually a driver DMA target, well, then one of the other options is probably a better bet. Regardless of whether the memory is likely to be a DMA target or not, allocating pageable kernel memory is probably the way to go. The sole exception to this case is when the memory is likely to be accessed on the paging path; typically anything in interrupt context or on the low-level I/O path. Or anything in a file-system. Or in a networking (when paging across NFS)... basically almost anything. On Feb 1, 2006, at 12:28 AM, Nikita Danilov wrote: Mike Smith writes: I'm not sure I agree with the NFS comment either. The real requirement is that you not deal with pageable memory while on the default-pager paging path. We rarely allocate swap files out on NFS. ;-) There used to be strong cross-chatter between paging out to the default pager and paging out dirty NFS (file) pages. Mostly this was because there was only one pageout thread (and if it got stuck in a page frame allocation while paging out NFS it couldn't be used to page out to the swap files). But there are now independent threads for paging internal (swap) and external (vnode) memory objects/files. Not that there aren't other reasons to allocate wired memory (latency requirements, etc...) - but I just wanted to clear this up. --Jim _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Is kernel aware of pthreads ?
by Jim Magee 01 Feb '06

01 Feb '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On Feb 1, 2006, at 6:11 AM, Sudarshan S wrote: --Jim _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… A multi-threaded user process makes a system call and control comes to kernel extension ( I mean proc_self( ) gives me the pid of the user process). If the kernel extension blocks using msleep( ), does it block the entire process or does it block only a specific thread of that process that is making a call. I am trying to understand the threading model implemented in Tiger. Each pthread corresponds to a unique kernel (Mach) thread. So msleep () will only block that one thread. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Re(5): kernel dump server
by Peter Lovell 01 Feb '06

01 Feb '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On Jan 31, 2006, at 12:35 PM, Derek Kumar wrote: Cheers.....Peter _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… Hi Derek, I submitted rdar://4422761 a couple of days ago, and included an updated kdp_udp.c which solves this problem. Main issue was that there was no ARP request code there at all, and now there is. This also ARPs for the router_ip if that is defined in boot-args (this capability was mentioned in the code as a future enhancement). Thanks for filing a radar, I'll take a look. (It's likely that equivalent changes are awaiting release in this particular case, but contributions to the kernel are always welcome). Since it was in the same file, I also updated the code for creating the dump file name so that it now includes the xnu minor version. Notes in the code suggest that rdar://3735061 requested this capability. Great - as I recall, it was intended that we fix this by changing an internal submission script to stamp the kernel with the version number, rather than parsing the existing version string (which is created from OBJROOT (by config/newvers.pl), which includes the version number in an internal build environment). However, having the minor version numbers in crashdump filenames is definitely convenient. N.B.: You can also use the "showversion" kgmacro, which examines the magic address 0x501C - that holds a pointer to the version string (or "strings -" on the crashdump and grep that for the version pattern as a crude work-around if you don't have gdb handy). You can use that macro on live connections as well, to determine the version of the kernel you're connected to so you can find the right symbols. (An "implementation detail" - that macro is fine on PPC and live x86 connections, however it doesn't yet work on x86 crashdumps - the currently released x86 kernel doesn't have crashdump support, so it's not immediately relevant, but it will be eventually :-) Derek P.S.: From a quick look, if you're using this internally, your ARP request routine would have to change to allow control transfer to the debugger if a debugger packet arrives; also, the code to check for the crashdump protocol packets is probably unnecessary in your arp request routine. And apropos the comment about size checking, kdp_vm_read() is a debugger routine that checks for a valid translation for both source and destination pages in the currently active debugger pmap (virtual address space), and then does a physical mode copy, so it's a "safe read". We can get away with assuming the translation remains valid since the debugger runs with interrupts off on the active cpu, and the rest of the processors are signaled to spin on the debugger lock on entry. Hi Derek, the ARP request routine will allow the control transfer, in the same way that takes place during the dump itself. The "flag_panic_dump_in_progress" flag is already set and the routine kdp_arp_request (line 1277) calls kdp_poll which will cause the transfer (test around line 678) if a packet is received. The other testing etc I deliberately left alone as much as possible - my thought way to make the minimal set of changes necessary, rather than do a more extensive change which would be harder to integrate (or even accept). Also, some changes might have ramifications elsewhere and I wanted that risk to be as small as possible. For example, I considered and then rejected the temptation to allow a port-number specification as part of the _panicd_ip address, the reason being that it would require a change to the parser for boot- args (16 character limit, this arg could be 22). So, even though there are some redundancies, they were deliberate in order to make the changes easier to review and evaluate. This email sent to site_archiver(a)lists.apple.com

1 0

Is kernel aware of pthreads ?
by Sudarshan S 01 Feb '06

01 Feb '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Subject:To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=VtuewhynJC57Xl54LpXTkB7vcBAdJiQHnaWj2HhYr2+3NyF7+X4PfQJvWJIEECxPM+38hGSbFS5VN1AGeuwAocLShQDdcSPficPyIR91h4j6O7wiMdtILF1OMvagY9P3s7DPFZBZ56UrSQaoEsqPR50mEVIvLcCQsBCHqz1hZxI= ; _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: kmem_alloc replacement in Tiger
by Mike Smith 01 Feb '06

01 Feb '06

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On Jan 31, 2006, at 9:28 PM, Nikita Danilov wrote: On Jan 31, 2006, at 3:39 PM, Terry Lambert wrote: Since Herb was talking about using the memory for a large array, I recommended the API that I'd normally tell people to use for that usage. If this array was actually a driver DMA target, well, then one of the other options is probably a better bet. Regardless of whether the memory is likely to be a DMA target or not, allocating pageable kernel memory is probably the way to go. The sole exception to this case is when the memory is likely to be accessed on the paging path; typically anything in interrupt context or on the low-level I/O path. Or anything in a file-system. Or in a networking (when paging across NFS)... basically almost anything. Both of those are candidates, but I disagree with the "almost anything" assertion. And if you are on either of those paths, you should not be playing with large allocations in the first place, as handling that much data sounds suspiciously like being slow. 8) = Mike _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… Mike Smith writes: This email sent to site_archiver(a)lists.apple.com

1 0