March 2004 - Darwin-kernel - lists.apple.com

Re: sosend and mbuf, ip header, tcp header
by Matt Jaffa 11 Mar '04

11 Mar '04

Hi Justin, Thanks for the info, I have that book and never no where to look, but I should have looked at what hangs off the socket. I am doing that with my raw filter, just didn't do it with UDP or TCP, my mistake, I think I am rushing through this stuff too fast that I won't really be learning about the whole picture, I agree with you Justin, I want to learn all aspects about sockets and so on, but if I don't take the time to learn all about them, then it is not going to help much. Thanks a bunch, Matt On Mar 11, 2004, at 12:11 AM, Justin Walker wrote: Hi, Matt, On Wednesday, March 10, 2004, at 10:16 PM, Matt Jaffa wrote: I am interested in the send function(sosend), that is the function that i am filtering within my app. I am using this to extract information out of the mbuf: [snip] but when It prints out the information to the syslog, it is all wrong, am i getting the ip header and the tcp header correctly out of the mbuf structure? I mean this only in the nicest way :-} - you *really* need to take time and learn about the environment that you are coding for. Either read the kernel code, or get Stevens's TCP/IP Illustrated Volume 2. Spend time with it. The reason everything appears all wrong is that you are looking for something that is not there: there are no protocol headers in the mbuf chain when you get to look at it. If you look at the code in sosend(), you will see that, stripped to its essentials, the sequence is roughly this: while not done do get the data from the user, if necessary. check if it's ok to send. call the NKEs linked to the socket. call the protocol send code (pru_send) etc. od Given this, you should be able to determine that your NKE will see the mbufs before the protocol code sees them, so all that is in the mbufs is the data that is being passed in, either to TCP, UDP, or raw IP. The information you seem to want is in the control blocks that dangle off the socket. For TCP, this information (source and destination port/address) never changes, once the session is ESTABLISHED. For UDP or raw IP, these can change for each 'send'. Regards, Justin -- Justin C. Walker, Curmudgeon-At-Large * Institute for General Semantics | If you're not confused, | You're not paying attention *-------------------------------------- *-------------------------------* _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: sosend and mbuf, ip header, tcp header
by Justin Walker 11 Mar '04

11 Mar '04

Hi, Matt, On Wednesday, March 10, 2004, at 10:16 PM, Matt Jaffa wrote: I am interested in the send function(sosend), that is the function that i am filtering within my app. I am using this to extract information out of the mbuf: [snip] but when It prints out the information to the syslog, it is all wrong, am i getting the ip header and the tcp header correctly out of the mbuf structure? I mean this only in the nicest way :-} - you *really* need to take time and learn about the environment that you are coding for. Either read the kernel code, or get Stevens's TCP/IP Illustrated Volume 2. Spend time with it. The reason everything appears all wrong is that you are looking for something that is not there: there are no protocol headers in the mbuf chain when you get to look at it. If you look at the code in sosend(), you will see that, stripped to its essentials, the sequence is roughly this: while not done do get the data from the user, if necessary. check if it's ok to send. call the NKEs linked to the socket. call the protocol send code (pru_send) etc. od Given this, you should be able to determine that your NKE will see the mbufs before the protocol code sees them, so all that is in the mbufs is the data that is being passed in, either to TCP, UDP, or raw IP. The information you seem to want is in the control blocks that dangle off the socket. For TCP, this information (source and destination port/address) never changes, once the session is ESTABLISHED. For UDP or raw IP, these can change for each 'send'. Regards, Justin -- Justin C. Walker, Curmudgeon-At-Large * Institute for General Semantics | If you're not confused, | You're not paying attention *--------------------------------------*-------------------------------* _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

sosend and mbuf, ip header, tcp header
by Matt Jaffa 11 Mar '04

11 Mar '04

Hi, I have a NKE which is a socket filter. I am interested in the send function(sosend), that is the function that i am filtering within my app. I am using this to extract information out of the mbuf: int vi_send2(struct socket *so, struct sockaddr **addr, struct uio **uio, struct mbuf **top, struct mbuf **control, int *flags, register struct kextcb *kp) { int funnel_state; funnel_state = thread_funnel_set(network_flock, TRUE); char thiscmd[MAXCOMLEN + 1]; struct proc * testing; testing = current_proc(); printf("This process is communicating: %d\n",testing->p_pid); sprintf(thiscmd, "%s", testing->p_comm); thiscmd[MAXCOMLEN] = '\0'; printf("This is the proccess name: %s\n",thiscmd); struct ip * ipHeader; ipHeader = mtod(*top,struct ip *); u_short port; struct tcphdr * header = NULL; header = (struct tcphdr *)&((int32_t *)ipHeader)[ipHeader->ip_hl]; port = header->th_dport; printf("IP TCP :- TCP attempt by PID(%d) to %s (port %d)\n ",testing->p_pid,inet_ntoa(ipHeader->ip_dst),port); printf("The source port: %d\n", header->th_sport); ............. ........... ...... } but when It prints out the information to the syslog, it is all wrong, am i getting the ip header and the tcp header correctly out of the mbuf structure? Thanks, Matt _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Mac-on-Linux and VM internals
by Samuel Rydh 10 Mar '04

10 Mar '04

On Tue, Mar 09, 2004 at 12:40:57PM -0500, Jim Magee wrote: > > Well, that is not quite true. MOL does take full control of the MMU but > > in a completely OS independent manner(*). New versions of > > OSX won't break this code. In fact, the same code is used both under > > Linux and OSX (so the code is truly kernel independent). > > I'm sorry, but not to be picky, exactly how does your KEXT take control > of the exception vectors (after the system is running one OS) in a way > that is "totally independent" of the OS? MOL relocates the very first instruction of each exception vector (the instruction 0x200, 0x300 and so on) and inserts a branch to the new exception vector. > What if one of the other CPUs > is accessing the exception vectors you are changing at the time? That is safe. The other CPU will either run the old code or take the branch to the new code (which essentially just branches back to old code). Removing the hook is equally safe. After the branch has been overwritten with the old instruction, no CPU can be in the MOL hook once a few CPU cycles has elapsed. > What happens when we load two different MOL-like products' KEXTs at the > same time? Won't they get in each other's way? Well, they would have to patch and depatch in the same order, opposite order which would admittedly be a problem if one didn't coordinate things. > What if we decide, later on, to change the exception handlers we > have installed (maybe to more optimized versions for the platform)? You mean, on the fly after the system has booted? Sound like a worse hack than what MOL does without the (good) excuse that MOL can't easily make modifications to the kernel proper... > Won't we overwrite yours? Sure... but how realistic is that scenario? > All these issues require your solution to have knowledge of how the > "base" OS works - and usually in ways that the base OS isn't willing to > be exposed I'm having difficulties seeing a "real world" case here... > (never mind the potential need for your KEXT to be > re-released for every new hardware platform that ships - as not all > these mechanisms are supported in a binary-compatible way between > hardware releases either). Well, "every new hardware platform" is a gross ex-aggregation :-). Sure, major new CPU features like AltiVec or the G5 needs to be explicitly supported. > I know you probably know all that already. But I just wanted to keep > you a little more honest. ;-) Well, there is one dependency which I did forget; MOL needs to be able to allocate a few bytes of RAM within the range of an absolute branch (it is impossible to safely patch the exception vectors otherwise). > > Actually, if the three points above were provided by the kernel then > > the MOL VM engine would be guaranteed to function. I bet that this > > is a lot less invasive than the current support for BlueBox :-). > > Probably not under my definition of invasive. > > If MOL goes out to lunch, what does the machine look like? Is there > any way to recover it? Well, if there is a bug in the kernel module then it can obviously bring down the system. However, the VM part of MOL is designed to be able to report errors and continue gracefully even if a "catastrophic" event like an unexpected exception should occur. In fact, MOL was able to gracefully recover from the double exception hardware problem (certain 750 models can take a double exception if both the TAU and the DEC exceptions are enabled simultaneously) and print a meaningful error message. I actually tracked down this hardware bug quite some time before it got documented in a chip errata because I got the "MOL Internal Error..." message... > How much of it has to be in the kernel address space? The CPU and the MMU virtualization. Part of the CPU virtualization (i.e. the emulation of certain supervisor instructions) could go into userspace but the performance impact due to the double context switch makes a bit expensive. > Does the user know it is MOL that went out to lunch (when it > does go out to lunch), or does it appear that the whole > system/Darwin/Mac OS X kernel have gone out to lunch? Rhetorical question :-). Badly written kernel code will always cause stability problems. > How compact is the code that actually intervenes with the rest of the system? Well, MOL is split into two parts; the kernel module part and a userspace process. The userspace code handles hardware emulation (e.g. emulates the PCI bus), supports client side drivers, handles I/O and so forth. It constitutes the bulk of the code and it runs as an ordinary user process without any magic attributes. The userspace process talks with the kernel module through an ioctl interface (mostly to initialize and configure the virtual machine). It also triggers the switch to virtualization mode (but the switch itself is handled by the kernel module). The kernel module it essentially a black box which does not talk much with the rest of the world. It consists of two parts, the low-level (assembly) code which handles exceptions and (essentially) runs with the MMU off and the high-level C code which maintains the MMU virtualization. It is only the C-code that invokes external code. The only kernel services used are atomic operations, memory allocations and spinlocks. Well, MOL also has to be able to obtain the physical address of wired down memory (the assembly code runs with the MMU off...). > Or more importantly, how entangled is it with the rest of the MOL code? It is clearly separated. > Your original statement about needing to wire down all of MOL to keep the > system running seems somewhat telling in this regard. No no... not the all of MOL. Just the virtualized RAM. The RAM has to be wired down (currently) because the MOL module has no way of knowing that a particular page is about to be swapped out (or whatever). If the kernel module could be told (through some suitable mechanism) about this event, then it would be trivial to flush any references to that page. That is, MOL is designed to function as a software TLB cache with respect to the operating system when pageable memory is used for RAM. > What if someone suspended a thread running inside the MOL > task (maybe with a debugger)? Would the system keep running? Yes, no problem. > Are there key places where it wouldn't? Nope. > Almost none of these things are issues with Classic/Virtual-PC, etc... > on Mac OS X. Sure, we pay a small performance hit as a result. But I > consider that "less invasive." Well, you are comparing Apples with Oranges :-). Classic does not really need a full blown MMU virtualization since MacOS 9 does not utilize the MMU to any significant degree. As for Virtual-PC... well, the huge performance impact here is the CPU emulation. If the cost of emulating the MMU is of the same magnitude, then it is probably an acceptable overhead. For MOL the situation is very different (assuming a MMU demanding OS like Linux or OSX is run within MOL). The CPU virtualization has close to zero overhead. A costly MMU virtualization would really impact performance. The example below should clarify my argument: Virtual-PC: 1 virtual-pc cycle = 20 cpu emulation cycles + 5 mmu emulation cycles MOL: 1 MOL-cycle = 1 instruction cycle + 5 mmu emulaion cycles. That is, MOL's performance would be 1/5 of what it is today while VPC only would only loose 20% of its speed... No doubt, the real-world numbers are a bit different but this is the core of the issue. /Samuel _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Mac-on-Linux and VM internals
by Jim Magee 09 Mar '04

09 Mar '04

On Mar 9, 2004, at 11:35 AM, Samuel Rydh wrote: > On Tue, Mar 09, 2004 at 10:11:41AM -0500, Jim Magee wrote: >> >> On Mar 7, 2004, at 10:34 AM, Samuel Rydh wrote: >>> In order to maximize speed, MOL works at a very low level. Basically >>> MOL hooks all exception vectors (in an OS independent manner) and >>> takes >>> complete control over the MMU. The main benefit is speed (the MMU >>> virtualization >>> costs almost nothing). >> >> Hmmm. In the first paragraph you state that you are doing probably >> the >> most unsupported thing possible. And you are worried about using a >> handful of not-guaranteed-to-be-supported-long-term SPIs in the >> second? >> Interesting. ;-) > > Well, that is not quite true. MOL does take full control of the MMU but > in a completely OS independent manner(*). New versions of > OSX won't break this code. In fact, the same code is used both under > Linux and OSX (so the code is truly kernel independent). I'm sorry, but not to be picky, exactly how does your KEXT take control of the exception vectors (after the system is running one OS) in a way that is "totally independent" of the OS? What if one of the other CPUs is accessing the exception vectors you are changing at the time? What happens when we load two different MOL-like products' KEXTs at the same time? Won't they get in each other's way? What if we decide, later on, to change the exception handlers we have installed (maybe to more optimized versions for the platform)? Won't we overwrite yours? All these issues require your solution to have knowledge of how the "base" OS works - and usually in ways that the base OS isn't willing to be exposed (never mind the potential need for your KEXT to be re-released for every new hardware platform that ships - as not all these mechanisms are supported in a binary-compatible way between hardware releases either). I know you probably know all that already. But I just wanted to keep you a little more honest. ;-) > (*) The _only_ OS dependencies are the following: > > i) Availability of the hash table lock (to guard against two > simultaneous > tlbsync which might deadlock on SMP systems). > > ii) Which SPRGs are used as scratch register by exception handlers > (Linux uses SPRG0/SPRG1, Darwin uses SPRG3 and partially SPRG2). > > iii) MOL needs a VSID range which is not used by the kernel. > >> Note that we went out of our way NOT to take your/MOL's approach in >> supporting other virtual machines in Darwin/Mac OS X. >> And we did it for the reason I just mentioned: it results in a >> pretty unsupportable >> situation long-term (especially if binary compatibility of the KEXTs >> doing this sort of thing is required). > > Actually, if the three points above were provided by the kernel then > the MOL VM engine would be guaranteed to function. I bet that this > is a lot less invasive than the current support for BlueBox :-). Probably not under my definition of invasive. If MOL goes out to lunch, what does the machine look like? Is there any way to recover it? How much of it has to be in the kernel address space? Does the user know it is MOL that went out to lunch (when it does go out to lunch), or does it appear that the whole system/Darwin/Mac OS X kernel have gone out to lunch? How compact is the code that actually intervenes with the rest of the system? Or more importantly, how entangled is it with the rest of the MOL code? Your original statement about needing to wire down all of MOL to keep the system running seems somewhat telling in this regard. What if someone suspended a thread running inside the MOL task (maybe with a debugger)? Would the system keep running? Are there key places where it wouldn't? Almost none of these things are issues with Classic/Virtual-PC, etc... on Mac OS X. Sure, we pay a small performance hit as a result. But I consider that "less invasive." --Jim [demime 0.98b removed an attachment of type application/pkcs7-signature which had a name of smime.p7s] _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

sysctl() and kvm_getfiles() changed in Panther?
by Martin Akesson 09 Mar '04

09 Mar '04

Hello all! I'm experiencing some errors using kvm_getfiles on Panther. Actually it is simply not working as advertised any more. So I checked the sources and found that kvm_getfiles() used sysctl() with a [CTL_KERN, KERN_FILE] mib to get this data from the kernel. I quickly wrote a small program that would replicate the kvm_getfiles() call and tested it on both Jaguar and Panther, output is below: ------------------------- JAGUAR Call sysctl() ask for size of [CTL_KERN, KERN_FILE] sysctl() returned 0 and gives size = 15772 bytes Allocating data pointer with size 15772 bytes Call sysctl() and ask to have [CTL_KERN, KERN_FILE] data copied to our data pointer sysctl() returned 0 and says 15412 bytes were copied Initial filehead.ls_first = 0x209E30C PANTHER Call sysctl() ask for size of [CTL_KERN, KERN_FILE] sysctl() returned 0 and gives size = 17392 bytes Allocating data pointer with size 17392 bytes Call sysctl() and ask to have [CTL_KERN, KERN_FILE] data copied to our data pointer sysctl() returned 0 and says 4 bytes were copied Initial filehead.ls_first = 0x0 ------------------------- Obviously something has changed as sysctl() now only returns 4 bytes which is nothing more than a null pointer, eg. there are no open files. Is there a new api for this or what should I do to fix this? I really need a list of open files in the system. Thank you, Martin _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Mac-on-Linux and VM internals
by Samuel Rydh 09 Mar '04

09 Mar '04

On Tue, Mar 09, 2004 at 10:11:41AM -0500, Jim Magee wrote: > > On Mar 7, 2004, at 10:34 AM, Samuel Rydh wrote: > >In order to maximize speed, MOL works at a very low level. Basically > >MOL hooks all exception vectors (in an OS independent manner) and takes > >complete control over the MMU. The main benefit is speed (the MMU > >virtualization > >costs almost nothing). > > > >... [snip] ... > >Also... the documentation explicitly states that vm_map_xxx can't be > >used safely > >from a KEXT. Why is that? I believe I don't have to use those > >functions but > >it would be interesting to know exactly why there is a difference > >between > >a KEXT and the kernel proper. > > Hmmm. In the first paragraph you state that you are doing probably the > most unsupported thing possible. And you are worried about using a > handful of not-guaranteed-to-be-supported-long-term SPIs in the second? > Interesting. ;-) Well, that is not quite true. MOL does take full control of the MMU but in a completely OS independent manner(*). New versions of OSX won't break this code. In fact, the same code is used both under Linux and OSX (so the code is truly kernel independent). (*) The _only_ OS dependencies are the following: i) Availability of the hash table lock (to guard against two simultaneous tlbsync which might deadlock on SMP systems). ii) Which SPRGs are used as scratch register by exception handlers (Linux uses SPRG0/SPRG1, Darwin uses SPRG3 and partially SPRG2). iii) MOL needs a VSID range which is not used by the kernel. > Note that we went out of our way NOT to take your/MOL's approach in > supporting other virtual machines in Darwin/Mac OS X. > And we did it for the reason I just mentioned: it results in a > pretty unsupportable > situation long-term (especially if binary compatibility of the KEXTs > doing this sort of thing is required). Actually, if the three points above were provided by the kernel then the MOL VM engine would be guaranteed to function. I bet that this is a lot less invasive than the current support for BlueBox :-). /Samuel _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Mac-on-Linux and VM internals
by Jim Magee 09 Mar '04

09 Mar '04

On Mar 7, 2004, at 10:34 AM, Samuel Rydh wrote: > In order to maximize speed, MOL works at a very low level. Basically > MOL hooks all exception vectors (in an OS independent manner) and takes > complete control over the MMU. The main benefit is speed (the MMU > virtualization > costs almost nothing). > > ... [snip] ... > Also... the documentation explicitly states that vm_map_xxx can't be > used safely > from a KEXT. Why is that? I believe I don't have to use those > functions but > it would be interesting to know exactly why there is a difference > between > a KEXT and the kernel proper. Hmmm. In the first paragraph you state that you are doing probably the most unsupported thing possible. And you are worried about using a handful of not-guaranteed-to-be-supported-long-term SPIs in the second? Interesting. ;-) Note that we went out of our way NOT to take your/MOL's approach in supporting other virtual machines in Darwin/Mac OS X. And we did it for the reason I just mentioned: it results in a pretty unsupportable situation long-term (especially if binary compatibility of the KEXTs doing this sort of thing is required). But in case your last question is serious: The difference is that when we release a new kernel update, we release the whole thing. If we make a change to the vm_map_xxx() implementations that require a change in the call sites, we can make those in the same binary. If there are [binary] KEXTs out there that use the same interface, we don't have that luxury anymore. So, we declare such interfaces "unsafe" (meaning we provide no release-to-release binary compatibility guarantees for any KEXT making such calls). You can still use them if you must. But your KEXT will have to link against a very specific version of the kernel's KEXT versions - and the KEXT system will refuse to load your KEXT when the next version of the base kernel comes out. In the past, we have been lax about "upping" the binary compatibility version number for the base system. That's because we hoped developers only used the interfaces we recommended, rather than all the interfaces available, and we didn't want to refuse to load KEXTs that "played nice." But we are well on the way to defining a set of Kernel Programming Interfaces (KPIs) and their associated linkage KEXTs. These linkage KEXTs greatly restrict the set of available interfaces, but they greatly increase the stability of the compatibility versions. All the KEXTs that "play nice" will link against those KPI KEXTs instead of the kernel as a whole. Once this transition is well underway, we will not be so reluctant to up the compatibility version number for the base system. The result: A system that should be much more stable/reliable for all our users. But KEXTs that have to do "interesting things" may have to be released more often to better match the base system. --Jim [demime 0.98b removed an attachment of type application/pkcs7-signature which had a name of smime.p7s] _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: 2 machine debugging problem...
by Garth Cummings 08 Mar '04

08 Mar '04

Hi Marek, On Mar 4, 2004, at 11:16 PM, Marek Kozubal wrote: Hello! I've been doing 2 machine debugging for a while now. One of the machines I am trying to debug however has become a pain. It won't let me do proper back traces or other features in kgm_macros either. Both machines are running 10.2.8 and both machines have the developer tools installed (dec '03 for jaguar). I've also tried it from another machine or 2 with the same results. For example, using the 10.2.8 kernel debug disk image from Apple: [villain:~] marek% gdb /Volumes/KernelDebugKit/mach_kernel GNU gdb 5.3-20021014 (Apple version gdb-250) (Sat Dec 7 02:14:27 GMT 2002) Copyright 2002 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "powerpc-apple-macos10". (gdb) source /Volumes/KernelDebugKit/kgmacros Loading Kernel GDB Macros package. Type "help kgm" for more info. (gdb) target remote-kdp (gdb) attach 10.0.0.3 Connected. [Switching to process 3] (gdb) showpid 434 /* this is the Finder's PID */ task vm_map ipc_space #acts pid proc command Cannot access memory at address 0x154 (gdb) showalltasks task vm_map ipc_space #acts pid proc command Cannot access memory at address 0x0 Is there some setting, or version mis-match that would cause these Macro's not to work? This does sound like a version mismatch between the Kernel Debug Kit you're using and the kernel running on the target system. The two must be the same version. This is covered in the ReadMe included with each Kernel Debug Kit. Note that there are two Kernel Debug Kits for Mac OS X 10.2.8: one for the G5, and another for everything else. Double-check the build number in About this Mac or System Profiler: the G5 build of 10.2.8 was 6S90. The machine has Panther installed on other partitions, I've done the usual boot-args="debug=0xe" and making the arp entry perminate and all that. That only affects GDB's ability to attach to another system's kernel. It has no effect on the kgmacros. The macros rely on the symbols in mach_kernel. However, this and other issues (like bt not working properly, not being able to inspect variables when bt does seem to work) is making things quite hard to track down. So if anyone has any ideas, it'd be most helpful! Thanks in advance! --- Marek Kozubal marek(a)portents.com HTH, --gc _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Mac-on-Linux and VM internals
by Samuel Rydh 07 Mar '04

07 Mar '04

Hello, I'm currently working on porting Mac-on-Linux to Darwin (MOL is virtual machine capable of running Mac OS 9, Mac OS X and Linux). In order to maximize speed, MOL works at a very low level. Basically MOL hooks all exception vectors (in an OS independent manner) and takes complete control over the MMU. The main benefit is speed (the MMU virtualization costs almost nothing). The low-level parts of MOL is currently implemented as a KEXT/IOKIT driver (I will probably turn it into a pure KEXT since the IOKIT abstractions are of little use in this case). Currently, I have MOL working if I wire down all the RAM of the guest OS (I have successfully booted MacOS 9). However, this is a quite suboptimal solution. I would like to be able to use pageable memory with MOL. Some background about how MOL virtualizes the MMU: - MOL "allocates" a range of VSIDs used exclusively by MOL. - MOL maintains a private PTE hash (under Linux it is possible to share the PTE hash with the kernel since it is only used as a cache). - MOL is solely responsible for inserting and flushing PTEs from the private PTE hash. - MOL cross-references the inserted PTEs in various tables. For instance, MOL must be able to flush an ea range of a particular VSID quickly. Under Linux, pageable memory is handled as follows: - All RAM pages are "owned" by the userspace part of MOL. - When MOL maps in a userspace page, it looks to the kernel as if it is the userspace process that accesses it. MOL looks up the physical address and inserts one (or several) PTEs referencing it. - When the userspace page is swapped out, the kernel flushes the PTE of the userspace mapping. MOL hooks the pte_flush_page function and makes sure all the PTEs inserted by MOL which alias the page are also invalidated. What would be the best way to implement something similar under OSX? I'm thinking about hooking mapping_remove() or hw_rem_map(). Another possibility might be implementing a custom swapper. Also... the documentation explicitly states that vm_map_xxx can't be used safely from a KEXT. Why is that? I believe I don't have to use those functions but it would be interesting to know exactly why there is a difference between a KEXT and the kernel proper. /Samuel _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0