September 2002 - Darwin-kernel

10.1 and 10.2 compilation confusions
by Stéphane Sudre 13 Sep '02

13 Sep '02

According to the Project Builder release notes, when you need your Kernel extension to work under 10.1, you have to compile it with the gcc 2.95 version. I'm in a more complex situation after various attempts. I'm compiling my kext under 10.1. I'm compiling my userland tool on 10.2 using gcc 3 or 2.95. In these conditions, the userland tool can't communicate with the kext using NKE sockets in 10.2. If I compile the userland tool under 10.1 too, it can communicate with the kext in 10.1 and 10.2. If I compile the kext and the userland too under 10.2 with gcc 3, then it's working on 10.2 but not on 10.1. So am I right to conclude that to maintain 10.1 compatibility, we need to compile under 10.1 both the kext and userland tool? _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Darwin & SMT?
by Bernie Zenis 13 Sep '02

13 Sep '02

On Thursday, September 12, 2002, at 08:57 PM, Niall Dalton wrote: On Wed, 11 Sep 2002, Bernie Zenis wrote: As far as I am aware, there has been no work done on the possibility of removing speculative instruction execution, out of order instruction execution, nor branch prediction in SMT processors. Some/All of those There has been some work comparing an in-order and out-of-order SMT processors: S. Hily, A. Seznec `` Out-Of-Order Execution May Not Be Cost-Effective on Processors Featuring Simultaneous Multithreading '', IRISA Report No 1179, March 1998, short version appears in proceedings of HPCA-5, Orlando, Jan. 1999. The report is available from http://www.irisa.fr/caps/PROJECTS/Architecture/CacheArchitecture_US.html# SECTION04 It comes down to the fact that if you are going for maximum throughput and ready threads are available, then an in-order SMT gives almost the same performance as an out-of-order SMT. Last I heard, the SMT simulator is still Alpha only. Anyone been working on a PowerPC version? SMTSIM is still Alpha only afaik. One other simulator worth checking out is James Burns' (ex-USC, now with Intel) simulator that allows one to explore the tradeoffs between SMT, chip-multiprocessor (CMP) and a mixed SMT/CMP. It is based on SimpleScalar, which now also has a PowerPC frontend. It may be possible, with sufficiently motivated hacking, to come up with some version combining those.. I would assume that IBM's internal simulator - which _may_ be available for academic usage - can simulate SMT PowerPC as this is due for Power 5 I believe. Thanks for the info. I'm not someone who is "in-the-loop." So I usually just check http://www.cs.washington.edu/research/smt/ occasionally for new papers or new links to satisfy my curiosity for "What's new in SMT." However, it seems to me that that site isn't the central location for all things SMT as strongly as it used to be. SMT is just getting too popular; darn! :-). If you have any other good places to look for SMT activity, I'd appreciate knowing them. -Bernie _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Darwin & SMT?
by Niall Dalton 13 Sep '02

13 Sep '02

On Wed, 11 Sep 2002, Bernie Zenis wrote: > As far as I am aware, there has been no work done on the possibility of > removing speculative instruction execution, out of order instruction > execution, nor branch prediction in SMT processors. Some/All of those There has been some work comparing an in-order and out-of-order SMT processors: S. Hily, A. Seznec `` Out-Of-Order Execution May Not Be Cost-Effective on Processors Featuring Simultaneous Multithreading '', IRISA Report No 1179, March 1998, short version appears in proceedings of HPCA-5, Orlando, Jan. 1999. The report is available from http://www.irisa.fr/caps/PROJECTS/Architecture/CacheArchitecture_US.html#SE… It comes down to the fact that if you are going for maximum throughput and ready threads are available, then an in-order SMT gives almost the same performance as an out-of-order SMT. > Last I heard, the SMT simulator is still Alpha only. Anyone been working > on a PowerPC version? SMTSIM is still Alpha only afaik. One other simulator worth checking out is James Burns' (ex-USC, now with Intel) simulator that allows one to explore the tradeoffs between SMT, chip-multiprocessor (CMP) and a mixed SMT/CMP. It is based on SimpleScalar, which now also has a PowerPC frontend. It may be possible, with sufficiently motivated hacking, to come up with some version combining those.. I would assume that IBM's internal simulator - which _may_ be available for academic usage - can simulate SMT PowerPC as this is due for Power 5 I believe. Regards, niall _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: What should we do when headers are missing in 10.2?
by Stéphane Sudre 12 Sep '02

12 Sep '02

On Wednesday, September 11, 2002, at 07:39 PM, Justin C. Walker wrote: On Wednesday, September 11, 2002, at 09:38 AM, Stiphane Sudre wrote: On Wednesday, September 11, 2002, at 06:19 PM, Justin C. Walker wrote: On Wednesday, September 11, 2002, at 03:42 AM, Stiphane Sudre wrote: In 10.2, there are some headers that vanished. Yeah; it's part of Apple's plan to deal with 'changing APIs'. Jim's message describes the "private API" define, but doesn't mention that you really are missing the files. They should be in the Kernel Framework, though, so you can either reference them there (-I...) or copy them over. They are not :-( What does 'lsbom' on your system show? Well, following the recommendation from Allan N. I did a lot of lsbom on various Pkgs and none of them included the MIA files. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: What should we do when headers are missing in 10.2?
by Andrew Gallatin 11 Sep '02

11 Sep '02

Jim Magee writes: > >> The real question is: Does anything depend upon that portion of the > >> address space being locked down while I/O is in progress? It would be > >> difficult to see how they could, because the prepare gets released > >> before their notion of an I/O operation is marked as complete. But > >> it's just something to investigate. > > > > Nothing depends on that part of the address space being locked down. > > I didn't really mean in your specific case. I meant in general, since > this is a general API. I meant in general. I can't see how any application behaviour would change, except to possibly get faster. Right now, a process which frees wired memory is stopped in its tracks until the memory is unwired. Eg, it never has a chance to do any work until after the free() or munmap() completes. With the new behaviour, the application would be able to return from the free/munmap call immediately, but the contents of its address space at the point where control returned to it would be the same as the old way. It would just get to return quicker. The only ramification I can see is if some driver wanted to touch the data that had been transferred. Eg, the driver wires the user memory, initiates the dma, the dma completes, and it fondles the bits that were just dma'ed to/from userspace, then it unwires the memory. Of course, that hypothetical driver should have also mapped the memory into the kernel to do the fondling, or it should be doing a copy{in,out} so it should still be safe... Drew _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: What should we do when headers are missing in 10.2?
by Jim Magee 11 Sep '02

11 Sep '02

On Wednesday, September 11, 2002, at 01:35 PM, Andrew Gallatin wrote: Jim Magee writes: Doing what you ask would require "prepare" to create and hold its own reference to the underlying memory objects rather than "glom-ing" on to the user's reference (leaving the user free to release his own with impunity). This could be done, but it would slow prepare down even more in its current form (and it is in dire need of a performance boost already). In fact, we are in the process of investigating possible enhancements to the implementation. The leading design does involve burrowing all the way down to the object, so we do have the opportunity to think about taking our own references again. This would allow unprepare to be much more efficient, and subsequent prepares in the same object could also be quicker. But is also allows a possible redress of your issue. Excellent! Also, memory locking on MacOSX is currently slower than on any other unix-like platform we support. It would be good to see it sped up. The "speed up" thing is an almost certainty. I think that goes hand-in-hand with the deallocate fix as well. But I can't promise that at this point (I'm not doing the code). But I will continue to push for it - and it sounds promising. The real question is: Does anything depend upon that portion of the address space being locked down while I/O is in progress? It would be difficult to see how they could, because the prepare gets released before their notion of an I/O operation is marked as complete. But it's just something to investigate. Nothing depends on that part of the address space being locked down. I didn't really mean in your specific case. I meant in general, since this is a general API. However, the physical memory (formerly) backing the portion of the apps virtual address space which it has just freed cannot be recycled. Otherwise we risk either corrupting memory if those pages end up belonging to some other task. Ideally, the memory would be removed from the app's address space when it called free(), and the physical pages would held in limbo, only freed back to the system when we unprepared it and some reference count dropped to zero. Of course. PS: I think, last we left it, we also talked about changing the timing between file-descriptor closing and VM cleaning. We intentionally clean up the Mach port space (which IOKit UserClient interfaces typically use) before the VM space so that UserClient memory descriptors would be freed before attempting to free the memory in an exiting process. Closing file descriptors should happen even before that (to allow file-descriptor-based drivers to release their holds on memory). While this may not be necessary, if we do the work above, it is still the cleaner way to go. This like what I was asking for... Has the behaviour changed? I haven't looked into this since I wrote my reaper thread work around, which uses APPLE_API_PRIVATE apis. No it still closes file descriptors AFTER the memory deallocation [attempt]. But I am working on a proposal with a couple of the other kernel engineers as we speak. Radars from the outside world (developers) are always good to help justify these changes. I would appreciate it if you could write a couple up for the two issues discussed here. --Jim _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: What should we do when headers are missing in 10.2?
by Eric Hall 11 Sep '02

11 Sep '02

On Wed, Sep 11, 2002 at 11:36:39AM -0700, Eric Hall wrote: > > % less /Library/Receipts/DevSDK.pkg/Contents/Resources/package_version > Major: 6 > Minor: 429 > % lsbom /Library/Receipts/DevSDK.pkg/Contents/Archive.bom | egrep -i ip_var > % > > Other netinet/ files are present, looks to be matching previous posts. > This is from DevTools from the 6C115 seed. I'm grabbing the July DevTools > download now, I'll see what it has to offer. > [please excuse my self-followup] Same package_version, same lsbom results. Bummer. Its not in the August update either for those who are wondering. So, are the headers only in the 10.2 retail install CD, not in the developer seeds or the developer tools? -eric _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: What should we do when headers are missing in 10.2?
by Eric Hall 11 Sep '02

11 Sep '02

On Wed, Sep 11, 2002 at 10:39:29AM -0700, Justin C. Walker wrote: > On Wednesday, September 11, 2002, at 09:38 AM, Stiphane Sudre wrote: > > >On Wednesday, September 11, 2002, at 06:19 PM, Justin C. Walker wrote: > > > >>On Wednesday, September 11, 2002, at 03:42 AM, Stiphane Sudre wrote: > >> > >>>In 10.2, there are some headers that vanished. > >> > >>Yeah; it's part of Apple's plan to deal with 'changing APIs'. Jim's > >>message describes the "private API" define, but doesn't mention that > >>you really are missing the files. They should be in the Kernel > >>Framework, though, so you can either reference them there (-I...) or > >>copy them over. > > > >They are not :-( > > Yeah; I just saw another message with that info. Not sure what's up, > since Allan's message indicates that they should be there (I assume > DevSDK is not a previous install...). What does 'lsbom' on your system > show? > % less /Library/Receipts/DevSDK.pkg/Contents/Resources/package_version Major: 6 Minor: 429 % lsbom /Library/Receipts/DevSDK.pkg/Contents/Archive.bom | egrep -i ip_var % Other netinet/ files are present, looks to be matching previous posts. This is from DevTools from the 6C115 seed. I'm grabbing the July DevTools download now, I'll see what it has to offer. -eric _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: What should we do when headers are missing in 10.2?
by Allan Nathanson 11 Sep '02

11 Sep '02

Scratch my "lsbom" command/output below. I executed those commands on my 10.1 system and not my 10.2 system. It looks as if the ip_var.h header wasn't included in the Dev packages. - Allan On Wednesday, September 11, 2002, at 12:28 PM, Allan Nathanson wrote: On Wednesday, September 11, 2002, at 12:05 PM, Michael Cashwell wrote: On Wednesday, September 11, 2002, at 11:30 AM, Jim Magee wrote: It should be in /usr/include/netinet/ip_var.h. It should also be in /System/Library/Frameworks/Kernel.framework/Headers/netinet/ ip_var.h. That's because you're building a kernel extension which cannot use the headers in /usr/include. ... Disturbing. I did a completely clean install of the retail 10.2, its dev tools and the August patch to the dev tools and searching all local disks yields no files with "ip_var" in the name. If anyone learns what's up with this please let us know. Thanks. g3> lsbom -s /Library/Receipts/DevSDK.pkg/Contents/Resources/DevSDK.bom | grep ip_var ./System/Library/Frameworks/Kernel.framework/Versions/A/Headers/ netinet/ip_var.h ./usr/include/netinet/ip_var.h _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: What should we do when headers are missing in 10.2?
by Justin C. Walker 11 Sep '02

11 Sep '02

On Wednesday, September 11, 2002, at 08:49 AM, Andrew Gallatin wrote: Jim Magee writes: That's just the way BSD headers do things. To change it would break BSD [in-kernel] compatibility. But having BSD [in-kernel] compatibility also implies something else. As you will see, the definition of struct ipovly [the type for the ti_i field] is marked __APPLE_API_PRIVATE. That's you indicator that you have to be prepared to have your software track the OS on a release-for-release basis. I see your point, but ipovly is not a good example. Unless you're planning to reimpliment IP so that you can't talk to anybody else, I doubt that ipovly will be changing much ;) I think 'ipovly' is an implementation-specific thing, and does not affect wire content. Regards, Justin -- Justin C. Walker, Curmudgeon-At-Large * Institute for General Semantics | If you're not confused, | You're not paying attention *--------------------------------------*-------------------------------* _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0