December 2008 - Darwin-kernel

Re: Obtaining non-exported symbol from kernel on runtime (without the debug symbols)
by mm w 15 Dec '08

15 Dec '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:in-reply-to:mime-version:content-type:references; bh=siavYvv9/Zyz+9Kz+yTjYfokUsrMC6ge6UXLFNTDYFc=; b=rg2SGiQTgMzfWZXRUGFIoFeeQoUdDhuBaNUTUdY5h4Q9TPJI2gmmRnXy/Lyi3UMj21 7CpDD87F9Q6ikHx4vxYsZAjiksjheWPGhgmjkIsZq5Owp3ePvDr1JiVeQOayyCU1X6qW /r344PDFPUCUBYEaUTaHrnTyKHLNtR9JTV8SM= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:in-reply-to:mime-version :content-type:references; b=UJSAsWCe8m49pa9stiuLl9WhOEuLFMgg77WuAU5834kZwA20WdJZ6Cm+o6mvHElU2b iTM1x+NAi4GWICg/1keR0MQJ7cqR9z/ZZ2fU9lDC30b3fqmgz3QUMrn5LglmjCtanEFm lF86b2vu1eQZcp2lBxV0i64w7TMvUaC0XNZqc= _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Obtaining non-exported symbol from kernel on runtime (without the debug symbols)
by Quinn 15 Dec '08

15 Dec '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com S+E -- Quinn "The Eskimo!" <http://www.apple.com/developer/> Apple Developer Relations, Developer Technical Support, Core OS/Hardware _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… At 10:54 -0500 15/12/08, Amanda Walker wrote: Interesting. I'd be curious about how extensive the changes were to use through the KPIs introduced in Darwin 8. I haven't had to do that conversion, but just looking at kpi_interface.h, it appears as though the changes would be pretty minimal (ifnet_t and its accessors and mutators strike me as more ecapsulation than obfuscation). The folks that I know that did this did it by converting their entire code base to using Mac OS X-style accessor functions and then defining macro versions of those accessors on other platforms. It was a lot of scut work, and a political challenge to get the core code maintainers to accept the change, but it was very simple technically. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Obtaining non-exported symbol from kernel on runtime (without the debug symbols)
by Amanda Walker 15 Dec '08

15 Dec '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Amanda Walker wrote: > No, they have not. Darwin has never used the "FreeBSD kernel API". At least for network drivers, it was pretty close before the mbuf and ifnet obfuscations were added in 10.4. Before these obfuscations, I wrote a device driver for an HPC interconnect which does ethernet emulation (really encapsulation) which shared the ethernet driver transmit and receive code between FreeBSD, MacOSX, and Tru64. --Amanda _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… On Dec 15, 2008, at 10:03 AM, Andrew Gallatin wrote: Agreed--the same can be said for most of the IP stack (and as an aside, the effort than went into making the FreeBSD version SMP-aware shows just how much of a downside manipulating kernel structures directly can be--it took a lot of heavy lifting by a lot of people to get rid of that giant lock). And it's still pretty close from a conceptual standpoint. But I don't think the Darwin 8 changes were made out of spite or laziness :-). Interesting. I'd be curious about how extensive the changes were to use through the KPIs introduced in Darwin 8. I haven't had to do that conversion, but just looking at kpi_interface.h, it appears as though the changes would be pretty minimal (ifnet_t and its accessors and mutators strike me as more ecapsulation than obfuscation). This email sent to site_archiver(a)lists.apple.com

1 0

Re: Obtaining non-exported symbol from kernel on runtime (without the debug symbols)
by Andrew Gallatin 15 Dec '08

15 Dec '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=simple/simple; d=cs.duke.edu; s=mail; t=1229353399; bh=KbypKlq8DPdg2VVKcpr9Ye5BRElkvDtzKIlUwGLgZIw=; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:References: In-Reply-To:Content-Type:Content-Transfer-Encoding; b=gN8AROA6PpHH Ykl030+0pDmeitOsLF3Gi3nAoiq1nsKAFbbjmUBjf1111g3e1WcNWLmtTxbm2Azn4uB oA+t/Os7DG8HbzB+te2oyCpxYsGoEt5r61/S3RtNZPYUh57GV63E3rQqBId8a7Nl2yH 8yG62cdbH/mUTB61ssJ0uI7YY= User-agent: Thunderbird 2.0.0.16 (X11/20080724) Amanda Walker wrote: > No, they have not. Darwin has never used the "FreeBSD kernel API". At least for network drivers, it was pretty close before the mbuf and ifnet obfuscations were added in 10.4. Before these obfuscations, I wrote a device driver for an HPC interconnect which does ethernet emulation (really encapsulation) which shared the ethernet driver transmit and receive code between FreeBSD, MacOSX, and Tru64. Drew _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Obtaining non-exported symbol from kernel on runtime (without the debug symbols)
by Amanda Walker 15 Dec '08

15 Dec '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Stop. You keep repeating this, and it is not true. Honestly, why don't you start bringing up some technical statements here? I care about why kauth is severely limited, feature lacking and almost useless in XNU. I care about why most process, BSD thread or locking API has been crippled in XNU. Because XNU is not FreeBSD or NetBSD, despite sharing some DNA. If you plan on breaking ABI on every minor update, you might as well stop releasing source. But then stop marketing XNU or Mac OS X as an open source or 'openly available and developer friendly' platform. If your developers can't write drivers because you've intentionally crippled your ABI/API, your platform does not qualify as open or developer friendly. In the end only Apple is able to provide closed source or binary-only drivers, So yeah, let's all drink the kool-aid and spice it up with some LSD and candy. No wonder some run out of arguments when they start repeating themselves over and over like a broken record. --Amanda _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… On Dec 15, 2008, at 3:26 AM, John D. wrote: You've already used the whole FreeBSD kernel code base, Everyone who has responded to you has been giving you technical answers. You just aren't listening to them, since you don't like those answers, and keep raising political objections to them. It's not. Quite a number of successful products use it quite nicely, thank you. No. How about "If you plan on breaking compatibility with FreeBSD or NetBSD, you might as well call your OS something else." Which is, in fact, exactly what happened. *Years* ago. "Open source and developer friendly" has *nothing* to do with maintaining compatibility with the internals of other operating systems. Sure it does. It just means you'll have to find a different way to write your driver than you expected to. Having developed both FreeBSD and IOKit drivers, I can say that IOKit is much more "developer friendly" in most ways than the FreeBSD or NetBSD kernel driver infrastructure. Not so. I've personally shipped several closed source drivers (and made plenty of money from them). And thanks to well defined KPIs and binary compatibility, I've been able to ship single builds that work across multiple major revisions of the kernel. If you think this is what has been happening on this thread, you are wrong. You are throwing a temper tantrum because Darwin does not conform to your expectations from FreeBSD. You aren't the first person to do this, but it's generally not the best way to do engineering. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Obtaining non-exported symbol from kernel on runtime (without the debug symbols)
by Amanda Walker 15 Dec '08

15 Dec '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Mike, no offense meant. But you could tell me something I can't read already off Apple's own marketing panflets. I thought this was a technical list. It is. Stop with the grandstanding and condescension already. All I asked here was information about accessing otherwise crippled interfaces which are available by default to FreeBSD kernel modules. Since the code base is essentially the same, except for some minimal changes and glue code, I would expect some of the interfaces to be available. Anyway, let's take the discussion back to technical grounds, please. I neither have time nor interest on going on the marketing or political mumbo jumbo. If there's a mailing list sponsored by Apple explicitly for that purpose, I will subscribe to it when I need to discuss those matters. --Amanda _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… On Dec 15, 2008, at 12:21 AM, John D. wrote: Darwin is not FreeBSD. It provides a different set of interfaces to kernel modules. That expectation is wrong. The code base is *not* essentially the same, though they do share some code (and more often, share similar code with a common ancestry). Saying "you should not expect Darwin to work like FreeBSD" is not political mumbo jumbo. It is the technical answer to your question. "But it should" is a political objection, which *you* raised. If you want to develop for FreeBSD, please do! But don't come waltzing in to darwin-kernel complaining that Darwin works differently. It's a different operating system, despite some family resemblances. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Obtaining non-exported symbol from kernel on runtime (without the debug symbols)
by Amanda Walker 15 Dec '08

15 Dec '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Correct me if I'm interpreting your words wrongly, buy did you just say that anyone who steps out of the limited KPI interfaces is illegitimate? That sounds naive. It's not their fault if you have crippled the FreeBSD kernel API. I'm not saying you don't have any legit reasons to do so, but it's a fact that you've done it. --Amanda _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… On Dec 15, 2008, at 12:15 AM, John D. wrote: On Mon, Dec 15, 2008 at 5:33 AM, Dean Reece <dean.reece(a)apple.com> wrote: It isn't a hassle for legitimate developers because they loose that status as soon as they step outside our KPI space. Thus far, we have not introduced strong protections to prevent developers getting at things we don't export, but that may change if we see customer problems resulting from such practice. In the sense of "can expect support", yes. I mean, you can hack on anything you want, but KPIs and APIs exist specifically to be a contract with developers about what won't change (at least without warning). Going outside of them means you're at your own risk. No, they have not. Darwin has never used the "FreeBSD kernel API". Darwin shares some code with NetBSD and FreeBSD, but it is in no sense a "crippled" FreeBSD. Believing that it is will probably just cause you ongoing frustration. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Obtaining non-exported symbol from kernel on runtime (without the debug symbols)
by Amanda Walker 15 Dec '08

15 Dec '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Excuse me, but monkey business is taking FreeBSD and Mach, creating a hybrid kernel, and crippling the API in a manner that doesn't let third-party developers exercise the best of both systems. Is that common sense? --Amanda _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… On Dec 15, 2008, at 12:02 AM, John D. wrote: To some of us, yes. Try maintaining binary compatibility for a Linux or FreeBSD kernel module across major kernel versions. At least in Darwin, if you use supported KPIs, it's *possible*. It sounds like you've fallen into a common error--thinking "I did X under Linux or FreeBSD, so I'll just tweak it a little bit for Mac OS X", and then discovering that it's not that simple. This is not a bug, though. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Obtaining non-exported symbol from kernel on runtime (without the debug symbols)
by Philip Aker 15 Dec '08

15 Dec '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On 2008-12-15, at 02:13:50, Mike Zuhl wrote: Honestly, why don't you start bringing up some technical statements here? You keep resorting to either avoiding answering altogether, or replying with meaningless comments about why people should blindly take your statements and superior all-seeing knowledge for granted. Philip Aker echo astwta@lvpc.dslh@nl | tr a-z@. p-za-o.@ Democracy: Two wolves and a sheep voting on lunch. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… On Dec 15, 2008, at 12:26 AM, John D. wrote: Think higher level, John. Another thing to consider is that Apple employees have contractual obligations such that they must observe company policies about revealing portions of Apple's copyright code and future development plans for it. So anyone of them might actually want to throw you a lifeline but could only do so at the risk of their job and a potential career ending lawsuit. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Obtaining non-exported symbol from kernel on runtime (without the debug symbols)
by Mike Zuhl 15 Dec '08

15 Dec '08

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On Dec 15, 2008, at 12:26 AM, John D. wrote: Honestly, why don't you start bringing up some technical statements here? You keep resorting to either avoiding answering altogether, or replying with meaningless comments about why people should blindly take your statements and superior all-seeing knowledge for granted. --Mike Zuhl _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… Think higher level, John. A kernel that is a stable environment for all flavors of KEXTs over many versions is a very valuable thing. In order to have stability and still have room to innovate and improve you must have a defined interface between KEXTs and the rest of the kernel. This API is in effect a promise from the kernel developers to the KEXT developers that certain facilities will always be available. Defining this interface is a balancing act. The interface must be rich enough to create efficient KEXTs, yet be limited enough to allow the kernel developers to make improvements. Think about Snow Leopard: I don't know the details of Grand Central but I do know that converting a BSD kernel to use "scores of cores" requires lots of changes in lots of places. The kernel developers can do this while retaining their "promise" to the KEXT developers because the kernel developers have reserved "design room" on their side of the API. Defining a perfect API requires perfect knowledge of the future, which is not often available. If the designers allow too much to be available in the API, it can make future functions or performance improvement impossible. If too little is a available in the API, then KEXT developers are hampered. After an API is published it is far easier to allow additional functionality in the API than to remove functionality (which costs real money to KEXT developers). Because of this, API designers tend to err on the side of less functionality and provide a mechanism like DTS to allow KEXT developers to make a case for more. So the issue isn't FreeBSD, NetBSD, or Linux. The issue isn't kauth or the syscall table. The issue is a _software engineering_ issue of defining interfaces so both sides of the interface can both be stable and make progress. It is literally a case where "less is more." By making a stable interface everyone really can get more done. John, you are complaining either about the whole concept of interfaces, or how the current OS X interface is defined. If it's the former, you need to learn more about software engineering before complaining. If it's the latter, then file a bug to ask that a function be added to the API. This email sent to site_archiver(a)lists.apple.com

1 0