On 30.10.2007, at 11:29, Quinn wrote: At 21:00 -0600 29/10/07, William Kucharski wrote:
It's not divulging trade secrets to give general areas which are/remain closed source.
OK, I'll bite (-:
I know of three main areas:
o A significant chunk of what gets removed is comments. They might, for example:
- contain information related to upcoming hardware
#if UNANNOUNCED_APPLE_HARDWARE_SUPPORT . . #endif
a simple preprocessor run will remove it...
- explain workarounds for bugs in third party code, where Apple
doesn't want to be seen as publicly criticising a third party
- explain workarounds for Apple code, where Apple doesn't want
to be seen as being critical of itself
#if NASTY_COMMENT . . #endif
o Some stuff is code that we received under non-disclosure from another party, or code based on information that we received under non-disclosure from another party.
#if SECRETS_UNDER_NDA . . #endif
o Some stuff is removed because it relates to upcoming Apple products. For an extreme example, consider that, prior to the iPhone announcement, it would have been extremely foolish to reveal full support for the ARM platform.
Well that one didn't work in the past. I remember having seen i386 folders and i386 stuff when PPC was the only one available. And this in the standard SDK, not even the kernel sources ;-)
Having said that, Apple works hard to ensure that:
a) Darwin can build and run as a standalone OS.
b) You can slide a "mach_kernel" built from open source underneath Mac OS X, and things still work.
..still work or still work exactly the same way? So, while the Darwin kernel source is slightly different from Apple's internal source, you'd be hard pressed to find a situation where that makes any difference in practice. If you discover such a situation, please let us know by filing a bug.
how could we know if we can't compare? (* pulling out disassembler to see whats different now... *)
With regards SCTP, we already have plenty of bugs covering this. The two lead bugs are:
<rdar://problem/3893228> Protocol plug-in KPI
<rdar://problem/4605154> Requirement for SCTP implementation on Kernel
The first covers the requirement for a general protocol plug-in KPI, and the second covers the specific requirement for an SCTP implementation.
Again, fixing this is a matter of priorities. In the three years since I filed <rdar://problem/3893228> I've encountered /two/ third party developers that need a protocol plug-in KPI:
A. The first was commercial developer with a significant source base that was dead in the water on 10.4.x without this KPI. At the time this way a major kerfuffle. After a prolonged discussion, these folks eventually decided to change their in-kernel protocol code so that it was no longer an actual protocol (IIRC they hosted it on top of a UNIX domain socket, so the majority of their code could continue to use sockets for read, write, select and so on). They're now fully hosted on KPIs, and their code is more-or-less binary compatible with Leopard.
B. The second is SCTP, as built by Michael and used by Andreas. This project continues to use the kernel's protocol plug-in mechanism, even though this isn't an official KPI. As such, it doesn't get the benefits that KPIs provide. That it, it is sometimes broken by dot releases of the OS, and it pretty much always broken by a major release. And it can't be fixed without Darwin source.
not to mention that I am "dead in the water" too without this KPI... We have no other choice. So far during those years we have seen the following theoretical approaches
- Running it on ethernet level and building our own IP stack - Running it as IP Filter and communicate with some kind of SCTP over TCP tunnel with userspace API
You are too good engineers to know that those concepts are really hard to do, by far not performant, impossible to maintain against a clean implementation of FreeBSD and just braindamaged idea..
- Running it in userspace This is what we have been doing before. It works but it lacks serious issues which are: a) multihoming doesnt work properly. Outgoing packets can not be controlled to leave on a specific IP interface. b) if userspace Application crashes, sockets dont get closed etc. Creates timeout issues with remote ends. (basically you have to wait 2 minutes before restarting). c) it requires the app to be root.
- Patching kernel - Running as kext
those last two methods have been proven to work and are stable, reliable and tested well. We have sent probably a billion SMS this way over the years. And SCTP is just at the lowest level of it. All protocols above that then have be done in userspace. Michael, Andreas, you have a decision to make. You can continue down your current path, with the negative consequences that I've described above. It's possible that Apple will eventually introduce a protocol plug-in KPI, but I can't say if or when that will happen. However, given that this is /not/ in 10.5, it seems likely that you'll be suffering this pain for years to come.
if we have a choice... The only real choice would be to kick out darwin and use Linux / Solaris / FreeBSD...
Alternatively, you can find some other way to implement the functionality that you need. Perhaps you can move your protocol into user space? Or perhaps rehost your in-kernel code on top of UNIX domain sockets? I realise that this will be technically challenging. Ultimately, it's a question of balancing the pain of this change with the pain that you suffer by not being hosted on supported KPIs.
see above. I'm sorry to be so blunt about this. I'm sorry that we don't have a better story for you. I wish that we had infinite resources and could accommodate all of the needs of all of our developers. Alas, that just isn't the case.
I know. Thats why we are in favor to the KEXT approach which just would need some publicly exported functions and the rest can be done externally.
Andreas Fink
Fink Consulting GmbH Global Networks Schweiz AG BebbiCell AG IceCell ehf
--------------------------------------------------------------- Tel: +41-61-6666330 Fax: +41-61-6666331 Mobile: +41-79-2457333 Address: Clarastrasse 3, 4058 Basel, Switzerland --------------------------------------------------------------- ICQ: 8239353 MSN: email@hidden AIM: smsrelay Skype: andreasfink Yahoo: finkconsulting SMS: +41792457333
|