Re: Leopard Sources?

At 21:00 -0600 29/10/07, William Kucharski wrote:

It's not divulging trade secrets to give general areas which are/remain closed source.

OK, I'll bite (-:

I know of three main areas:

o A significant chunk of what gets removed is comments.  They might, for example:

 - contain information related to upcoming hardware

 - explain workarounds for bugs in third party code, where Apple
   doesn't want to be seen as publicly criticising a third party

 - explain workarounds for Apple code, where Apple doesn't want
   to be seen as being critical of itself

o Some stuff is code that we received under non-disclosure from another party, or code based on information that we received under non-disclosure from another party.

o Some stuff is removed because it relates to upcoming Apple products.  For an extreme example, consider that, prior to the iPhone announcement, it would have been extremely foolish to reveal full support for the ARM platform.

Having said that, Apple works hard to ensure that:
a) Darwin can build and run as a standalone OS.
b) You can slide a "mach_kernel" built from open source underneath Mac OS X, and things still work.

So, while the Darwin kernel source is slightly different from Apple's internal source, you'd be hard pressed to find a situation where that makes any difference in practice.  If you discover such a situation, please let us know by filing a bug.

With regards SCTP, we already have plenty of bugs covering this.  The two lead bugs are:

<rdar://problem/3893228> Protocol plug-in KPI
<rdar://problem/4605154> Requirement for SCTP implementation on Kernel

The first covers the requirement for a general protocol plug-in KPI, and the second covers the specific requirement for an SCTP implementation.

Again, fixing this is a matter of priorities.  In the three years since I filed <rdar://problem/3893228> I've encountered /two/ third party developers that need a protocol plug-in KPI:

A. The first was commercial developer with a significant source base that was dead in the water on 10.4.x without this KPI.  At the time this way a major kerfuffle.  After a prolonged discussion, these folks eventually decided to change their in-kernel protocol code so that it was no longer an actual protocol (IIRC they hosted it on top of a UNIX domain socket, so the majority of their code could continue to use sockets for read, write, select and so on).  They're now fully hosted on KPIs, and their code is more-or-less binary compatible with Leopard.

B. The second is SCTP, as built by Michael and used by Andreas.  This project continues to use the kernel's protocol plug-in mechanism, even though this isn't an official KPI.  As such, it doesn't get the benefits that KPIs provide.  That it, it is sometimes broken by dot releases of the OS, and it pretty much always broken by a major release.  And it can't be fixed without Darwin source.

Michael, Andreas, you have a decision to make.  You can continue down your current path, with the negative consequences that I've described above.  It's possible that Apple will eventually introduce a protocol plug-in KPI, but I can't say if or when that will happen.  However, given that this is /not/ in 10.5, it seems likely that you'll be suffering this pain for years to come.

Alternatively, you can find some other way to implement the functionality that you need.  Perhaps you can move your protocol into user space?  Or perhaps rehost your in-kernel code on top of UNIX domain sockets?  I realise that this will be technically challenging. Ultimately, it's a question of balancing the pain of this change with the pain that you suffer by not being hosted on supported KPIs.

I'm sorry to be so blunt about this.  I'm sorry that we don't have a better story for you.  I wish that we had infinite resources and could accommodate all of the needs of all of our developers.  Alas, that just isn't the case.