Re: pthread_cond_signal and priority inversion
Re: pthread_cond_signal and priority inversion
- Subject: Re: pthread_cond_signal and priority inversion
- From: Paul Davis <email@hidden>
- Date: Tue, 24 May 2016 09:21:56 -0400
Basically, if you want inter-thread explicit communication then unless you are able to use 100% wait-free data structures, there is no way to avoid a lock *somewhere*. And using 100% wait-free data structures is more or less impossible for systems of any notable complexity, and this is specifically true of any general purpose OS kernel (such as mach) where the tradeoffs required to do everything wait-free generally argue against it.
So, you're not going to be able to avoid locks showing up in the code path. You're working on a general purpose OS - even if OS X has reasonable RT-like capabilities, it isn't an RTOS.
The only way you can avoid the ones in use space is to avoid the explicit communication represented by any signalling mechanism (semaphores, pthread_cond_t and the like). You have to make sure that the non-RT thread(s) simply wake up enough to ensure that whatever communication is required for the RT thread(s) to continue to function is guaranteed to happen. That is, the non-RT threads wake on a timer, check for any requests/data/whatever from the RT threads (in a lock-free FIFO), and deal with it. This can be done with no user-space locking at all.
Having said that, I can tell you that I rejected this design within Ardour in favor of taking the risk of invoking a lock from user space. We generally use POSIX FIFOs, and we write a single byte to the write side of the FIFO from the RT thread in order to wake up the non-RT thread(s). There is no explicit lock in our code, but this does traverse the kernel filesystem code, which implies some locking takes place. How much locking depends on what kernel is involved, and the answer is pretty different on OS X, Linux and Windows (we support all three with the same code).
I'm not sure that our rejection of the wait-free approach was correct, but it does avoid having the non-RT thread waking over and over and over and over just to check if anything needs to be done, rather than waking only when work is actually required.