Re: pthreads and standard C library calls and maybe magic Xcode switches
Re: pthreads and standard C library calls and maybe magic Xcode switches
- Subject: Re: pthreads and standard C library calls and maybe magic Xcode switches
- From: Cem Karan <email@hidden>
- Date: Wed, 22 Aug 2007 10:55:58 -0400
In an effort to reduce the traffic on the list on this, I'm going to
combine my replies to several people into a single message.
BTW, this list has turned into a perfect example of what happens when
locking goes wrong! I'm answering replies that people send me
directly, but because I'm in digest mode, and because not everyone
CCs me directly, I keep seeming like I'm asking questions about
things that have already been answered. Kind of fascinating in a
train crash sort of way...
On Tue, 21 Aug 2007 14:53:24 -0700, Shawn Erickson wrote:
On 8/21/07, Cem Karan <email@hidden> wrote:
I'm continuing to work on my pthreads based program, and realized
that I've run into a potential problem.
I think you misunderstand what volatile means and when you need to
use it.
In the examples you have given the use of volatile isn't needed since
the locking constructs you are using will ensure proper memory
barriers exist.
-Shawn
Well, my understanding of it is that volatile signals that the
compiler cannot optimize away a memory access, unless it can prove
that the result is never used within the program (which means that if
any thread uses the value, it won't be optimized away, or reordered,
or cached). Also, access of and writes to volatile qualified objects
require strict evaluation of the requirements in section 5.1.2.3 of
the C99 standard, which basically means that if you have a caching
protocol that doesn't require perfect cache coherency (e.g. NUMA
http://en.wikipedia.org/w/index.php?title=Non-
Uniform_Memory_Access&oldid=151468872#Cache_coherent_NUMA_.28ccNUMA.
29), then by marking the variable as volatile, you are telling the
compiler that it MUST push everything out to RAM. Am I right?
On Wed, 22 Aug 2007 10:24:30 +0200, Jonas Maebe wrote:
On 21 Aug 2007, at 23:55, Greg Guerin wrote:
There should be hardware in the cores/processors that keeps CPU
caches
coherent. That's why CPU's (or their memory controllers) have
cache-coherency protocols. E.g.:
<http://en.wikipedia.org/wiki/MESI_protocol>
That only happens automatically on some cpus (such as x86). On
others, such as the PowerPC, you have to assist the cpu in taking
care of the cache/memory coherency. See e.g.
http://ridiculousfish.com/blog/archives/2007/02/17/barrier/
Right, I was thinking of NUMA (see my above comment), but that is the
exact problem I'm hoping to avoid.
That said, all pthread locking functions are guaranteed to act as a
memory barrier, so if you use those you do not have to add any
additional barriers yourself.
OK, thanks, that was what I was wondering about. My concern is that
as Apple introduces new hardware in the future, it may choose to use
somewhat more exotic processors that won't maintain their cache
coherency. In that case, my code would break, and I really, really,
REALLY hate tracking down thread problems, especially in old code.
On Wed, 22 Aug 2007 07:37:05 -0400, Cem Karan wrote:
On Aug 21, 2007, at 5:53 PM, Shawn Erickson wrote:
On 8/21/07, Cem Karan <email@hidden> wrote:
I'm continuing to work on my pthreads based program, and realized
that I've run into a potential problem.
I think you misunderstand what volatile means and when you need to
use it.
In the examples you have given the use of volatile isn't needed since
the locking constructs you are using will ensure proper memory
barriers exist.
Are you saying that the locking constructs will ensure that the cache
is always flushed back to main memory? That locking and unlocking is
guaranteed as a sequence point for that, and therefore, regardless of
how a function is compiled (e.g., linked in static library that
wasn't volatile qualified), as long as there is a lock, it flushes to
main memory? I was talking to one of my coworkers to find out if
this is true, and he wasn't sure; that worried me because he is on
the C standards committee, and knows a heck of a lot more than I do
about all this...
And this is the threading problem that I'm having on the list! :)
Jonas answered my question earlier, but only on the list, and this
hit the digests later... As a suggestion to anyone else wanting to
comment on this thread, write to the list only; that way, everyone,
including those of us in digest mode, will see all messages in the
correct order.
On Wed, 22 Aug 2007 14:18:19 +0200, Olivier Tristan wrote:
If you're using your memory for real time data feed from outside, lock
can hurt performances a lot, then take a look at some LockFree
RingBuffer impl.
I know of a few lock-free algorithms, but I'm always worried that
I'll get the implementation wrong in some extremely subtle manner; do
you know of any good books that explain these algorithms well? I can
search, but I don't know if I'm getting good books or bad.
I hope that you didn't explain correctly your problem to your friend,
otherwise my faith in the C standards committee has been lost.
No, he understood my problem correctly, but he had to be careful; C
can work on NUMA architectures, where cache coherency isn't
guaranteed. Because of that, and because he didn't know how the pro
macs are designed, he had to be careful in answering.
That said, he did say that there is a lot of work currently being
done on C to define what the language guarantees in the face of
threads. Currently there is some ambiguity as to what must occur,
and what may occur, so, more than likely, at some point in the future
those requirements will be specified.
On Wed, 22 Aug 2007 15:47:28 +0200, Jonas Maebe wrote:
Are you saying that the locking constructs will ensure that the
cache is always flushed back to main memory? That locking and
unlocking is guaranteed as a sequence point for that, and
therefore, regardless of how a function is compiled (e.g., linked
in static library that wasn't volatile qualified), as long as there
is a lock, it flushes to main memory? I was talking to one of my
coworkers to find out if this is true, and he wasn't sure; that
worried me because he is on the C standards committee, and knows a
heck of a lot more than I do about all this...
This is not defined in the C standards, but in the POSIX (pthreads)
standards (POSIX/pthreads have nothing to do with C).
Agreed, and I am aware of that. However, the cache coherency problem
(which is related to the language) only becomes troublesome in the
face of threads. In a single process model (only the main thread),
all of the concerns I've mentioned before go away.
Thanks,
Cem Karan
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Xcode-users mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden