• Open Menu Close Menu
  • Apple
  • Shopping Bag
  • Apple
  • Mac
  • iPad
  • iPhone
  • Watch
  • TV
  • Music
  • Support
  • Search apple.com
  • Shopping Bag

Lists

Open Menu Close Menu
  • Terms and Conditions
  • Lists hosted on this site
  • Email the Postmaster
  • Tips for posting to public mailing lists
Deadlock in bound stream pair during NSURLTask upload
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Deadlock in bound stream pair during NSURLTask upload

  • Subject: Deadlock in bound stream pair during NSURLTask upload
  • From: Jens Alfke <email@hidden>
  • Date: Wed, 16 Mar 2016 11:21:15 -0700
I’ve started getting a consistent hang in one of our unit tests, in which we’re uploading 30MB of data to an HTTP server via a PUT request. The body is a stream of data generated at runtime; to stream this to an NSURLTask we call 
CFStreamCreateBoundPair, set the resulting input stream as the task’s HTTPBodyStream, and then schedule the paired output stream with our runloop and write to it when notified space is available.

This is on OS X 10.11.4 (with both of the latest two betas.) It hasn’t occurred on another machine running 10.11.3. I’m going to file a Radar, but I’m also asking here in case anyone else has run into this and/or knows of a workaround.

My code's write call blocks forever like this — it’s been notified the output stream has space, but while writing the output stream is stuck acquiring a mutex:

    frame #0: 0x00007fff88e8cde6 libsystem_kernel.dylib`__psynch_mutexwait + 10
    frame #1: 0x00007fff965b7e4a libsystem_pthread.dylib`_pthread_mutex_lock_wait + 89
    frame #2: 0x00007fff9528e274 CoreFoundation`boundPairWrite + 52
    frame #3: 0x00007fff951aafa5 CoreFoundation`CFWriteStreamWrite + 437
  * frame #4: 0x0000000107116b5d CouchbaseLite`-[CBLMultiStreamWriter writeToOutput](self=0x000060c00004f780, _cmd="writeToOutput") + 429 at CBLMultiStreamWriter.m:254
    frame #5: 0x0000000107117712 CouchbaseLite`-[CBLMultiStreamWriter stream:handleEvent:](self=0x000060c00004f780, _cmd="stream:handleEvent:", stream=0x000060d000056170, event=NSStreamEventHasSpaceAvailable) + 1506 at CBLMultiStreamWriter.m:288
    frame #6: 0x00007fff95200094 CoreFoundation`_signalEventSync + 180
    frame #7: 0x00007fff9521d29e CoreFoundation`_cfstream_shared_signalEventSync + 478
    frame #8: 0x00007fff951de881 CoreFoundation`__CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 17

At the same time, the CFNetwork thread is also blocked inside boundPairRead, presumably the mate of the stream we’re writing to:

* thread #7: tid = 0x5393e, 0x00007fff88e8cde6 libsystem_kernel.dylib`__psynch_mutexwait + 10, name = 'com.apple.NSURLConnectionLoader'
  * frame #0: 0x00007fff88e8cde6 libsystem_kernel.dylib`__psynch_mutexwait + 10
    frame #1: 0x00007fff965b7e4a libsystem_pthread.dylib`_pthread_mutex_lock_wait + 89
    frame #2: 0x00007fff9528dc57 CoreFoundation`boundPairRead + 151
    frame #3: 0x00007fff951b49c5 CoreFoundation`CFReadStreamRead + 389
    frame #4: 0x00007fff96c22b1e CFNetwork`RequestBodyStreamProvider::readBodyStream(bool) + 308
    frame #5: 0x00007fff95200094 CoreFoundation`_signalEventSync + 180
    frame #6: 0x00007fff9521c67b CoreFoundation`_cfstream_solo_signalEventSync + 251
    frame #7: 0x00007fff951fd8fc CoreFoundation`_CFStreamSignalEvent + 476
    frame #8: 0x00007fff9528dec8 CoreFoundation`boundPairRead + 776
    frame #9: 0x00007fff951b49c5 CoreFoundation`CFReadStreamRead + 389
    frame #10: 0x00007fff96c22b1e CFNetwork`RequestBodyStreamProvider::readBodyStream(bool) + 308
    frame #11: 0x00007fff95200094 CoreFoundation`_signalEventSync + 180
    frame #12: 0x00007fff9521d29e CoreFoundation`_cfstream_shared_signalEventSync + 478
    frame #13: 0x00007fff951de881 CoreFoundation`__CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 17
    frame #14: 0x00007fff951bdf37 CoreFoundation`__CFRunLoopDoSources0 + 423
    frame #15: 0x00007fff951bd4df CoreFoundation`__CFRunLoopRun + 927
    frame #16: 0x00007fff951bced8 CoreFoundation`CFRunLoopRunSpecific + 296
    frame #17: 0x00007fff96a71111 CFNetwork`+[NSURLConnection(Loader) _resourceLoadLoop:] + 412

The latter stack looks very suspicious — it’s got two re-entrant calls to RequestBodyStreamProvider::readBodyStream stacked on top of each other. Frame 12 is a stream signaling an event, which triggers a read in frame 9, but somehow the read triggers another event in frame 6, which triggers a nested read in frame 3.
This kind of re-entrant call usually smells trouble to me, and in this case I strongly suspect that the outer call has a lock on the stream’s mutex, and so when the inner call tries to lock it again it deadlocks. This would then cause my thread to get stuck to the tar-baby as well when it tries to write to the paired stream.

—Jens
 _______________________________________________
Do not post admin requests to the list. They will be ignored.
Macnetworkprog mailing list      (email@hidden)
Help/Unsubscribe/Update your Subscription:

This email sent to email@hidden
  • Follow-Ups:
    • Re: Deadlock in bound stream pair during NSURLTask upload
      • From: "Quinn \"The Eskimo!\"" <email@hidden>
  • Prev by Date: Re: Whatever happened to Network Link Conditioner on iOS?
  • Next by Date: Re: Deadlock in bound stream pair during NSURLTask upload
  • Previous by thread: Re: Whatever happened to Network Link Conditioner on iOS?
  • Next by thread: Re: Deadlock in bound stream pair during NSURLTask upload
  • Index(es):
    • Date
    • Thread