Re: NSTask/NSPipe STDIN hangs on large data...
Re: NSTask/NSPipe STDIN hangs on large data...
- Subject: Re: NSTask/NSPipe STDIN hangs on large data...
- From: "Joe Pezzillo" <email@hidden>
- Date: Wed, 29 Jan 2003 17:08:18 -0700
Daryn,
Thanks again for your helpful info and suggestions. I think I'm getting
much, much closer!
Now (after trying to work on the rest of the much larger project for
awhile), I've got a couple of line-by-line writing functions, one each for
Synch and Asynch. I've tested them under the previously working conditions
to make sure they still do and they do.
Your hypothesis about the buffering must be correct, since now I find that
it's the amount of data returned from the grep that chokes it. In my tests,
greps that return less than 16K work fine; a grep that returns too much more
than that hangs; if the grep starts matching a lot at the beginning of the
file, it quickly hangs; if the grep matches a lot toward the end of the
file, it hangs later. Voila!, no doubt blocking exactly as you were
suggesting once there's too much data that needs to be dealt with on the
read end.
Of course, I thought that's what "readInBackgroundAndNotify" was supposed to
deal with in the first place for the Asynch version. Then I noticed the
RunLoop modes option to readInBackgroundAndNotify, figured maybe I must be
starving myself during the do/while that breaks the data into lines, so I
went on a tangent and included a message to the current RunLoop similar to
what I already do in my test object to wait for the Asynch tests to finish,
but to no avail...maybe I'm doing the run loop wrong or can specify a better
option, I'll have to look at that now, too, I guess.
However, I did try to add the polling you suggested using fcntl(), starting
with the Synch function, but even after reading and re-reading the manpage
(and digging out my tattered old copy of K&R to no avail), I must just not
be understanding it.
If I do this:
fileStatus = fcntl([readHandle fileDescriptor], F_SETFL, O_NONBLOCK);
It always seems to return 0, which I had been presuming (hoping?) meant no
error, but then if I try to read right away, I get a "Resource temporarily
unavailable" exception. Am I just supposed to catch the exception and ignore
it? That is, when it no longer complains that the resource is unavailable
means that there's some data to read?
Otherwise, just setting that NONBLOCK flag (and not attempting a read)
didn't seem to help.
So then I tried using select() based on its manpage,
snippet: {
int readyFDs;
fd_set fdreadset;
struct timeval zeroTV = {0,0};
FD_ZERO(&fdreadset);
FD_SET([readHandle fileDescriptor], &fdreadset);
readyFDs = select( 1, &fdreadset, nil, nil, &zeroTV );
if( readyFDs == -1 ) {
NSLog( @"TaskHandler:runCMDUsingLF:readyFDs == -1" );
} else if ( readyFDs != 0 ){
NSLog( @"TaskHandler:runCMDUsingLF:readyFDs:%d", readyFDs );
}
} end snippet
And it only returns 0 for readyFDs every time.
Not to mention the fact that all of a sudden I'm not doing just Cocoa
anymore! I'm still never sure if I'm doing the C pointers right (that's why
I tried to quit doing C in the first place for cryin out loud!!!), but it
compiles and runs through thousands of lines of tests, so unless the C
pointers are my problem, the code seems to work, I just don't know how to
use it (yet).
Also per your suggestion, I started to look at CFStreams in the CFNetwork
PDF (10/02 version, approx page 90), and I couldn't find a way to make a
read stream from my fileHandle, CFReadStreamCreateWithFile takes a file
NSURL, and unless I'm missing something really obvious (hey, I'm still new
to this!), I don't see how to get a path to pass to NSURL from the STDIO
fileHandle.
I truly appreciate all the time you've taken to help me with this, your
insights are very valuable. If you can suggest a book or online reference
for this stuff, I'll gladly devour it and spend another day testing before
asking any more questions!
Thanks again!
Joe
<email@hidden>
CC: Cocoa-Dev, Bill Bumgarner
On 1/24/03 10:31 PM, "Daryn" <email@hidden> wrote:
>
What I was actually suggesting, but I wasn't entirely clear, was to
>
send the input line by line (newline delimited), and to poll for output
>
after sending each line. That should generally avoid deadlock cases.
>
For polling, you can use fcntl(2) to set O_NONBLOCK on the file
>
descriptor, or use select(2) with a zero second timeout.
>
>
I think CFStreams might be worth investigating too.
>
>
On Monday, January 20, 2003, at 10:16 PM, Joe Pezzillo wrote:
>
> Daryn-
>
>
>
> Thanks for your insightful reply.
>
>
>
> Based on your info regarding the pipe buffer size, I tried yet another
>
> approach to the problem which was to not only chunk the STDIN data into
>
> smaller chunks, but then to try and writeData + close to flush the
>
> input
>
> pipe after each chunk is written. However, even if I try to get the
>
> Pipe and
>
> its fileHandleForWriting before each write, once I've closed the
>
> handle I
>
> can't get it back, so the first chunk gets written but that's it
>
> before an
>
> exception is raised. Did you have another idea of how I could implement
>
> around this 8KB buffer so I can send more data to (certain) UNIX
>
> commands
>
> via an NSTask/NSPipe? How would I do the output polling you mention
>
> (quoted
>
> below) other than via readInBackgroundAndNotify or a blocking read
>
> loop?
>
> Especially if the pipe hangs during the write command, I'll never get a
>
> chance to poll for any output on STDOUT.
>
>
Daryn
_______________________________________________
cocoa-dev mailing list | email@hidden
Help/Unsubscribe/Archives:
http://www.lists.apple.com/mailman/listinfo/cocoa-dev
Do not post admin requests to the list. They will be ignored.