Re: readInBackgroundAndNotify and rsync output
Re: readInBackgroundAndNotify and rsync output
- Subject: Re: readInBackgroundAndNotify and rsync output
- From: Robert DuToit <email@hidden>
- Date: Sat, 29 Jan 2011 09:01:46 -0500
Hi Ken,
On Jan 29, 2011, at 5:15 AM, Ken Thomases wrote:
> On Jan 29, 2011, at 2:16 AM, Robert DuToit wrote:
>
>> Thanks Ken,
>
> You're welcome.
>
>> Your thoughtful answer clarified what I have been experiencing.
>
> I'm glad I could help.
>
>> On recent machines readInBackground is definitely a small toll compared to rsync - it is on older slower Macs that the toll got closer to rsync's write and really crunched it.
>
> First, it shouldn't be the readInBackground, per se, that accounts for any significant overhead. It's presumably the parsing you're doing.
Yes I can see that from my tests with and without parsing. On the old Leopard ppc iMAc (well pretty archaic I admit) the parsing cuts rsync cpu about 75%! The parsing for readInBackground is:
-(void)readPipe:(NSNotification *)aNotification{
NSString *tempText;
NSData *theData = [[aNotification userInfo] objectForKey:NSFileHandleNotificationDataItem];
tempText = [[NSString alloc] initWithData:theData encoding:NSASCIIStringEncoding];
if (!tempText || tempText == nil || [tempText isEqualToString:@""]){
[applicationUtilityClass operationEnded];
[NSApp endSheet:progressWindow];
[self finishUp];
[tempText release];
return ;
}
NSCharacterSet * newLineSet = [NSCharacterSet characterSetWithCharactersInString:@"\r\n\v\f"];
NSArray *lines = [tempText componentsSeparatedByCharacterSet:newLineSet];
NSEnumerator * lineEnum = [lines objectEnumerator];
NSString * line;
NSString *lastPercent = nil;
NSString *lastLine= nil;
while (line = [lineEnum nextObject]) {
if (([line rangeOfString:@"%"].location != NSNotFound) && ([line rangeOfString:@"to-check="].location != NSNotFound)){
int endofEquals = ([line rangeOfString:@"to-check="].location + [line rangeOfString:@"to-check="].length);
lastPercent = [line substringFromIndex:endofEquals];
}else if ([line rangeOfString:@"/"].location != NSNotFound && [line rangeOfString:@"="].location == NSNotFound && [line rangeOfString:@"%"].location == NSNotFound){
lastLine = line;
}
}
if (lastPercent != nil)
{
NSArray *lastBits;
lastBits = [[[lastPercent componentsSeparatedByString:@")"] objectAtIndex:0] componentsSeparatedByString:@"/"];
if([lastBits count] > 1){
[percentCopiedTextField setStringValue:[NSString stringWithFormat:@"%@ of %@ files to check ",[lastBits objectAtIndex:0],[lastBits objectAtIndex:1]]];
int progNum = (([[lastBits objectAtIndex:1]doubleValue] - [[lastBits objectAtIndex:0]doubleValue]) / [[lastBits objectAtIndex:1]doubleValue]) *100;
double nowBar = [progressBar doubleValue];
if (progNum > nowBar) {
[progressBar setDoubleValue:progNum];
}
}
}
if (lastLine != nil)
{
[filepathsTextField setStringValue:lastLine];
}
[rsyncOutput readInBackgroundAndNotify];
[tempText release];
}
I scan the lines to get some progress and then update the progress once - I figured this cut down on setting the progress through each line of enum. But I am finding that the UI freezes sometimes when the screen saver has been on or display is off. The UI is flexible and progress cancel works so I think the readInBackground is doing what it is supposed to. Not sure why the freeze though, except maybe just because of the insane speed of updating the progress continually somehow gets stuck when display is off. The main app spikes up to 15% or more with the parsing. Not that much perhaps but rsync is clocking two processes at up to 30% and the fans are whirring. Other apps i've studied seem to have a much tamer output parser.
In the end I will probably use the log file option since I can control the speed of the timer.
>
> Second, I would think that the performance of rsync would scale at roughly the same rate as your parsing code. So, I would expect it to be able to keep up just as well on older system as newer ones. *shrug*
>
>
>> My app for years had been using an external log for rsync to write to and a timer for polling and it does work nicely, though with rsync the file can get enormous now with TB's of data transfer
>
> I would think it would be the number of files, not the total data size, that would be relevant.
Yes this is definitely true.
>
>> and there is overhead with streaming calls to NSTask 'tail' and 'ps' to see if rsync is still running.
>
> You should definitely not be creating subprocesses to run 'tail' or 'ps'. 'tail' is just a file reader, and your app can directly open and read the file. To monitor whether rsync is still running, you may be able to use the techniques illustrated in these articles:
I know - I didn't see any easy way to directly read in just some last lines of the log as 'tail' does assuming you don't want to read in 1 million lines of a log every fraction of a second and then parse. I'll research this more.
'ps' is another matter. I spent two days with sysctl and could get the process for a pid easy enough but then there is another problem. the fork and exec functions leave a zombie of rsync which I haven't been able to clear off. All the examples I have seen say to use 'wait()' in the parent after fork but that kills rsync immediately. All the systcl functions I have used return 1 for rsync because of the zombie which does leave in a minute or so. I am not concerned really about the short lived zombie but don't really want progress hanging around till it gets removed. ps grabs the "z" in the process table easily. If anyone knows of a way to kill the zombie, from my previous posted Helper Tool code, please let me know.
>
> http://developer.apple.com/library/mac/#technotes/tn/tn2050.html
> http://developer.apple.com/library/mac/#qa/qa2001/qa1123.html
>
> If you adapt the latter, you don't have to read in and scan the whole process list each time. The sysctl(3) man page shows alternative third-level names to use instead of KERN_PROC_ALL, including KERN_PROC_PID which lets you query a single known PID.
>
> It may also be possible to use a pipe to the rsync process not for communicating its output but just to monitor its lifetime. If you redirect most of rsync's output to a file (e.g. using the --log-file option), then you can monitor its stdout (and/or stderr) from your app. When it closes, the process has presumably exited.
I hadn't thought of that - I was redirecting the stdout in the Helper Tool using a timer to poll the log but never thought of the rsync option itself! I'll have to check this out. That would do away with the ps call nicely. Isn't the stdout already redirected to the file though via rsync? Maybe the two can coincide. If that does work I can poll the file in the readInBackground handler but then I may be back to the same readInBackground I have now. Perhaps I can run the polling timer alonside the readInBackground function...
>
> Cheers,
> Ken
Thanks Ken. This has been an enlightening discussion.
Rob
>
_______________________________________________
Cocoa-dev mailing list (email@hidden)
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden