Re: Unit testing and Xcode integration
Re: Unit testing and Xcode integration
- Subject: Re: Unit testing and Xcode integration
- From: Greg Hurrell <email@hidden>
- Date: Wed, 16 Aug 2006 02:30:33 +0200
El 16/08/2006, a las 1:40, Nir Soffer escribió:
- Failure does not stop the test. For example:
- (void)testFail
{
id obj = nil;
STFail(@"next line should NOT run");
// some code that should NOT run if we failed above
STAssertNotNil(obj, nil);
}
When you run this, you get 2 failures instead of one, and the code
that should not run does run.
I actually prefer that behaviour. If my code has 100 failing tests in
it I want to know about all of them, not just the first one. But I
suppose OCUnit could be modified to optionally throw an exception on
failing a test, and to abort when such an exception occurs.
- Using ugly C macros instead of nice ObjC like ObjcUnit uses:
[self assert:result equals:expected];
[self assert:result equals:expected message:@"description..."];
Macros are pretty much necessary because without them you can't
transparently include __FILE__ and __LINE__ in the test, and without
those you lose clickable results in the build results window.
Your examples would therefore really need to be:
[self assert:result equals:expected file:__FILE__ line:__LINE];
[self assert:result equals:expected file:__FILE__ line:__LINE
message:@"description..."];
Which isn't so nice...
- Test may crash the testing tool, which does not let you see other
tests until you fix the crash. I guess it will be hard to change
this, but it is very annoying
Yes, crashes are very annoying, but I guess a precondition for
testing code is that it actually be runnable (ie non-crashing)... The
testrunner could install signal handlers and try to catch this, but
it's probably best to just let the programmer fix the crash.
- Bogus results because of the integration with build:
- "Running custom shell script" - should be: "Running test suite"
- "error: -[Class test]: description" - should be "Failure: -
[Class test]: description"
- "Build failed" - should be "Tests failed"
- The status line: "Executed 354 tests, with 0 failures (0
unexpected) in 3.610 (4.389) seconds" is not displayed in the
results pane, only in the console.
- Inconsistent terms - "failures" in the console, "errors" in Xcode
build results.
All of these and several other related criticisms that you've made
aren't really problems with OCUnit; changing them would require Xcode
to be changed and only Apple can do that because Xcode, unlike
OCUnit, is closed source.
Specifically, Xcode will treat console messages beginning with
"error:" as errors and with "warning:" as warnings. You could try
something else like "Passed: " and Xcode will not display it (and
even if it did it wouldn't be what you want because it would be
highlighted in ugly red "look-at-me-I'm-an-error" text.
This is one of the reasons why several have spoken favorably of
UnitKit's Growl integration. Instead of seeing nothing in the no-
failure case you could see a Growl notification telling you that
everything went well.
- Lot of junk in the console output - because of the integration
with build instead of run. You my want to use the console to see
all the missing information in the build results pane, but then you
have to cope with this:
Well, with the exception of the test results summary, the only
information in the console that you should care about is failed test
information. The whole point of unit testing is that you don't have
to think about the passing tests, only the failed ones.
- Random order of test suites. I expect to see everything sorted in
the same way in all runs. It makes the output much easier to read
when you know where to look for results.
I imagine that could be easily fixed with some minimal changes, and
OCUnit is open source...
G
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Cocoa-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden