Re: Unit testing and Xcode integration
Re: Unit testing and Xcode integration
- Subject: Re: Unit testing and Xcode integration
- From: Nir Soffer <email@hidden>
- Date: Wed, 16 Aug 2006 05:04:45 +0300
On Aug 16, 2006, at 3:30 AM, Greg Hurrell wrote:
El 16/08/2006, a las 1:40, Nir Soffer escribió:
- Failure does not stop the test. For example:
- (void)testFail
{
id obj = nil;
STFail(@"next line should NOT run");
// some code that should NOT run if we failed above
STAssertNotNil(obj, nil);
}
When you run this, you get 2 failures instead of one, and the
code that should not run does run.
I actually prefer that behaviour. If my code has 100 failing tests
in it I want to know about all of them, not just the first one. But
I suppose OCUnit could be modified to optionally throw an exception
on failing a test, and to abort when such an exception occurs.
After a test fails, your other tests (within the same test method)
may not be valid any more. For example:
STAssertNotNil(obj);
STAssertEqualObjects([obj name], @"Foo"]);
With your preferred behavior, when obj is nil, you will get two
failures, but the second failure is bogus. obj may be correct, but
you test code failed to obtain the instance, or the object
responsible to return the instance failed. This pattern is quite
common in my tests.
- Using ugly C macros instead of nice ObjC like ObjcUnit uses:
[self assert:result equals:expected];
[self assert:result equals:expected message:@"description..."];
Macros are pretty much necessary because without them you can't
transparently include __FILE__ and __LINE__ in the test, and
without those you lose clickable results in the build results window.
Your examples would therefore really need to be:
[self assert:result equals:expected file:__FILE__ line:__LINE];
[self assert:result equals:expected file:__FILE__ line:__LINE
message:@"description..."];
Which isn't so nice...
Good point - clickable results are important.
- Bogus results because of the integration with build:
- "Running custom shell script" - should be: "Running test suite"
- "error: -[Class test]: description" - should be "Failure: -
[Class test]: description"
- "Build failed" - should be "Tests failed"
- The status line: "Executed 354 tests, with 0 failures (0
unexpected) in 3.610 (4.389) seconds" is not displayed in the
results pane, only in the console.
- Inconsistent terms - "failures" in the console, "errors" in
Xcode build results.
All of these and several other related criticisms that you've made
aren't really problems with OCUnit; changing them would require
Xcode to be changed and only Apple can do that because Xcode,
unlike OCUnit, is closed source.
I agree, its not OCUnit fault, its the integration with Xcode.
- Lot of junk in the console output - because of the integration
with build instead of run. You my want to use the console to see
all the missing information in the build results pane, but then
you have to cope with this:
Well, with the exception of the test results summary, the only
information in the console that you should care about is failed
test information. The whole point of unit testing is that you don't
have to think about the passing tests, only the failed ones.
You don't have to think about it or look at the running tests, but
you like a complete report of all the tests, not only the failures.
Best Regards,
Nir Soffer
Attachment:
smime.p7s
Description: S/MIME cryptographic signature
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Cocoa-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden