Re: Automated user-interaction regression testing tools?
Re: Automated user-interaction regression testing tools?
- Subject: Re: Automated user-interaction regression testing tools?
- From: Mason Mark <email@hidden>
- Date: Sat, 1 Dec 2001 09:47:44 -0800
I agree that a fully scriptable and recordable app would be great, and
make testing easier. ;-) For that matter, chaining together Objective-C
action methods and running them against the app is not hard either, and
can put an app through its paces reasonably well.
The problems are:
a) Our apps do not generally even work, much less support scripting and
recording, at the pre-pre-pre-alpha stage that I want to start building
their regression test libraries (as early as possible).
b) Higher-level testing mechanisms like AppleScript can be great, but if
I had to choose one or the other I would choose the lower-level, more
primitive "bang on the keyboard" types of tools. Although not very
elegant, they capture the whole process, including details that an AS
interface to the app might bypass, and also help you deal with latent
bugs in the OS when they affect an app.
For instance, these kinds of tools would have helped me to more easily
isolate bug #2794144--a nasty bug that crashes NSUndoManager, but only
under rare circumstances which were pretty hard to generate manually
before the steps to trigger the bug were known. And had I been able to
isolate that, I would have realized that it was an OS bug and not ours,
and so not wasted many hours debugging a complex undo implementation
that wasn't broken, blah blah blah.
The Palm OS emulator ships with a simple but very useful thing called
Gremlins which basically simulates low-level user interactions, and in
my experience this has sometimes proved very useful.
I have had some luck using CGPostKeyboardEvent() to playback my own
hand-rolled keyboard event scripts. But it's tedious, and I was kind of
hoping some tools for doing this existed already...maybe it's an
opportunity for the enterprising developer, ;-)
Thanks for the responses,
--
Mason Mark
Five Speed Software, Inc.
On Saturday, December 1, 2001, at 07:55 AM, Ricky Sharp wrote:
On Saturday, December 1, 2001, at 06:18 AM, Finlay Dobbie wrote:
On Saturday, December 1, 2001, at 03:22 am, Mason Mark wrote:
We're looking for something that will allow us to record and play
back mouse actions and keystrokes, so that we can store long
sequences of user-GUI interactions and incorporate them into our
regression testing system.
I'm not sure how to record them, but you can synthesise events via the
CoreGraphics API. See CGRemoteOperation.h
That's a good place to start. Although, I think that if the app is
completely scriptable and recordable (AppleScript), that would
potentially give you a killer setup. Plus, your users would most
likely benefit too from having that capability with the app.
For my Carbon apps, I do have an automated testing system that is based
off a simple timer. Every n ms, an entry is read from an input file
(basically an op code). Most op codes deal with "messages" that are to
be sent to the app. Other op codes provide a query mechanism to ensure
the app is in a valid state. It didn't take me long at all to come up
with this code...about 4 days total maybe and the concepts could
definitely be applied to a Cocoa app.
Hopefully the solution you choose would allow your scripts to work
independent of monitor settings. Also, if you change the layout of
some of your widgets on the screen, you may have to alter your scripts
if they deal with set physical locations. A good scripting solution
allows you to post events to widgets regardless of where that widget is
physically located.
------------------------------------------------------------
Ricky A. Sharp Instant Interactive(tm)
Founder & President <http://www.instantinteractive.com>