Re: [Wonder-disc] ERSelenium without ERExtensions
Re: [Wonder-disc] ERSelenium without ERExtensions
- Subject: Re: [Wonder-disc] ERSelenium without ERExtensions
- From: Chuck Hill <email@hidden>
- Date: Wed, 27 Jun 2007 11:58:54 -0700
Hi Michael,
On Jun 27, 2007, at 5:53 AM, Michael Bushkov wrote:
Hi Chuck,
Chuck Hill wrote:
One of my goals is to _not_ be the person who writes and maintains
the functional tests. Because of this, it is important to keep the
tests separate from the applications and frameworks that it tests.
We think of selenium tests as about useful addition to Unit tests -
and
therefore we use different approach - the person who writes a part of
the applications should prepare tests for this part. And that's why we
treat selenium tests as a part of the application and store them in
the
application's folder.
Yes, that is different from my use / intention. I try to keep my UI
components very simple. So simple that they don't need tests.
Obviously, using D2W, you have very different concerns and very
different test needs. I have never worked on automated testing for a
D2W app, I can see that would be quite a challenge.
As my projects have functional tests anyway (part of our process), I
prefer to have only one set of Selenium tests rather than a developer
level set of tests and a domain level set of tests.
Anyway, to let a particular person edit a set of tests you need only
grant him access to Resources/Selenium subfolder of your application's
SVN/CVS repository and I don't see how ERSelenium-driven tests layout
can affect that.
- Will open/click commands in your test work ok if you run them on
the host with different WO base URL - for example if you switch
from "http://localhost/cgi-bin/WebObjects"; to "http://somehost/";?
Here is what I do. There are very few references to the URL. I
leave it as localhost for development as that is more convenient when
testing locally. For integration testing, I use Selenium RC (remote
control) from an Ant script. Before launching the suite, the Ant
script does a search and replace on the host and application names.
As I said earlier, that is only in a few places (like the log in test
case as the start of a string of tests cases).
Well this is some kind of workaround. Here is what we do. We develop
tests mostly in Selenium IDE. We specify Base URL there as:
http://localhost/cgi-bin/WebObjects/BugTracker.woa/-42421/
And then in tests we use only relative links of the following form:
|open|/wa/SeleniumAction/resetSession||
|openAndWait|/||
Yes, I use the full URL to call direct actions. I don't recall why I
don't use the Base URL, maybe an oversight! Component actions are
called by ID.
Selenium-ide handles base url and processes the test correctly.
ERSelenium transforms open/openAndWait/etc commands on the fly by
converting them to absolute URLs - and this allows them to be
correctly
processed by selenium-core.
I think a base URL can be used with Selenium RC as well. I should
look into that again.
The advantage of this approach is that you don't need to do test
source
code postprocessing. The same source code will be equally processed in
selenium-ide and on any host where ERSelenium is used.
Besides Selenium-RC looks like a kind of overkill solution to us,
so we
try to avoid it and use only selenium-core, which is much more simple
and easier to install and use.
Why does it look like overkill to you?
Most of the tests don't refer to the URL. Instead, I make extensive
use of HTML ID attributes. So a test for login would look like:
<tr> <td>type</td> <td>userName</td> <td>chill2</td> </tr> <tr>
<td>type</td> <td>password</td> <td>chill2</td> </tr> <tr>
<td>clickAndWait</td> <td>loginButton</td> <td></td> </tr>
The use of IDs serves two purposes. One, it isolates the tests from
changes to the host, application name, etc. Two, it makes the tests
easier to read and understand. Again, this supports my goal of
having functional (domain) experts write and maintain the functional
tests. This is supported by a Selenium locator ordering that gives a
higher priority to the ID than to XPath etc.
Anjo points out that the use of ID is a Bad Idea (tm). Anjo, of
course, is dead wrong in this. :-) Anjo's objection was that ID
has to be unique in a page. This is true. For items in repetitions,
I append an index to the ID. For example, instead of id = "view", it
is rendered as id = "view.1"; or id = "view.row.1"; This does
require some code in the component that uses the repetition, but I do
not object to that as it supports my goal of readable tests. There
is probably a way to do this automatically; so far it has not seemed
worth the effort.
Another potential problem area is components that are used multiple
times on a page. Consider a text input that wraps up a WOTextField,
a required marker, and a validation message. The WOD of page using
this might look like:
FirstName: TextInputWithValidation { value = firstName; }
LastName: TextInputWithValidation { value = lastName; }
Obviously, if I implement TextInputWithValidation with a WOTextField
like this:
Text: WOTextField { value = ^value; id = "text"; }
the ID will get duplicated and bad things will happen. Instead, I
put the ID up a level:
FirstName: TextInputWithValidation { value = firstName; id =
"FirstName"; }
LastName: TextInputWithValidation { value = lastName; id =
"LastName"; }
Text: WOTextField { value = ^value; id = ^id; }
This has the advantage of being more readable as well as eliminating
the duplicate ID problem. Passing the ID down has avoided the
problem of duplicate IDs for me so far. Sometimes you need to do a
bit more work and concatenate the ID from multiple levels, but I have
not run into a situation where it required much work or thought to
come up with a unique ID that was also sensible to someone reading
the test.
I fully agree - using IDs is probably the best approach in writing
selenium-tests. And it's great if you can support proper id generation
in your application from the very start. But it's not always
possible -
there are a lot of applications, that don't have such support, but
should be tested.
Yes, IDs are something that is best to have from the start. They are
also worth retrofitting if you want to have people who would have a
hard time with XPath writing and maintaining the tests.
And I'm not an XPath admirer, but it can sometimes be
extremely useful:
|select|xpath=//label[contains(string(),'Payment
type')]/following-sibling::div/select|By cash*|
This will select "By cash" option in the listbox next to the "Payment
type" label.
But that assumes that you are using the HTML label tag. In my
experience that is not a commonly used tag. So you end up either
having to retrofit label tags or IDs in existing applications (or
using really hard to read Xpath expressions).
How do you handle "clicking" on component action links?
IMHO, both of these approaches (IDs and xpath) of specifying locators
should be used - each one dependent on the concrete situation
On Jun 26, 2007, at 3:19 PM, Steven Mark McCraw wrote:
ERSelenium's idea is to facilitate a lot of common tasks that
are usually solved manually to organize Selenium testing. Among
these tasks are: - Automatic test suites generation from your
folder structure.
I am curious as to what you are doing for this. To me, Selenium
tests are for functional testing and so have no relation at all
to project structure.
The reason for doing it this way (says the guy who's used
ERSelenium a few times now) is that is a piece of cake to create
test suites. There is no maintaining an external test suite. You
just create a folder called Selenium (or whatever you configure the
name to be) in the Properties folder of the project you want to
test, and any folder you embed within it becomes a test suite when
you run everything from a simple URL. You drop your selenium tests
in child folders of the Selenium folder and you are done.
Are the test cases ordered?
They are executed in alphabetical order (depending on their
filenames).
Actually, in our testing environment we prefer to think of each test
case as of independent one - so we don't rely on ordering. However, if
the ordering is important - you can name the files accordingly -
010_case1.sel 030_case2.sel, etc.
In my tests, I often have a series of 3 - 7 tests that need to be run
in order. I also have larger "groups" where the tests in each group
can be run in any order but the groups have an order (e.g. need to
setup the users before running the user login tests).
It is ridiculously easy, as opposed to the whole chrome:// test
suite thing, which I never got to work, although I did not spend
much time trying because ERSelenium became available just as I
began getting good and familiar with using Selenium.
The up side to user ERSelenium test suites (besides how easy it is
to create them) is that they run in any browser. All you have to
do is point to the 'StartSeleniumTesting' direct action class in
your application url, and away you go. I believe I read that if
you use chrome:// as your URL, you are stuck with Firefox as a
testing platform, because only Firefox supports that protocol.
Well, up to a point. Have you tried file uploads yet? :-) There
are security restrictions on JavaScript and running in chrome mode is
the only way (that I know of) to get around them.
Yes, this is an issue. If you do want to test the file uploads then
the
simplest idea is to avoid using SeleniumStartTesting with firefox and
use this kind of address:
chrome://selenium-ide/content/selenium/TestRunner.html?test=http://
localhost/cgi-bin/WebObjects/App.woa/-42421/wa/
SeleniumTestSuite&resultsUrl=http://localhost/cgi-bin/WebObjects/
App.woa/-42421/wa/SeleniumTestResults&auto=true
This will let you using all ERSelenium's features without any problems
with file uploads.
There should also be a WO-specific workaround that will allow using
file
uploads in Selenium tests in any browser. It's a good candidate for my
TODO list.
I think you will find this impossible without a chrome type of hack.
JavaScript is prohibited from doing this for security reasons. If
you do succeed, I'd like to hear about it! :-)
- Writing tests in the preferred format - wiki/html.
Is that different from Selenese?
The difference is that instead of using html tags to delimit
things, you use pipe symbols, and it's a lot more readable (says
me). Instead of
<tr> <td>type</td> <td>userName</td> <td>chill2</td> </tr> <tr>
<td>type</td> <td>password</td> <td>chill2</td> </tr> <tr>
<td>clickAndWait</td> <td>loginButton</td> <td></td> </tr>
You have
|type|userName|chill2| |type|password|chill2| |clickAndWait|
loginButton||
But I believe you can use this format with or without ERSelenium.
Actually, you can't use it without ERSeleniun unless you use
selenium-ide.
I use a WYSIWG HTML editor. :-)
This is an option of course :) But we find wiki-like format very handy
when reading svn logs. And we actually use selenium-ide to prepare the
tests.
- Unified format of setup/tear down methods with appropriate
error messages.
I don't see the need for these. For me, the Selenium tests
should be from the point of view of the user of the application.
So any setup / tear down outside of loading the bootstrap data
should be done through the UI and be part of the test. In some
rare occasions (mimicking interactions from other systems), I do
need some setup that is not possible from the UI. In that case,
I create a direct action to do this and call it from the test in
a new window. The direct action is only enabled during testing
runs.
I agree with you here. I've just been using direct actions. Not
sure what ERSelenium brings to the table here or how to use it.
This feature is of course not essential, but by using ERSelenium-style
setup/tear methods you get the following:
After the method is invoked you will get 1 of 3 standard messages:
"Action command succeeded.", "Action command failed." (when the
exception is thrown) or "Invalid action command." (when the action
method is absent). That way the standard practice to check that action
method performed well is to have the following line in the test after
the action call:
|assertTestPresent|Action command succeeded.|
You can avoid any output logic in your action methods - as long as
they
don't throw any exceptions they're considered successful. Of course,
it's a minor feature, but it can save a bit of your time.
Regards,
Chuck
--
Practical WebObjects - for developers who want to increase their
overall knowledge of WebObjects or who are trying to solve specific
problems.
http://www.global-village.net/products/practical_webobjects
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden