Practical Testing PHP Applications with Selenium

Testing software whilst ongoing development seems to be a boring job containing a lot of reoccurring
tasks.

You might end up entering the same test data, most of the time not
really context relevant one like for example „foo“ or
„bar“, into the same web-forms and get the feeling of doing
work that is not really useful for your daily developer business.
When timelines get closer and the development gets harder, these tests
are tending to be less important than the developing work on the
product.

<>Additionally new development
methodologies, like for example scrum, force you to make more
releases of your software in smaller time. Theoretically the effort
to develop the features of such a smaller footprinted phase gets
smaller too, but does the testing effort? Sometimes not: Seen over a
range of multiple product iterations the developed feature set will be
greater and more sophisticated as the code changes faster than we
could expect. As a reaction on all this effects we need to add more
testing. So what do we do? We use a person to click through the application, using its features as he or she thinks they are intended
to use. In the end we need another expert in the team for a boring
reoccurring task, finding bugs that are easy to find and most times,
simple to resolve. Did i say simple task,
reoccurring? Why not use a Software for the click-around testing in
our application to check if it still does what it needs to do ?

What is Selenium

Simply told, selenium is a javascript
library that opens your website in a own context to perform clicks
and other input operation to your application and checks the hereby
generated results with given data. This makes it possible to
perform reoccurring input works on your website and check the results
by a piece of code like you would do if you would perform the task of
checking for „Is it still working?“

A set of commands is
stored in a easy to note down format, may it be HTML or a piece of
ruby code. These Data represents the operations you need to perform
and the values to check as a result.

There Are 3 Kinds of Selenium Test
modes:

• Bot mode: Selenium Core
• Driven mode: Selenium Remote
  Control
• Record mode: Selenium IDE

Using Selenium IDE to automate a
Test

Just to run such a Test suite is not
the big part of the work, the main effort lies in the planning,
recording of the test. This is where selenium IDE comes to action.
This Firefox Plugin can Record a number of Commands just by browsing
your web app and using the options of the right-click context menu.

Here is a tutorial on Selenium by IBM.

Some hints that helped me a lot:

• Redirects can be workarounded with
  a pause command since pause is the same as waitForCondition with the
  difference that the check only happens once.

• Tables need html id attributes for
  easy handling of cell contents.

• The encoding of the testfiles must
  be the same as your website. Set up that in the options menu.

• Do not write new test methods until
  you have reviewed existing methods and commands to see if they do
  the job anyway

• Start with a login test

• Record a test that walks though
  the basic pages of you application and check for page headings or
  other unique parts of your page.

• Only test runs in Selenium IDE and
  not Selenium Core can perform file uploads, these tests need to be
  triggered one by one through the IDE (another manual job)

• Use real life values, neither „foo“
  nor „bar“ to check if your application works. If you do not have real
  life data, get it from your customer.

Recording a test with selenium IDE is
the wrong moment to look for new bugs, every click just needs to be in
the right place or your test may not do what you want in the end. If
you encounter problems that stop you from finishing test recording: Stop
recording the test, fix the problem, restart coding the test.
Calculate some changes to the codebase to make your test work easier
instead of deciding that this is test work and that is no coding
;). It will make your life easier and your application easier
maintainable. Books about software testing say that the effort to
automate a test is 10 times the effort to do the same test by hand. I
found out that these numbers seem to be real, so do not expect your
test suites to grow too fast, especially when you are new to test
recording.

Grouping Tests into a Test Suite and
Storing into CVS

Testing applications with big
testsuites can be very annoying: Its no fun to record and run big test
suites. So far some hits that helped me through the Test Hell:

• divide one large test into small
  subtests: create a separate test for the login, one test for each of the crud
  parts (create, read, update, delete)

• write a small shell script that
  initialises your database before you start the test run, in this case each
  test runs on a equal database.

• put your database export into cvs
  along with the test code. Changes to the database are a very common
  source for problems in PHP Application development, so its up to you
  to minimize that by testing only on a valid set of data.

• store the tests and the data in
  the same cvs as your code and make it the same revisions.

More Dirty Tricks ;)

Run your error_reporting on to E_ALL
to see really all errors on the screen. A test run of a good piece of
PHP code must run w/o errors or just a few expected ones. If you
have unit tests, run them before or after your Selenium Test runs in
the web Test Suite and let Selenium check the results (just by
checking if the „Test RUN OK“ appears in the screen). See here how you create a assertTextNotPreset check that will be very helpful in these cases.

Using Selenium RC

The last piece of
the selenium family is a Java console Application that starts
a webserver, listens to commands you send and works as a kind of
proxy to your browser. As the name says: a remote control to your
automated tests. So you can bind the events

1. checkout
   sourcecode

2. setup the
   test environment (copy files to webserver folders, initialise database)

3. run tests

into a small
shellscript. Check OpenQA Homepage for more information.

All in all Selenium provides a simple
way to execute Acceptance Tests on Web Applications and as soon as
you have a remarkable test suite, covering the basic steps of your
Application you will find out that it eases developer life and daily
usage lowers the number of sideffects (bugs) that pop up because of
changes to the codebase. Its a real timesaver, if you know how to use
it, so get some practice.