The case study is a simple webapp that is supposed to be an online banking system (it is of course greatly simplified). I am the product owner of this app, and one Valtech consultant developed it for me (because I can hardly code something now). I built the functional specifications with FitNesse, using test tables to interact with the developer and discuss the functional requirements. In turn, he used the fixtures in a TDD manner to drive his development. Below is an example of the functional requirement that describes an immediate money transfer :
As you can see, we mixed textual descriptions in a use-case style with an example to illustrate the description. The example is actually a test, specifying preconditions, actions, and verifications. We used a RowEntry fixture to specify the preconditions, i.e. to populate the database with test data, then we used an Action fixture to describe the actions, and a Row fixture to verify the results. The example is in french. If you cannot understand french, you just need to know that we spent some time to choose appropriate fixture names, so that the test tables do not feel like automated tests or freaky technical stuff inserted in plain text specifications. We really wanted that the test tables read like real examples, so we chose fixture names like "There exists the following accounts", or "Consult account details", which read more naturally than freaky names we often see with FIT/FitNesse examples "com.myapp.test.fixtures.ConsultAccount". We also put some effort to mature a domain specific testing language (DSTL), so that we could reuse the expressions for precondition or verification with different requirements. For example, we used a lot the fixture "There exists the following accounts" to set up initial test data.
Now let's move to the GUI testing. I made use of a Selenium-FitNesse bridge (the like of webtest) to integrate my automated GUI tests with my specifications. That bridge provides a testing mini-language to allow non-technical people like me writing automated test cases without knowing Selenium scripting. I created a couple of cases to test the webapp, below is an example of an automated test case that traces to the specification given above:
In this example, we see that the flow of actions and verifications is overwhelmed with interface details and strategic instructions (wait for page to reload, etc). The testing mini-language gives a better readability than the equivalent Selenium script would do (some test specialists like to call this "keyword-driven testing"), but still, the test case is not good at serving the specification side. Both types of tests are complementary and should co-exist. The first FIT-like type serves to drive and support the specification process, it also helps in building a regression safety net; the second Selenium-like type serves to test the application and make sure every piece of the software properly fits together. In the end, we should get something like this:
The screenshot above shows the organisation of my FitNesse wiki for the webapp. I created a topmost suite that contains two "sub-suites": one is the specification artefact, containing the FIT-like tests, one is the acceptance test artefact, containing the Selenium-like tests. I made use of links to trace tests from the second artefact to the first, so that I can have an idea of impacted GUI tests when I change the specifications. I can run seperately the specification tests and the acceptance tests, or within a single click. It should be clearer now that TDR and GUI fonctional testing serves a different purpose. I used FitNesse and Selenium because they are open-source, but the same kind of approach can be followed using test management proprietary tools like QualityCenter in combination with test robots and requirements management tools. It's not going to be the same price though.
No comments:
Post a Comment