Testing approach

How we test... with some do's and don'ts.

What we do

Test basis: requirements documentation, user specifications, wireframes, designs and anything that is used to derive test information.

  1. Test basis analysis — analysing the test basis.
  2. Test designing — create test artefacts (convert test basis into test cases). These test cases can be manual or automated.
  3. Test planning — plan how to run the test cases (how many cycles, resources, test environment, etc.)
  4. Test execution — run planned test cases.
  5. Test reporting — report on planned vs actual, defects, observations, and recommendations.

In the days of V-model or waterfall, each of these tasks sat in an elaborate phase of the project. This is still valid even in agile development. The scale of these phases are small, but the elements of all tasks still exist.

Test basis analysis

Test basis analysis is the most important part of software testing. Don't test unless you have clearly understood the product's functionality so you have the right test basis on which to base your test information. Finding a defect with early feedback — before it gets in the code — and delivering value to the customer are key aspects in agile development. We need to make sure of the following:

  • Proper definition of requirements, i.e., acceptance criteria on user stories or raise a flag
  • Analyse and understand the requirements together with your fellow Engineers and BAs, i.e., story kick-off sessions
  • Question the requirements and don't play chinese whispers
  • Be involved in the process to define or agree the acceptance criteria

Types of testing

  • Functional — what a product does.
  • Non-Functional — how it does it.

Almost all the projects at Friday use agile development methodology and this is how we categorise our testing:

  • Feature testing — functional testing within work in progress or iterations
  • Regression testing — checking if newly added code has not regressed the old code
  • Manual web testing — checking web rendering and responsiveness (adaptive/fluid) across browsers and devices
  • Automated testing — writing automated tests for both backend API code and functional web automation.

Automated testing

The reason we stress automated testing more and more is because it plays a vital role in iterative development by providing regular feedback with continuous integration, and increases repeatability and accuracy.

The types of automated testing we perform:

  • API test automation — writing automated functional tests for backend APIs.
  • Functional web testing — automating the functional behaviour of the product.
  • Performance testing — measuring how a product behaves in load and stress.
  • Security/Pen testing — inject security faults to see how a product behaves. We do some basic security testing in-house, however the majority of testing is done by external security testing agencies.