Testing Process

How software testing fits into the overall development process.

When to engage with the testing team

  • After initial engagements with the client, and once we have a high level set of features or functionality for the product or service required.
  • When we have a fair understanding of these high level features or a functionality set.
  • When we have a high level view of technical implementation.
  • Definitely before statements of work are finalised.

There are even times when, in the initial scoping sessions, input from the testing team is beneficial.

The initial engagement with the testing team

These are key items that should be answered:

  • Resourcing required — the number of testers and skill-set.
  • High level testing approach.
  • Environment and infrastructure needs.
  • Any internal or client-mandated requirements for external security/penetration testing.
  • When can the testing team start probing the requirements with business analysts or product managers and get them into shape.

Testing within the development lifecycle

Almost all of the projects we build use agile development methodology (Lean or Scrum), and testing is a key step in the process. Every project involving Engineering uses an agile board with Testing Queue and In Testing process states. These are the key things that should be taken care of:

  • Testers are involved at the stage where high level specs are realised to detailed specs.
  • Testers are involved in framing acceptance criteria on JIRA tickets along side BAs.
  • Definition of Test Completion (at feature level):
    • All automated and manual tests are complete.
    • All these tests have been executed.
    • Tickets have correct comments stating testing "Passed".
    • No new defect tickets are raised for failing feature tickets.
    • All testing feedback is captured as comments in the failing tickets.
  • Testers participate in every agile team meeting such as stand-ups, prioritisation, planning and retrospectives.
  • Testers work closely with engineers, raising flags where required.
  • What things should be flagged within sprints or WIP:
    • Prioritisation of defects from the backlog.
    • Defects if the numbers are high in the backlog to help the engineering team focus on clearing these, then move to new requirements.
    • Failing automated tests off CI test runs, to agree on how they should be cleared.

External Testing requirements

For some projects, external Security/Pen testing may be required. These are usually mandated by a client, but for some, we may need to check and agree with the client upfront. These are the scenarios in which external testing may be required:

  • New hosting equipment or providers.
  • Changes to infrastructure within existing providers.
  • Any new digital application using customer sensitive information, or otherwise mandated by client.

During the costing exercise for a project, the client must be asked if they require any external testing. If yes, the testing team need to be informed as they need to engage with an external agency, and do this with the assigned Friday Project Manager. The testing team may be required to provide a full list of the product's existing capability/functionality to help the external agency cost their services and for the actual testing.

Business as usual (BAU)

Nearly all projects under maintenance and support are termed BAU. Our development process, testing process, and testing approach should not be different from what we do for all other projects. These are a few things we need to take care of:

  • The overall process and approach are not compromised, no matter what the release timeline is.
  • The test plan is emailed out to project team and key internal stakeholders.
  • The test plan includes scope, out-of-scope, resource (testers, environments, browsers and devices), and a rough time line of various activities
  • Agreement has been achieved with the Friday project manager when the code freeze should be achieved to allow a final round of manual regression and exploratory testing.
  • Testers send out daily reports with updates so any quality flags are raised as early as possible.