Defect management

How to raise, assign severity and prioritise defects.

What and why

Defect tracking is important in software engineering as complex software systems typically have tens or hundreds or thousands of defects: managing, evaluating and prioritizing these defects is a difficult task. Use of a defect tracking system can make the management task much easier. We use JIRA for defect management.

The defect management process contains the following elements:

  • Defect discovery – identification and reporting of potential defects. The information captured in the directory should be enough to reproduce the defect and allow the development team to determine the root cause and impact.
  • Defect analysis and prioritisation – The development team determines if the defect corresponds to an actual defect, if the defect has already been reported, and what the impact and priority of the defect is.
  • Defect resolution – here the development team determines the root cause, implements the changes needed to fix the defect, and documents the details of the resolution in the defect management software, including suggestions on how to verify the defect is fixed.
  • Defect verification – the build containing the resolution to the defect is identified, and testing of the build is performed to ensure the defect truly has been resolved, and that the resolution has not introduced side effects or regressions.
  • Defect communication – this encompasses automatic generation of defect metrics for management reporting and process, improvement purposes, as well as visibility into the presence and status of defects across all disciplines of the software development team.

Type of defects

All our projects use an agile development methodology and usually we encounter these situations:

  • Feature errors — defects found if the acceptance criteria on user stories are not met or components/layouts fail testing.
  • Regression Defects — errors introduced due to the side effects of a bug fix.

Raising defects

A defect must use the standard template for any defect with any severity level. At the minimum level the defect details must have the following sections presented in this order:

  • Defect description.
  • Steps to reproduce.
  • Expected result.
  • Actual result.
  • Also, if required defects should have screen shots added as attachments on JIRA.
  • For web-based defects, URL where defect lives.
  • Also, if required defects should have screenshots added as attachments on JIRA.

The information captured about the bug should be enough to reproduce the defect and allow development to determine its root cause and impact. Apart from the template above, all defects must have Severity mentioned.

Rules around defects:

  • Feature errors – defects found if the acceptance criteria on stories are not met or components/layouts fail the internal testing.
    • Don’t raise a defect.
    • Edit the user story to add your comment with issue description and assign it back to the developer.
    • Mark the user story as "Blocked" or flag it on JIRA and move it from testing review to "To Do" state.
  • Regression defects – defects on features from old release and on features delivered in previous sprints.
    • Raise a defect and assign it to the product backlog.
    • These defects are prioritised in the backlog and follow the standard agile process lifecycle, picked up in the sprint planning session.
    • Every bug must have a severity (S1 to S4) attached to it.

Severity definitions

  • Severity — signifies the degree of impact the defect has on the development or application being tested. It is the extent to which the defect can affect the software.
  • Priority — signifies the level of urgency of fixing the bug. In other words Priority means how fast/ how soon it has to be fixed.

All the bugs raised should adhere to the standard severity definition, S1 (being most severe) to S4 (minor).

  • S1 (Critical) — the defect affects critical functionality or critical data. It does not have a workaround.
  • S2 (Major) — the defect affects major functionality or major data. It has a workaround but is not obvious and is difficult.
  • S3 (Minor) — the defect affects minor functionality or non-critical data. It has an easy workaround.
  • S4 (Trivial) — the defect does not affect functionality or data. It does not even need a workaround. It does not impact productivity or efficiency; it is merely an inconvenience. Example: small layout discrepancies or spelling/grammatical errors.

Release criteria

We have the following set release criteria:

  • S1 100% (fixed, verified and closed)
  • S2 100% (fixed, verified and closed)
  • S3 70% (fixed, verified and closed)
  • S4 30% (all fixed, verified and closed)