Foundry Academy · Software QA · Lesson 5 of 6

Defect severity and reproducibility

Report defects with reproducible evidence, user and business impact, appropriate severity, and a clear distinction between severity and repair priority.

Start the lesson

Your learning work, on this device

No signup, cloud storage, cross-device sync or verified completion. Saving is optional. This browser profile is shared with anyone who can use it; private mode, browser cleanup or storage limits may remove work. Use only fictional or non-sensitive material. Export a copy before relying on this device.

Not saved. Worksheets, answers and practice notes currently last only in this tab.

Practice markers are self-reported, never credentials.

01 · Explanation

Defect severity and reproducibility

Objective: Report defects with reproducible evidence, user and business impact, appropriate severity, and a clear distinction between severity and repair priority.

A useful defect report reduces uncertainty. Record a concise title, environment, build, preconditions, exact steps, expected result and source, actual result, frequency, evidence, affected users, impact, workaround, and related changes. Confirm reproducibility when safe, then state if the issue is intermittent or not reproduced. Avoid conclusions about root cause unless investigation supports them. Duplicate reports should be linked without discarding distinct environments or impacts. Protect sensitive data and use the organization's confidential vulnerability path for security findings rather than a broadly visible ticket.

Severity describes impact; priority reflects sequencing under current constraints. A severe defect may wait briefly for a specialist or safe fix, while a minor defect may be repaired immediately because it blocks a launch asset. Define levels with examples covering safety, security, privacy, financial loss, data integrity, legal or accessibility barriers, core workflow failure, degraded function, and cosmetic impact. Critical classifications require an escalation and containment rule. Product and engineering owners may set priority, but changes to severity should preserve the reasoning and evidence. After repair, verify the exact defect and nearby regression risk in the corrected build.

Before you begin

  • Confirm F07 and F08; record that exact reproduction steps, environment, approved focus target, assistive-technology version, timestamps, screenshots, recordings, logs, severity, priority, and remediation decision are absent.
  • STOP. If reproduction steps, environment, expected focus target, assistive-technology version, or supporting evidence is missing or conflicts with F07/F08, route the report to authorized accessibility, product, engineering, and QA owners; do not claim reproduction approval, execution, severity, priority, remediation, closure, or release impact.

Original overview module anchor →

02 · Compare the artifacts

Supported work. Visible uncertainty.

This is a fictional, sanitized software-quality case. All systems, users, transactions, and records are fabricated; testing remains authorized and bounded and provides no security, legal, accessibility, or defect-free guarantee.

BlueMesa checkout release

BlueMesa is a fictional business-to-business storefront preparing release 4.8 of its checkout. The approved requirement says authenticated buyers may purchase approved catalog items up to their account credit limit, apply one active contract discount, choose an allowed shipping method, and receive an accessible confirmation. The release candidate is build 4.8.17 in staging. Six browsers and two mobile devices are in scope, but the test plan lists only the latest desktop Chrome. A sample account has a $10,000 credit limit and an existing $9,600 open balance. The cart contains a $700 item, a 10 percent contract discount, $55 shipping, and an 8.25 percent tax rule. The product owner expects the order to be blocked, while the developer says discounts should be considered before the credit check; the requirement does not state calculation order. In preliminary testing, the keyboard focus disappears after the shipping modal closes, the error summary is not announced by the screen reader, and refreshing after payment timeout sometimes creates a second pending order. Logs show the same idempotency key on both requests, but the payment sandbox returns different response identifiers. The tax service was upgraded after the last regression baseline. A defect titled checkout broken includes no build, data, steps, expected result, actual result, or evidence. Marketing already announced Friday availability, and the release manager proposes accepting all medium defects. Learners must define risk and acceptance, control environments, write reproducible tests, explore edge cases within authority, classify defects, and make an evidence-bounded release recommendation. They must not conduct unapproved security testing or claim exhaustive coverage.

Supported example — reference only

Defect ID
A11Y-01
Build/environment
Not supplied
Preconditions
Shipping modal is open; exact page/account state is not supplied.
Ordered steps
Learner-proposed: navigate by keyboard, close the modal, inspect focus, then trigger the validation error and observe screen-reader announcement.
Expected result
Approved focus-return target and error-summary announcement behavior are not supplied.
Actual supplied observation
Focus disappears after modal close and the error summary is not announced — M05-I01 (F07); M05-I02 (F08).
User impact
Keyboard and screen-reader users may lose location or error context; affected-user count is not supplied.
Evidence attachments/status
Steps, browser/OS, assistive technology, recording, accessibility tree, and timestamp are pending.
Severity decision
Pending authorized accessibility review
Priority decision
Pending product/engineering triage
Owner
Accessibility reviewer and frontend engineering owner
Status
Open — reproduction evidence pending

A well-handled evidence gap

Defect ID
A11Y-GAP-02
Build/environment
Not supplied
Preconditions
Existing checkout-broken defect record; reproducible setup is not supplied.
Ordered steps
Not supplied
Expected result
Not supplied
Actual supplied observation
Existing defect lacks reproducibility and expected-result evidence — M05-I05 (F12).
User impact
Not supplied; must not be inferred from the defect title.
Evidence attachments/status
Safe account, environment, steps, expected target, capture files, and execution result are not supplied.
Severity decision
Blocked — evidence incomplete
Priority decision
Blocked — evidence incomplete
Owner
QA lead and accessibility reviewer
Status
Blocked — setup and oracle not supplied

Flawed approach — do not copy

Marking this accessibility defect-report starter “approved and complete” without the required evidence or reviewer is a flawed submission. Stop closure, severity, priority, or release claims when reproducibility, expected behavior, or accessibility evidence is absent.

Repair: Rework the accessibility defect-report starter as an evidence-backed draft, not an approved result. Create a stable defect ID and concise observed-behavior title. Record build and environment only if supplied; otherwise mark them not supplied. Separate the disappearing-focus and unannounced-error observations into reproducible expected-versus-actual statements. Check the revision against this requirement: Observed behavior, proposed reproduction, expected-state gap, impact boundary, and evidence request are distinct. If the required evidence is still absent, keep the decision blocked and identify the missing input or authorized reviewer.

Full case record, ambiguities and all assignments →

03 · Bounded practice

Build the accessibility defect-report starter.

Rewrite and classify the known failures using impact and reproducibility evidence.

Deliverable: Two defect-report starters for the observed accessibility and duplicate-order issues, one incomplete-evidence remediation record for F12, and a severity-versus-priority decision proposal.

Complete a bounded starter and gap analysis using only CB01, F07, F08, F09, F10, F12, and the assignment-scope record below. Populate supported fields, label every unavailable field “not supplied,” and cite the input ID for each material statement. You may design a proposed template, control, question, or decision rule, but must label it as a learner proposal rather than observed case evidence. Do not contact people, access live systems, run tests, sign records, claim approval, or invent names, dates, quotations, transactions, results, or source documents.

Exact supplied inputs for this assignment
  • M05-I01 · F07 — Keyboard focus disappears after the shipping modal closes.
  • M05-I02 · F08 — The error summary is not announced by the screen reader.
  • M05-I03 · F09 — Refreshing after payment timeout can create a second pending order.
  • M05-I04 · F10 — Duplicate requests share an idempotency key but receive different sandbox response identifiers.
  • M05-I05 · F12 — The existing checkout-broken defect lacks reproducibility and expected-result evidence.
  • M05-B01 · CB01 — Use CB01, the full versioned case brief printed once at the start of this packet, as a citable narrative source for details not normalized into F01–F12. Preserve its uncertainty language and do not treat narrative detail as approval, complete operational records, or professional judgment.
  • M05-S01 · F07, F08, F09, F10, F12 — Build a starter version of “Two defect-report starters for the observed accessibility and duplicate-order issues, one incomplete-evidence remediation record for F12, and a severity-versus-priority decision proposal.” from the listed case facts. Treat requested structures, controls, questions, calculations, and templates as learner-designed proposals. Where an operational record or result is absent, add a gap entry naming the missing evidence and authorized owner instead of fabricating it.

Operating procedure

  1. Create a stable defect ID and concise observed-behavior title.
  2. Record build and environment only if supplied; otherwise mark them not supplied.
  3. Separate the disappearing-focus and unannounced-error observations into reproducible expected-versus-actual statements.
  4. Draft minimal reproduction steps as a learner proposal and identify missing setup details.
  5. Name accessibility evidence to capture without fabricating attachments.
  6. Keep severity separate from priority and route both decisions to authorized roles.
  7. Final-QC source IDs, reproducibility gaps, user impact boundary, evidence request, owner, and pending status.
Field-by-field guidance
Defect ID
Assign a stable training defect ID.
Build/environment
Record exact supplied build/environment or Not supplied.
Preconditions
State the required starting UI/account state without inventing it.
Ordered steps
Write a bounded reproducibility sequence and keep execution pending.
Expected result
State an approved observable accessibility behavior or mark it Not supplied.
Actual supplied observation
Quote or paraphrase only the packet-supplied observation with source IDs.
User impact
Describe the bounded accessibility/business impact question; do not invent affected-user counts.
Evidence attachments/status
List required captures and mark them Not supplied or Pending.
Severity decision
Leave severity pending for authorized review unless the packet supplies a decision.
Priority decision
Leave priority pending and distinguish it from severity.
Owner
Name the accessibility, engineering, product, or QA role.
Status
Use Open, Blocked, or Draft; never claim reproduction or repair.
Accessibility defect-report starter · learning draft
Defect IDBuild/environmentPreconditionsOrdered stepsExpected resultActual supplied observationUser impactEvidence attachments/statusSeverity decisionPriority decisionOwnerStatus

Start with 1 rows; the complete workbook specifies 1 stable rows for this artifact. Add rows here or use the full download. No action is saved until you explicitly choose saving above.

Download complete six-module workbook (.md) · Structured case packet (.json)

Keep private client data, unpublished inventions, personal identifiers and credentials out of these public learning tools.

Module 5 · 2-item formative check

Defect severity and reproducibility

Choose an answer and request feedback. Read why each option does or does not fit the evidence. Answers stay in this tab unless you choose device-only saving; they are never submitted.

Question 1 of 2 · MODULE 5 · knowledgeWhich statement correctly distinguishes defect severity from repair priority?
Question 2 of 2 · MODULE 5 · scenarioF07 and F08 confirm accessibility observations, F09 confirms possible duplicate pending orders, F10 is conflicting about duplicate identifiers, and F12 confirms an incomplete checkout-broken defect. What may the tester responsibly report?

Answer either question to review its reasoning.

Inspect the artifact, not just your quiz answers

  • Observed behavior, proposed reproduction, expected-state gap, impact boundary, and evidence request are distinct.
  • F07 and F08 are both cited.
  • No reproduction, severity, remediation, or closure is fabricated.

Stop: Stop closure, severity, priority, or release claims when reproducibility, expected behavior, or accessibility evidence is absent.

Go: Proceed to qualified review when the report separates observed facts, proposed steps, expected-state gap, and requested evidence.

Escalate: Escalate keyboard and screen-reader barriers to accessibility and product owners; release impact remains their decision.

04 · Evidence to keep

Leave with usable work.

Submit complete reports, severity rationales, duplicate analysis, retest evidence, and the regression scope selected around each fix.

Download your artifact CSV and, if wanted, export the learning-work JSON above. Neither export is a reviewed submission or certificate. Device-only saving is optional; you must press Save my work now after edits.

When all six artifacts are ready, compare the full packet against the track rubric. Qualified human review is still required before real-world decisions.

Software developer reviewing code and test evidence on a tablet.
Learn the standard. Practice the work.
Developer testing and reviewing software code in a modern office.
Leave with evidence you can inspect.

Sources, scope and review boundaries

Curriculum 2026.10.08-learning-paths-1. External source dates below are record checks, not continuing guarantees. Verify current requirements before consequential use.

nist-sp-800-218 · Official guidance

NIST SP 800-218 Secure Software Development Framework Version 1.1

Primary NIST guidance for integrating secure software practices across the development lifecycle.

Open reviewed external source ↗

owasp-web-security-testing-guide · Official guidance

OWASP Web Security Testing Guide

The OWASP flagship testing resource; security techniques require explicit authorization and appropriate specialist competence.

Open reviewed external source ↗

ws-software-qa-operating-standard · Academy internal operating standard

Wealth Synergy software-QA internal operating standard

Academy-selected risk, test-case, defect, evidence, regression, and release-review controls. This is an internal operating standard selected by Foundry Academy; it is not law, accreditation, licensure, or an external-standard requirement.

Version 1.0 · reviewed 2026-09-01 · owner: Foundry Academy curriculum owner

A future Wealth Synergy private professional-development certificate would be issued only after its assessment, capstone, identity, reviewer, retention, access, deletion, appeal, and issuance controls pass quality review. No credential is currently issued. Any future certificate would not be an accredited academic qualification, professional license, or government certification.