System Prototyping, Verification, and Testing Methodologies

SS3 Term 1 • Week 3

About 45 minutes

Learning goals: Differentiate between Verification ('Are we building the product right?') and Validation ('Are we building the right product?'). • Compare Black-Box Testing techniques (Equivalence Partitioning, Boundary Value Analysis) with White-Box structural testing. • Deconstruct the Testing Hierarchy: Unit Testing, Integration Testing, System Testing, and User Acceptance Testing (UAT). • Write automated unit tests with assertions to detect regression defects.

In 1999, NASA lost the \$125 million Mars Climate Orbiter spacecraft as it entered the Martian atmosphere. The disaster was traced to a software calculation defect: one engineering team used imperial units (pound-force seconds), while another team used metric units (Newton-seconds). The software was never adequately tested before launch.

In computer systems engineering, writing code is only half the battle; proving that the code operates reliably, securely, and correctly under all conditions is what separates amateurs from software engineers.

In this lesson, we study Prototyping Models, clarify the vital distinction between Verification and Validation, master Black-Box and White-Box Testing, and write automated unit test suites.

Learning Outcomes

By the end of this lesson, you will be able to:

Distinguish between Verification and Validation (V&V).
Execute Black-Box test techniques: Equivalence Partitioning and Boundary Value Analysis (BVA).
Analyze White-Box testing: Statement coverage and branch decision coverage.
Implement automated Unit Testing in Python using assertions and test harnesses.

Key Vocabulary

Term	Technical Definition
Software Prototyping	The creation of incomplete, preliminary versions of a software application to visualize UI, test concepts, and clarify requirements.
Verification	The evaluation of software artifacts to ensure they satisfy technical architectural specifications ("Are we building the product right?").
Validation	The evaluation of the final executable system to ensure it satisfies user business needs ("Are we building the right product?").
Black-Box Testing	Testing software functionality without inspecting or knowing the underlying internal source code or algorithms.
White-Box Testing	Testing internal code structure, logic branches, loop paths, and exception handlers with full access to source code.
Unit Testing	The automated testing of individual functions, modules, or classes in complete isolation from the rest of the application.

The Testing Hierarchy: From Code to Customer

Software testing is structured as an inverted pyramid called the Testing V-Model:

       Development Phases                           Testing Phases
       ──────────────────                           ──────────────
       Business Requirements ───────────────> User Acceptance Testing (UAT)
               │                                      ▲
       System Architecture   ───────────────> System & Performance Testing
               │                                      ▲
       Module Design         ───────────────> Integration Testing
               │                                      ▲
       Source Code (Modules) ───────────────> Unit Testing

Unit Testing: Tests individual functions in isolation (e.g., does calculate_tax() compute correct VAT?).
Integration Testing: Tests communication interfaces between joined components (e.g., does the Web API correctly communicate with the PostgreSQL database?).
System Testing: Tests the complete integrated application against end-to-end specifications (security, load, failover).
User Acceptance Testing (UAT): Conducted by actual clients and educators to certify that the platform meets real school operations.

Black-Box Testing Techniques

When testing software as a black box, testers do not look at the Python or Java source code. They evaluate inputs and outputs.

1. Equivalence Class Partitioning (ECP)

Divide input ranges into valid and invalid partitions. Testing one value from each partition represents the entire group: - Example: A school grading form accepts ages between $10$ and $18$. - Partition 1: Ages $< 10$ (Invalid) $\rightarrow$ Test input: $7$ - Partition 2: Ages $10$ to $18$ (Valid) $\rightarrow$ Test input: $14$ - Partition 3: Ages $> 18$ (Invalid) $\rightarrow$ Test input: $22$

2. Boundary Value Analysis (BVA)

Bugs cluster at boundaries! For the age range $10$ to $18$, test: - Lower Boundary: $9$ (invalid), $10$ (valid minimum), $11$ (valid inside) - Upper Boundary: $17$ (valid inside), $18$ (valid maximum), $19$ (invalid outside)

Invalid (<10)          Valid Range (10 to 18)         Invalid (>18)
... 7, 8, 9   |   [10]  11, 12, 13, 14, 15, 16, 17  [18]   |   19, 20, 21 ...
          ▲        ▲                                  ▲        ▲
      Off-boundary Minimum                         Maximum  Off-boundary

Practical Activity: Automated Unit Testing Harness in Python

Let us write a robust student grading calculator and implement an automated Unit Test Suite with Boundary Value Analysis.

Step 1: Create `unit_test_lab.py`

# The Production Module to be Verified
def compute_waec_grade(score):
    """
    Computes WAEC Letter Grade:
    A1: 75 - 100
    B2: 70 - 74
    B3: 65 - 69
    C4: 60 - 64
    C5: 55 - 59
    C6: 50 - 54
    D7: 45 - 49
    E8: 40 - 44
    F9: 0 - 39
    Raises ValueError for invalid scores (< 0 or > 100).
    """
    if not isinstance(score, (int, float)):
        raise TypeError("Score must be numeric")
    if score < 0 or score > 100:
        raise ValueError("Score out of valid bounds (0-100)")

    if score >= 75:
        return "A1"
    elif score >= 70:
        return "B2"
    elif score >= 65:
        return "B3"
    elif score >= 60:
        return "C4"
    elif score >= 55:
        return "C5"
    elif score >= 50:
        return "C6"
    elif score >= 45:
        return "D7"
    elif score >= 40:
        return "E8"
    else:
        return "F9"

# Automated Test Suite Harness
def run_grade_tests():
    test_cases = [
        # (Score, Expected Grade, Description)
        (100, "A1", "BVA Upper Limit (100)"),
        (75,  "A1", "BVA Lower Limit A1 (75)"),
        (74.9,"B2", "BVA Upper Limit B2 (74.9)"),
        (70,  "B2", "BVA Lower Limit B2 (70)"),
        (50,  "C6", "Credit boundary C6 (50)"),
        (40,  "E8", "Pass boundary E8 (40)"),
        (39.9,"F9", "Fail boundary F9 (39.9)"),
        (0,   "F9", "BVA Lower Limit (0)")
    ]

    passed = 0
    print("=== EXECUTING AUTOMATED UNIT TESTS ===")
    for score, expected, desc in test_cases:
        actual = compute_waec_grade(score)
        assert actual == expected, f"FAIL on {desc}: expected {expected}, got {actual}"
        print(f"PASS: {desc:<30} | Score {score:>5} => {actual}")
        passed += 1

    # Test Exception Boundaries
    error_cases = [-1, 101, "Amina"]
    for err_input in error_cases:
        try:
            compute_waec_grade(err_input)
            print(f"FAIL: Exception was NOT raised for invalid input {err_input}")
        except (ValueError, TypeError):
            print(f"PASS: Correctly rejected invalid input: {err_input}")
            passed += 1

    print("-" * 55)
    print(f"ALL {passed} TESTS PASSED CLEANLY! ZERO REGRESSION DEFECTS.")

run_grade_tests()

Step 2: Execute the Test Suite

python unit_test_lab.py

Expected Results and Model Output

When you run unit_test_lab.py, your terminal will display:

=== EXECUTING AUTOMATED UNIT TESTS ===
PASS: BVA Upper Limit (100)          | Score   100 => A1
PASS: BVA Lower Limit A1 (75)        | Score    75 => A1
PASS: BVA Upper Limit B2 (74.9)      | Score  74.9 => B2
PASS: BVA Lower Limit B2 (70)        | Score    70 => B2
PASS: Credit boundary C6 (50)        | Score    50 => C6
PASS: Pass boundary E8 (40)          | Score    40 => E8
PASS: Fail boundary F9 (39.9)        | Score  39.9 => F9
PASS: BVA Lower Limit (0)            | Score     0 => F9
PASS: Correctly rejected invalid input: -1
PASS: Correctly rejected invalid input: 101
PASS: Correctly rejected invalid input: Amina
-------------------------------------------------------
ALL 11 TESTS PASSED CLEANLY! ZERO REGRESSION DEFECTS.

Common Mistakes to Avoid

Testing Only Happy Path Values: Never test only average numbers like $80$ or $55$. You must test boundaries ($74.9, 75.0$), zeros, negative numbers, and strings to catch edge crashes.
Confusing Bug Fixing with Testing: Finding bugs (Testing) and fixing bugs (Debugging) are separate skills. A good tester attempts to systematically break the software by feeding unexpected inputs.
Assuming Code Passes Just Because It Compiles: Compilation only proves that syntax is grammatically correct. It does not prove that calculations, boundary logic, or security policies are correct.

Notebook Summary

Verification checks technical conformance to specification; Validation checks user satisfaction and fitness for purpose.
Black-Box Testing uses Equivalence Partitioning and Boundary Value Analysis (BVA) to detect defects at range boundaries.
White-Box Testing evaluates internal statement coverage, conditional branches, and path logic.
The Testing Pyramid ascends from granular Unit Tests to Integration, System, and User Acceptance Testing (UAT).

Optional Extension Challenge

Modify compute_waec_grade to intentionally introduce an off-by-one bug (e.g., change if score >= 75 to if score > 75). Run run_grade_tests() and observe how the assertion captures the regression defect immediately!