03 Testing Fundamentals: Coverage Concepts Quiz

Correct Answer: B

Statement coverage (C0) measures the percentage of executable statements in your code that are executed during testing.

Formula:

\(\text{C0} = \frac{\text{Executed Statements}}{\text{Total Statements}} \times 100\%\)

Example:

def example(x):
    if x > 0:          # Statement 1
        print("pos")   # Statement 2
    return x           # Statement 3

If you test with example(5):

• Statements executed: 1, 2, 3 (all 3)
• C0 = 3/3 = 100%

If you test with example(-5):

• Statements executed: 1, 3 (statement 2 skipped)
• C0 = 2/3 = 67%

Why other options are wrong:

• A: That would be “function coverage,” not statement coverage
• C: Coverage is about execution, not a simple ratio of tests to code
• D: This describes branch outcomes, not statement execution

Correct Answer: A

Branch coverage (C1) measures the percentage of decision outcomes (True/False branches) that are executed during testing.

Formula:

\(\text{C1} = \frac{\text{Executed Branches}}{\text{Total Branches}} \times 100\%\)

Example:

def example(x):
    if x > 0:          # Decision with 2 branches: True, False
        return "pos"   # True branch
    return "neg"       # False branch (implicit else)

If you test with example(5):

• True branch: Executed
• False branch: NOT executed
• C1 = 1/2 = 50%

To get 100% C1, you need BOTH:

• example(5) → True branch
• example(-5) → False branch

Why other options are wrong:

• B: Counting if-statements isn’t coverage; it’s a code metric
• C: This isn’t a standard coverage measure
• D: This describes “execution count,” not branch coverage

Correct Answer: B

A coverage criterion is a formal rule that defines what elements of a program must be exercised (covered) by a test suite for that suite to be considered “adequate” according to that criterion.

Examples of coverage criteria:

• Statement Coverage (C0): Every statement must be executed at least once
• Branch Coverage (C1): Every branch outcome must be executed at least once
• Path Coverage: Every possible path through the code must be executed
• Condition Coverage: Every boolean sub-expression must be True and False

Why this matters:

Different criteria have different strengths (fault detection) and costs (number of tests needed). Choosing the right criterion depends on:

• Risk level of the code
• Available testing resources
• Required confidence level

Why other options are wrong:

• A: That’s a performance metric, not a coverage criterion
• C: Coverage is about what’s tested, not a fixed count
• D: Coverage criteria are for test adequacy, not grading

Correct Answer: B

Subsumption means that one coverage criterion is “stronger” than another. If criterion A subsumes criterion B, then achieving 100% coverage under A automatically guarantees 100% coverage under B.

C1 subsumes C0 because:

To cover both branches of a decision:

if condition:
    statement_A   # True branch
else:
    statement_B   # False branch

• You MUST execute statement_A (to cover True branch)
• You MUST execute statement_B (to cover False branch)
• Therefore, both statements are covered

The reverse is NOT true:

You can have 100% C0 without 100% C1:

def risky(x, logging=True):
    if logging:
        log_event(x)   # Always executed if logging=True
    return process(x)  # Always executed

Testing only with logging=True gives:

• C0: 100% (all statements executed)
• C1: 50% (only True branch of if logging tested)

Why other options are wrong:

• A: Subsumption is about coverage, not speed
• C: The opposite is true; C1 is more thorough
• D: C1 typically requires MORE tests (must cover both branches)

Correct Answer: B

Test adequacy is a property of a test suite relative to a specific coverage criterion. A test suite is adequate if it achieves 100% coverage according to the chosen criterion.

Examples:

• C0-adequate: Every statement is executed by at least one test
• C1-adequate: Every branch outcome is executed by at least one test

Important distinction:

• Adequate means “meets the criterion’s requirements”
• Adequate does NOT mean “tests are correct” or “code is bug-free”

A test suite can be C0-adequate while still:

• Having weak assertions
• Missing edge cases
• Failing to detect actual bugs

Why other options are wrong:

• A: Tests passing is about correctness, not adequacy
• C: Adequacy isn’t about count; it’s about coverage
• D: Speed is a performance concern, not adequacy

Correct Answer: B

To achieve 100% C1 on this code, you must cover:

1. True branch (value > 0 is True) → Executes return "positive"
2. False branch (value > 0 is False) → Executes return "negative"

By covering both branches, you’ve necessarily executed:

• Line 2: if value > 0: (the condition itself)
• Line 3: return "positive" (True branch statement)
• Line 5: return "negative" (False branch statement)

All 3 executable statements are covered, so C0 = 100%.

Visual proof:

Why other options are wrong:

• A: C1 doesn’t “count” statements; it counts branch outcomes
• C: Test length isn’t related to subsumption
• D: The order of calculation doesn’t affect the relationship

You can verify your answer by running pytest-cov! Create these files:

src/grading.py:

def categorize(score):
    if score >= 90:
        return "A"
    elif score >= 80:
        return "B"
    elif score >= 70:
        return "C"
    else:
        return "F"

tests/test_grading.py:

from src.grading import categorize

def test_high_score():
    assert categorize(95) == "A"

Run coverage:

uv run pytest tests/ --cov=src --cov-report=term-missing

Prerequisites: Python 3.12+, uv package manager, pytest and pytest-cov installed (uv add pytest pytest-cov).

Correct Answer: B

Let’s count the executable statements:

def categorize(score):
    if score >= 90:      # Statement 1 (condition check)
        return "A"       # Statement 2 (executed)
    elif score >= 80:    # Statement 3 (NOT executed - short-circuit)
        return "B"       # Statement 4 (NOT executed)
    elif score >= 70:    # Statement 5 (NOT executed)
        return "C"       # Statement 6 (NOT executed)
    else:
        return "F"       # Statement 7 (NOT executed)

With categorize(95):

• Statement 1: Executed (condition is True)
• Statement 2: Executed (returns “A”)
• Statements 3-7: NOT executed (function returns early)

Calculation:

• Executed: 2 statements
• Total: 5 statements (the 4 returns + the function’s entry/first condition)
• C0 = 2/5 = 40%

Note: Exact counts may vary depending on how your coverage tool counts statements (some count if and elif separately, some don’t). The key insight is that only a small portion of the code is executed.

Why other options are wrong:

• A (25%): Too low; we executed more than 1 out of 4
• C (50%): We didn’t execute half the statements
• D (100%): We clearly skipped the B, C, and F branches

Verify your answer using pytest-cov with branch coverage enabled:

tests/test_grading.py:

from src.grading import categorize

def test_scores():
    assert categorize(95) == "A"
    assert categorize(85) == "B"

Run with branch coverage:

uv run pytest tests/ --cov=src --cov-branch --cov-report=term-missing

The --cov-branch flag enables branch coverage reporting. Look for the “Branch” and “BrPart” columns in the output.

Correct Answer: B

Let’s identify all branches:

if score >= 90:      # Branch 1 (True), Branch 2 (False)
    return "A"
elif score >= 80:    # Branch 3 (True), Branch 4 (False)
    return "B"
elif score >= 70:    # Branch 5 (True), Branch 6 (False)
    return "C"
else:
    return "F"

Total branches: 6 (each condition has True and False outcomes)

With our tests:

Branches covered: 1, 2, 3 = 3 branches Branches NOT covered: 4, 5, 6 (need scores like 75 and 50)

Calculation:

C1 = 3/6 = 50%

To achieve 100% C1, we’d need:

• categorize(95) → A (covers score >= 90 True)
• categorize(85) → B (covers score >= 90 False, score >= 80 True)
• categorize(75) → C (covers score >= 80 False, score >= 70 True)
• categorize(50) → F (covers score >= 70 False)

Correct Answer: B

The “Missing” column shows line numbers that were not executed by any test in your test suite.

What this means:

• Lines 18, 19, 20: Not covered by tests
• Lines 35, 36, 37: Not covered by tests
• These lines represent untested code paths

What to do about it:

1. Look at the code on those lines
2. Understand why they weren’t executed:
   • Is it a conditional branch that wasn’t triggered?
   • Is it error handling code?
   • Is it dead code that should be removed?
3. Add tests that execute those lines (if they’re valid code)

Example scenario:

# Line 18-20 might be:
if special_case:
    handle_special()
    return early_result

# Your tests never triggered special_case = True

Why other options are wrong:

• A: Coverage tools don’t detect syntax errors; they measure execution
• C: Missing lines aren’t necessarily bad; they just need tests
• D: The importance of code isn’t related to whether it’s tested

Correct Answer: B

Let’s analyze what “Missing: 3, 7” means:

• Line 3 missing: return "Error" was never executed
  • This means value is None was never True
  • So value is None was always False
• Line 7 missing: return "Zero" was never executed
  • This means value == 0 was never True
  • So value == 0 was always False

What was likely tested:

• validate(5) → Positive (lines 2, 4, 6, 8 executed)
• validate(-3) → Negative (lines 2, 4, 5 executed)

What was NOT tested:

• validate(None) → Would execute line 3
• validate(0) → Would execute line 7

Why other options are wrong:

• A: Line 5 (return "Negative") is NOT missing, so value < 0 was True at some point
• C: Line 5 is NOT missing, so value < 0 was covered
• D: Lines 3 and 7 are missing, so not all conditions were fully tested

Correct Answer: C

100% coverage means every statement was executed, but it says nothing about:

• Whether assertions verify correct behavior
• Whether all important inputs were tested
• Whether the logic is actually correct

Example of 100% coverage with no bug detection:

def divide(a, b):
    return a / b   # Bug: doesn't handle b=0

def test_divide():
    result = divide(10, 2)
    assert result is not None  # Weak assertion! Doesn't check value

This achieves 100% coverage but:

• Doesn’t verify the actual result (should be 5)
• Doesn’t test b=0 (which would crash)
• Provides false confidence

What coverage DOES tell you:

• Which code was not executed at all (definitely not tested)
• Gaps in your test suite

What coverage does NOT tell you:

• Whether tests actually verify correctness
• Whether edge cases are handled
• Whether the code is bug-free

Why other options are wrong:

• A: Correctness requires proper assertions, not just execution
• B: You can achieve 100% coverage with very few inputs
• D: Coverage measures execution, not correctness

Correct Answer: C

The test achieves high coverage by executing:

• total = sum(items) → 60
• if discount_percent > 0: → True
• discount = total * (discount_percent / 100) → 6
• total = total - discount → 54
• return total → returns 54

But the assertion assert result > 0 is extremely weak:

• It would pass for result = 1 (wrong!)
• It would pass for result = 100000 (wrong!)
• It would pass for result = 54 (correct)

A proper test should verify the exact expected value:

def test_calculate_total_with_discount():
    # Arrange
    items = [10, 20, 30]
    discount_percent = 10

    # Act
    result = calculate_total(items, discount_percent)

    # Assert - verify the exact expected value
    expected = 54  # (10+20+30) - 10% = 60 - 6 = 54
    assert result == expected

Key insight:

Coverage tells you the code ran, but only good assertions tell you the code worked correctly.

Why other options are wrong:

• A: The test likely does achieve good coverage (all lines executed)
• B: The syntax is valid Python
• D: While true, the weak assertion is the critical problem

Correct Answer: B

An incorrect calculation formula might still execute perfectly, achieving 100% coverage, while producing wrong results.

Example:

def calculate_area(radius):
    return 3.14 * radius * radius  # Bug: should use math.pi

def test_area():
    result = calculate_area(10)
    assert result is not None  # Passes! But result is slightly wrong

This achieves 100% coverage but:

• The formula uses 3.14 instead of math.pi
• The result is approximately 314 instead of 314.159…
• A weak assertion won’t catch this

What coverage CAN detect (indirectly):

• A (missing import): Would crash when that module is used
• C (syntax error): Python won’t run at all
• D (undefined variable): Would raise NameError when executed

Key insight:

Coverage detects crashes (code that fails to run) but not logical errors (code that runs but produces wrong output).

This is why you need:

1. Good coverage (code is executed)
2. Strong assertions (behavior is verified)
3. Edge case testing (boundary conditions checked)

Correct Answer: B

70% coverage can be acceptable when the uncovered code is:

1. Dead code that should be removed
2. Defensive error handlers for “impossible” conditions
3. Platform-specific code not relevant to current environment
4. Debug/logging code that’s not critical to test

Examples of acceptable uncovered code:

def process_file(path):
    try:
        with open(path) as f:
            return f.read()
    except PermissionError:
        # Only happens in restricted environments
        log_error("Permission denied")
        return None
    except Exception as e:
        # Defensive catch-all, should "never" happen
        log_critical(f"Unexpected: {e}")
        raise

The except blocks might be uncovered if your tests only use accessible files. That’s often OK.

When 70% is NOT acceptable:

• Core business logic is uncovered
• Happy-path functionality isn’t tested
• The 30% contains critical code

Why other options are wrong:

• A: 100% isn’t always practical or meaningful
• C: Cost alone isn’t a good reason; prioritize critical code
• D: Project stage doesn’t determine acceptable coverage

Correct Answer: B

Coverage and correctness are different concepts:

High coverage + weak assertions = False confidence

def test_broken():
    result = buggy_function()  # Runs the code (coverage!)
    assert result is not None   # Doesn't check correctness

Low coverage + strong assertions = Incomplete testing

def test_one_path():
    assert calculate(10) == 100  # Correct for this input
    # But what about calculate(-5)? calculate(0)?

Ideal: High coverage + strong assertions + good test design

def test_positive():
    assert calculate(10) == 100

def test_negative():
    assert calculate(-5) == -50

def test_zero():
    assert calculate(0) == 0

def test_edge_case():
    assert calculate(0.001) == 0.01

Why other options are wrong:

• A: Coverage proves execution, not correctness
• C: Low coverage means untested code, not incorrect code
• D: They’re complementary but distinct concepts

Test your hypothesis! Create these files and run coverage:

src/age_classifier.py:

def classify_age(age):
    if age < 0:
        return "Invalid"
    elif age < 18:
        return "Minor"
    elif age < 65:
        return "Adult"
    else:
        return "Senior"

tests/test_age.py (initial tests that leave Line 9 missing):

from src.age_classifier import classify_age

def test_ages():
    assert classify_age(-5) == "Invalid"
    assert classify_age(17) == "Minor"
    assert classify_age(30) == "Adult"

Run coverage to see Line 9 missing:

uv run pytest tests/ --cov=src --cov-report=term-missing

Then add your chosen test and run again to verify it covers Line 9!

Correct Answer: D

Line 9 (return "Senior") is in the else branch, which executes when:

• age < 0 is False (age is 0 or positive)
• age < 18 is False (age is 18 or older)
• age < 65 is False (age is 65 or older)

So we need age >= 65 to reach line 9.

Analysis of each option:

Only option D covers line 9.

Why other options are wrong:

• A: Covers line 3 (age < 0 is True)
• B: Covers line 5 (age < 18 is True)
• C: Covers line 7 (age < 65 is True)

Correct Answer: B

Let’s trace what covers each missing line:

Line 3 (raise ValueError): Needs weight <= 0

• weight = -1 → Line 2 is True → Line 3 executes

Line 8 (return base_cost * 1.5): Needs weight > 0 AND express=True

• weight = 5, express = True → Reaches line 7, line 7 is True → Line 8 executes

Analysis of options:

Only option B covers BOTH missing lines.

Why other options are wrong:

• A: Only covers line 8, not line 3
• C: Only covers line 3, not line 8
• D: Both tests have positive weight, never covers line 3

Correct Answer: B

BrPart stands for “Branch Partial” - it counts branches where only one outcome (True OR False) was tested, but not both.

Example of partial branch coverage:

def process(value, debug=True):
    if debug:           # Branch: tested only when debug=True
        log(value)
    return transform(value)

If all tests use debug=True:

• True branch: Covered (log is called)
• False branch: NOT covered (log is skipped)
• This branch is “partially covered” → contributes to BrPart

Column meanings:

• Stmts: Total statements (20)
• Miss: Statements not executed (2)
• Branch: Total branch outcomes (10)
• BrPart: Partially covered branches (3)
• Cover: Overall coverage percentage (82%)

To fix partial branches:

Add tests that trigger the opposite condition outcome.

Why other options are wrong:

• A: BrPart counts incomplete branches, not complete ones
• C: Partial coverage isn’t a reason to delete code
• D: This is a coverage metric, not an error report

Correct Answer: C

For a financial transaction system, high coverage is important because:

• Bugs can cause financial loss
• Edge cases in money handling are critical
• Regulatory requirements may mandate testing

Why 85-90% instead of 100%:

• 100% is often impractical (error handlers, debug code)
• 100% can lead to testing implementation details
• Focus should be on meaningful coverage of critical paths

Best practice for critical systems:

1. Set baseline threshold (85-90%)
2. Require 100% on critical modules (e.g., transactions.py)
3. Allow lower coverage on non-critical utilities
4. Enforce strong assertions, not just execution

Example CI configuration:

- name: Run tests with coverage
  run: pytest --cov=src --cov-fail-under=85 --cov-report=term-missing

Why other options are wrong:

• A (50%): Too low for financial systems
• B (70%): Industry standard for general apps, but financial needs more
• D (100%): Impractical and can lead to gaming the metric

Correct Answer: B

CI coverage integration serves as a quality gate that:

1. Measures coverage on every pull request
2. Fails the build if coverage drops below threshold
3. Prevents “coverage debt” from accumulating
4. Makes coverage visible to the team

How it works:

# In GitHub Actions workflow
- name: Run tests with coverage
  run: uv run pytest --cov=src --cov-fail-under=80

# If coverage < 80%, the workflow fails
# PR cannot be merged until coverage is restored

Benefits:

• Automatic enforcement: No manual checking needed
• Early feedback: Developers know immediately if they need more tests
• Trend protection: Coverage can’t silently decrease
• Team visibility: Everyone sees coverage in PR checks

What CI coverage does NOT do:

• Fix bugs automatically
• Speed up tests
• Replace code review (humans still needed)

Why other options are wrong:

• A: CI measures; it doesn’t fix
• C: Coverage checks add time, not reduce it
• D: Code review catches design issues that coverage can’t