03 Brownfield Exercises: AI-Assisted Test Coverage in Locust

March 2026 (2219 Words, 13 Minutes)

Overview

In this exercise you use AI assistance to write tests that improve code coverage in a real open-source project — Locust, a Python load testing framework. You work in teams of 3-5 students. Every team works on the same package (Package A — small, self-contained Python utilities) so you can compare approaches across teams afterwards.

The core rule: every AI prompt that produces code gets its own commit containing both the code changes and a diary entry. This forces you to work in small, documented increments — exactly how AI-assisted development works in practice.

If you already completed the Locust Exploration exercise, you may have steps 1-6 of the setup done. Skim them and pick up where you left off.

Crafting your prompts: You are expected to write your own prompts for the AI. Think back to Lecture 2 — From Copy-Paste to Crafted Prompts and apply what you learned about structured, context-rich prompting. Your diary should document the prompts you used and how you iterated on them.

Setup

1. Clone the repository — we work on a teaching copy in our course organization:

   git clone https://github.com/hs-aalen-software-engineering/locust.git
   cd locust
   

2. Install dependencies:

   uv sync
   

   If this fails, common causes are: uv not installed (docs.astral.sh/uv), missing Python 3.10+, or gevent build errors on Windows. Use your AI tool to debug.

   Heads up: even mature open-source projects sometimes have incomplete dependency files. If uv sync — or pytest later — complains about a missing module, you may have stumbled upon a real bug in pyproject.toml / uv.lock. Try uv add <module> to fix it locally and record the discovery in your diary. That is a legitimate contribution.

3. Activate pre-commit hooks — these run ruff (linter + formatter) on every commit:

   pip install pre-commit
   pre-commit install
   

4. Verify pytest works — run the Package A test file to confirm your setup:

   uv run pytest locust/test/test_util.py -x -q
   

   If this passes, your setup is good.

5. Create your feature branch:

   git checkout -b feature/<team-name>-util-coverage
   # Example: feature/team3-util-coverage
   

6. Create your diary folder:

   mkdir -p diary/feature/<team-name>-util-coverage
   

How You Work: Commits and Diary

The One-Prompt, One-Commit Rule

Every prompt that produces code gets its own commit containing both the code changes and the diary entry. Here is what one cycle looks like:

1. You write a prompt and give it to the AI.
2. The AI generates code. You evaluate it, edit it, run tests.
3. You write a diary entry (e.g., 002-coverage-memoize.md) recording the prompt, tool, model, and iterations.
4. You git add both the test file and the diary file.
5. You commit: git commit -m "[diary] 002 — Tests for util/cache.py memoize()"

This is the workflow for every task in this exercise. No exceptions.

Diary Entry Template

Each diary entry follows this format:

# NNN — Short Title

**Date**: YYYY-MM-DD
**Tool**: [Claude Code / GitHub Copilot / ...]
**Model**: [model name]
**Iterations**: [number of follow-up prompts]

## Prompt

**YYYY-MM-DD HH:MM**

[The full prompt text as given to the AI.]

Diary Folder Structure

diary/feature/<team-name>-util-coverage/
├── 001-onboarding.md
├── 002-coverage-memoize.md
├── 003-coverage-retry.md
└── 004-coverage-calc-distribution.md

Why This Matters

The diary IS the evidence of AI-assisted work. A PR with good tests but no diary entries gets zero credit for the “PR and diary quality” criterion (25% of grade). Process matters more than polish. A well-documented attempt with corrections and reflections is worth more than a perfect result with no evidence of how you got there.

Definition of Done

Every PR is complete when:

1. All existing tests pass (no regressions)
2. New tests check actual behavior (not just line coverage)
3. Code passes the linter — activate pre-commit hooks (pre-commit install) or use the ruff VS Code extension, or run uv run ruff check . && uv run ruff format --check manually
4. Commit messages are clear and meaningful
5. AI Diary entries record the full process

Package A Tutorial: util/cache.py — Step by Step

Work through this tutorial as a team. It walks through one file end-to-end so everyone sees the full workflow before you split up and do the remaining files on your own.

File focus: locust/util/cache.py — the memoize() decorator (currently only 10% line coverage).

Step 1: Measure Current Coverage

Run the coverage command — note the two --cov-report flags. The html report is the one that matters:

uv run pytest locust/test/test_util.py locust/test/test_log.py locust/test/test_taskratio.py \
  --cov=locust/util --cov=locust/log \
  --cov-report=term-missing --cov-report=html -q

The terminal table shows a Cover percentage and Missing line numbers per file. That is useful as a quick glance — but the HTML report is where you actually learn what is missing:

1. Open htmlcov/index.html in your browser (just double-click it, or start htmlcov/index.html on Windows).
2. Click locust/util/cache.py.
3. Uncovered lines are highlighted red. Partially-covered branches are yellow. Covered lines are green.

Keep this page open in a tab. You will come back to it after every test run.

Step 2: Read the Code You Are Testing

Open locust/util/cache.py. Look for the memoize() decorator. It is a pure-Python decorator with no external dependencies — you can understand it in 5 minutes.

What to pay attention to:

• It accepts a timeout argument (how long a cached value is considered fresh).
• It accepts a dynamic_timeout flag that doubles the timeout on each cache miss.
• The wrapped function has a clear_cache() method attached to it.
• The cache key is built from the function arguments.

You need to understand the code well enough to judge whether the AI’s tests are correct. If you skip this step and blindly trust the AI’s output, you will end up with tests that look plausible but test nothing meaningful.

Step 3: Write Tests for memoize()

Now use your AI tool. Write a prompt that includes:

• The decorator name and its location (memoize() in locust/util/cache.py).
• What the decorator does (caches the return value of a function for timeout seconds, optionally doubles the timeout on each miss, exposes a clear_cache() method).
• The specific cases you want tested:
  • Cache miss (first call executes the function)
  • Cache hit (second call with the same args returns the cached value without re-executing)
  • Timeout expiry (after timeout seconds the next call re-executes)
  • dynamic_timeout doubling (verify the timeout grows after each miss)
  • clear_cache() method on the wrapped function

Do not use a vague prompt. Here is an example of what NOT to do:

Generate tests to get 100% coverage on cache.py.

Why this is bad: optimizing for coverage percentage produces meaningless tests. The AI will generate tests that call the function with trivial inputs and assert True — technically covering lines but verifying nothing. A test that does not catch a real bug is worthless, no matter how many lines it “covers.”

Instead, be specific about what behavior each test should verify. The AI produces better code when your prompt describes the what and why, not just “make the number go up.”

Step 4: Evaluate the AI Output

Before accepting any generated test, ask yourself three questions:

1. Does it test a specific behavior? A test that calls cached_fn() and asserts the return value is not None tests nothing useful.
2. Would this test fail if a real bug were introduced? If someone broke the timeout logic so cache entries never expired, would this test catch it?
3. Are the assertions checking the right thing? The assertions should verify the content of the return value or the side effect (e.g., that the underlying function was called the expected number of times), not just the type.

Delete any test you cannot explain. Better to have 3 meaningful tests than 8 superficial ones.

Step 5: Run Tests and Check Coverage

First, run the fast test command to verify nothing is broken:

uv run pytest locust/test/test_util.py -x -q

Then re-run the coverage command from Step 1:

uv run pytest locust/test/test_util.py locust/test/test_log.py locust/test/test_taskratio.py \
  --cov=locust/util --cov=locust/log \
  --cov-report=term-missing --cov-report=html -q

Refresh htmlcov/index.html in your browser and click cache.py again. The previously-red lines should now be green. If some are still red, the HTML report tells you exactly which ones — go back to Step 3 and add tests for those cases.

Step 6: Commit with Diary Entry

Write your diary entry — for example, diary/feature/<team-name>-util-coverage/002-coverage-memoize.md — using the template from the diary section above. Record the full prompt you used, the tool, the model, and how many iterations it took.

Then commit both files together:

git add locust/test/test_util.py diary/feature/<team-name>-util-coverage/002-coverage-memoize.md
git commit -m "[diary] 002 — Tests for util/cache.py memoize()"

Both the test file AND the diary file are in the same commit. This is the rule.

Step 7: Now Do the Remaining Files on Your Own

The tutorial above showed you the pattern. Apply Steps 2-6 yourself for each of the following files. Each file gets its own diary entry and its own commit.

File 2: locust/util/exception_handler.py — retry() decorator

Current coverage: 54%. Only the happy path is tested — the retry and failure paths are uncovered.

Write tests for:

• Success on first try (no retry needed)
• Success on retry (fails once, succeeds on second attempt)
• Failure after all retries exhausted
• Custom exception filtering

Target: 54% → near 100%.

File 3: locust/user/inspectuser.py — _calc_distribution()

Current coverage: 60%. Pure computation with simple User class stubs.

Write tests for:

• Equal task weights
• Mixed weights (some tasks weighted higher than others)
• fixed_count users
• Zero users edge case

Target: 60% → ~80%.

Use the HTML coverage report after each round to confirm the previously-red lines turn green.

Step 8: Create Your Pull Request

Push your branch and create a PR against master:

git push -u origin feature/<team-name>-util-coverage

Your PR description should include:

• What you changed and why
• Coverage numbers before and after, per file — for example:

## Coverage

| File | Before | After |
|------|--------|-------|
| util/cache.py | 10% | 98% |
| util/exception_handler.py | 54% | 96% |
| user/inspectuser.py | 60% | 81% |

• A link to the HTML coverage report screenshot (optional but useful).

Estimated total time: 60-75 minutes with AI assistance.

Deliverables

Submit one PR against master per team containing:

• New test files with coverage improvements
• Coverage numbers (before/after per file) in the PR description
• Diary entries in diary/feature/<team-name>-util-coverage/ documenting each AI interaction
• A PR description explaining what you changed and why

Rubric

Quick Reference

Fast Test Command

uv run pytest locust/test/test_util.py locust/test/test_log.py locust/test/test_taskratio.py -x -q

Runtime: ~8 seconds.

Coverage Command (with HTML report)

uv run pytest locust/test/test_util.py locust/test/test_log.py locust/test/test_taskratio.py \
  --cov=locust/util --cov=locust/log \
  --cov-report=term-missing --cov-report=html -q

Then open htmlcov/index.html.

Note on Pre-Existing Test Failures

A few tests in the full suite fail due to environment differences (SSL certs, config file paths, Windows-specific issues). None of these are in the Package A test files. If you encounter a failure in test_log.py:test_greenlet_exception_logger, this is a known pre-existing issue — ignore it.

Tips

• Run your test command after every change. The fast feedback loop (~8 seconds) is your biggest advantage.
• Keep htmlcov/index.html open in a tab. Refresh after every test run — it is the fastest way to see what is still missing.
• Read existing tests first. The patterns in test_util.py are your best guide for how new tests should look.
• Coverage is a means, not an end. A test that calls a function and asserts True increases coverage but tests nothing. Write tests that would catch a real bug.
• When the AI generates tests, evaluate each one. Ask: “If someone introduced a bug in this function, would this test catch it?” If the answer is no, the test is worthless.
• The diary IS the learning artifact. Process matters more than polish. A well-documented attempt with corrections and reflections is worth more than a perfect result with no evidence of how you got there.