Skip to main content
Engineering/pinning-tests

Pinning Tests

You need characterization tests on legacy behavior first.

Use this when you need to modify existing code that has no tests. Pinning tests capture the current behavior — even if buggy — so you can make changes safely without accidentally breaking things.

Process

Step 1: Gather inputs

Ask the user to provide:

  1. Code to pin — the function, class, module, or file that needs pinning tests. Accept pasted code or file paths.
  2. Tech stack — language and test framework (e.g., "JavaScript, Mocha" or "Python, pytest").
  3. What you plan to change — brief description of the upcoming modification, so tests can focus on the areas most at risk of unintended breakage.

Step 2: Check for existing acceptance criteria

Before analyzing the code, check if acceptance criteria exist for the code being modified:

  1. AC exists: Pin against the AC. Each Gherkin scenario should have a pinning test that verifies the current behavior matches (or diverges from) the specified AC.
  2. AC doesn't exist: Pin observed behavior. Document that no AC was found -- the pinning tests capture what the code does, not what it was supposed to do.
  3. AC partially exists: Pin against AC where available, pin observed behavior for the rest. Flag the gap.

Related skills: Use /test-coverage-audit to find which code paths have AC coverage before pinning.

Document the AC status in the test file:

// AC STATUS: [Has AC / No AC / Partial AC]
// Source: [story link or "no AC found"]

Step 3: Analyze the code

Before writing tests:

  1. Identify all public functions, methods, and entry points
  2. Trace the main execution paths (happy path, error paths, branches)
  3. Note any side effects (database writes, API calls, file operations, state mutations)
  4. Identify input types and boundary values
  5. Note any behavior that looks buggy -- pin it anyway, but flag it

Step 4: Write pinning tests

For each public function or entry point, write tests that:

  1. Capture current behavior — what the code does today, whether correct or not
  2. Cover main paths — the primary success case and the most obvious failure cases
  3. Hit branches — if there are conditionals, test both sides
  4. Test edge cases — null/undefined inputs, empty collections, boundary values
  5. Will break if behavior changes — the whole point is detecting unintended changes

Label every test clearly:

PINNING: (description of current behavior being captured)

Rules:

  • Do not fix bugs — if the code returns the wrong value, the pinning test asserts the wrong value. That's correct.
  • Do not improve the code — no refactoring, no cleanup, no optimization.
  • Do not add new functionality — only capture what exists.
  • Mock external dependencies — if the code calls APIs, databases, or file systems, mock them to capture the code's logic in isolation.

Step 5: Verify

If running in Claude Code with shell access:

  1. Run the pinning tests — they should all pass against the current code
  2. If any test fails, the test is wrong (it's not capturing actual behavior). Fix the test, not the code.

Step 6: Present results

Output:

  • The complete test file with all pinning tests
  • A coverage summary: which functions/methods are pinned, which aren't
  • A list of flagged behaviors: "This looks like a bug, but the pinning test captures current behavior. Consider fixing after your changes."

Ask the user:

  • Are there specific paths or behaviors you want pinned more thoroughly?
  • Ready to make your changes with this safety net in place?

Output location

Write test files directly if running in Claude Code with file access. Place them alongside the code being pinned, following the project's test file conventions. Otherwise, present in the conversation for the user to copy.

Example Output

Input

  • Code to pin: A Python utility function calculate_cart_total(items, promo_code=None) from checkout/pricing.py at Verdant Market, an e-commerce grocery platform. The function applies discounts, taxes, and promo codes to a list of cart items.
  • Tech stack: Python, pytest
  • What you plan to change: Refactoring the promo code logic to support stackable discounts — currently only one promo code applies at a time.

Output (abbreviated)

Pinning Tests: calculate_cart_total

AC STATUS: No AC found Source: No AC found — pinning tests capture observed behavior, not intended behavior.


Code Analysis

Entry point: calculate_cart_total(items, promo_code=None)

Execution paths identified:

  • Happy path: valid items list, no promo code
  • Happy path: valid items list with valid promo code
  • Branch: promo code "SAVE10" → 10% discount applied before tax
  • Branch: promo code "FREESHIP" → sets shipping to 0, does not affect subtotal
  • Branch: unknown promo code → silently ignored (⚠️ flagged)
  • Edge: empty items list
  • Edge: item with quantity=0
  • Edge: None passed as items

Side effects: None — pure function, no I/O.

Flagged behaviors:

⚠️ Unknown promo codes are silently ignored with no error or warning. This may be a bug — users could mistype a promo code and receive no discount with no feedback. Pinning test captures this behavior as-is.

⚠️ An item with quantity=0 is included in the total calculation as $0.00 but still triggers the tax calculation pass. Likely unintentional. Pinning test captures this behavior as-is.


Pinning Tests

# checkout/tests/test_pricing_pinned.py
#
# AC STATUS: No AC
# Source: no AC found — these tests capture observed behavior as of 2024-06-11.
# Do NOT update these tests to reflect desired behavior.
# Update them only if you intentionally change behavior during refactoring.

import pytest
from checkout.pricing import calculate_cart_total

# ---------------------------------------------------------------------------
# Fixtures
# ---------------------------------------------------------------------------

def make_item(name, price, quantity):
    return {"name": name, "price": price, "quantity": quantity}


BASIC_CART = [
    make_item("Organic Oats", 4.99, 2),
    make_item("Almond Milk", 3.49, 1),
]
# Subtotal: (4.99 * 2) + (3.49 * 1) = 13.47
# Tax (8.5% observed): 1.14
# Shipping (flat $5.99 observed): 5.99
# Expected total: 20.60


# ---------------------------------------------------------------------------
# Happy path — no promo code
# ---------------------------------------------------------------------------

class TestNoPromoCode:

    def test_PINNING_returns_correct_total_for_basic_cart(self):
        # PINNING: basic cart with no promo returns subtotal + 8.5% tax + $5.99 shipping
        result = calculate_cart_total(BASIC_CART)
        assert result["total"] == 20.60

    def test_PINNING_subtotal_excludes_tax_and_shipping(self):
        # PINNING: subtotal field reflects item prices only
        result = calculate_cart_total(BASIC_CART)
        assert result["subtotal"] == 13.47

    def test_PINNING_tax_is_8_5_percent_of_subtotal(self):
        # PINNING: tax rate observed to be 8.5%
        result = calculate_cart_total(BASIC_CART)
        assert result["tax"] == pytest.approx(1.14, abs=0.01)

    def test_PINNING_shipping_is_flat_5_99(self):
        # PINNING: shipping is a flat $5.99 regardless of cart size
        result = calculate_cart_total(BASIC_CART)
        assert result["shipping"] == 5.99


# ---------------------------------------------------------------------------
# Promo code: SAVE10
# ---------------------------------------------------------------------------

class TestPromoCodeSave10:

    def test_PINNING_SAVE10_applies_10_percent_discount_before_tax(self):
        # PINNING: SAVE10 reduces subtotal by 10% before tax is calculated
        result = calculate_cart_total(BASIC_CART, promo_code="SAVE10")
        assert result["subtotal"] == pytest.approx(12.12, abs=0.01)

    def test_PINNING_SAVE10_discount_field_is_populated(self):
        # PINNING: discount field reflects the dollar amount removed
        result = calculate_cart_total(BASIC_CART, promo_code="SAVE10")
        assert result["discount"] == pytest.approx(1.35, abs=0.01)

    def test_PINNING_SAVE10_does_not_affect_shipping(self):
        # PINNING: SAVE10 promo has no effect on shipping cost
        result = calculate_cart_total(BASIC_CART, promo_code="SAVE10")
        assert result["shipping"] == 5.99


# ---------------------------------------------------------------------------
# Promo code: FREESHIP
# ---------------------------------------------------------------------------

class TestPromoCodeFreeShip:

    def test_PINNING_FREESHIP_sets_shipping_to_zero(self):
        # PINNING: FREESHIP promo zeroes out shipping field
        result = calculate_cart_total(BASIC_CART, promo_code="FREESHIP")
        assert result["shipping"] == 0.00

    def test_PINNING_FREESHIP_does_not_change_subtotal(self):
        # PINNING: FREESHIP does not affect subtotal or discount field
        result = calculate_cart_total(BASIC_CART, promo_code="FREESHIP")
        assert result["subtotal"] == 13.47
        assert result["discount"] == 0.00


# ---------------------------------------------------------------------------
# ⚠️ Flagged: unknown promo code silently ignored
# ---------------------------------------------------------------------------

class TestUnknownPromoCode:

    def test_PINNING_unknown_promo_code_is_silently_ignored(self):
        # PINNING: unrecognized promo code applies no discount and raises no error
        # ⚠️ FLAGGED: This is likely a bug — user receives no feedback on bad promo code.
        #    After your refactor, consider raising a ValueError or returning a warning field.
        result = calculate_cart_total(BASIC_CART, promo_code="INVALIDXYZ")
        assert result["total"] == 20.60
        assert result["discount"] == 0.00


# ---------------------------------------------------------------------------
# Edge cases
# ---------------------------------------------------------------------------

class TestEdgeCases:

    def test_PINNING_empty_cart_returns_zero_subtotal(self):
        # PINNING: empty list produces $0.00 subtotal and $0.00 tax
        result = calculate_cart_total([])
        assert result["subtotal"] == 0.00
        assert result["tax"] == 0.00

    def test_PINNING_empty_cart_still_charges_shipping(self):
        # PINNING: shipping is applied even on an empty cart
        # ⚠️ FLAGGED: Charging $5.99 shipping on an empty cart is almost certainly a bug.
        result = calculate_cart_total([])
        assert result["shipping"] == 5.99

    def test_PINNING_zero_quantity_item_contributes_zero_to_subtotal(self):
        # PINNING: item with quantity=0 adds $0.00 to subtotal
        cart = [make_item("Ghost Item", 9.99, 0)]
        result = calculate_cart_total(cart)
        assert result["subtotal"] == 0.00

    def test_PINNING_none_items_raises_type_error(self):
        # PINNING: passing None as items raises TypeError (not a custom exception)
        with pytest.raises(TypeError):
            calculate