Repeatable

Tests should produce the same results every time

“If tests are flaky, developers will eventually ignore them.”

Part of the F.I.R.S.T. principles.

When to use

Always.

Why it matters

  • Trust: If a test fails “sometimes”, developers stop trusting the test suite (“Oh, that’s just the flaky test, ignore it”).

Signs of Violation

  • Tests that fail depending on the time of day.
  • Tests that fail on the CI server but pass locally.
  • Tests using Math.random() without a seed.

Explanation

Problem

Non-deterministic inputs (Time, Randomness, Network).

Solution

Control the environment. Mock the clock (2023-01-01). Seed the random number generator. Mock the network response.

Real world analogy

Scientific Experiment. If you mix Chemical A and B, it should explode every time, not just on Tuesdays. If it’s not repeatable, it’s not science.

Pros and Cons

Pros Cons
  • Reliability
    		•
  • Requires mocking environment variables/time
    		•

    Comparison

    • Flaky Tests: The enemy of Repeatability.

    Code example

    Typescript

    Bad (Violation)

    Good (Adherence)

    PHP

    Bad (Violation)

    Good (Adherence)