More Tests, More Confidence? Test Suites Are Investment Portfolios

Wait 5 sec.

Every test suite starts as an asset. The first hundred tests are almost universally useful. They cover the system's core behaviors, and they catch regressions. They give the team justified confidence that the software is behaving as intended. The team is proud of them. The CI pipeline is fast. The signal-to-noise ratio is high.Then the suite grows. It grows because the system grows and the coverage targets demand it. It grows because new engineers arrive and write tests in new styles, because incidents trigger the addition of regression tests that reproduce specific bugs, because automated test generation tools produce tests at scale. By the time the suite reaches several thousand tests, the team's relationship with it has changed in ways they may not have fully realized. The CI pipeline takes forty minutes. Flaky tests fail intermittently and are ignored. Large portions of the suite were written for features that have since been refactored or removed. Yet nobody is sure which portions, so nothing is deleted. The suite still passes, technically, but the team has quietly learned not to trust it.