I have seen systems with hundreds of passing tests, a successful build, a healthy deployment, and no honest answer to a simple question:
Can we safely put thousands of people on this today?
The problem is not that the tests are useless. The problem is that a green pipeline answers a much smaller question than people often think it does.
It usually tells us that, in a prepared environment, the code compiled and the selected checks passed. Production readiness includes code, but it also includes state, capacity, recovery, external providers, operational ownership, and the complete journeys people depend on.
Those are different forms of evidence.
What the green check actually proves
A good continuous integration pipeline can prove useful things:
- The repository can be installed from a clean checkout.
- Static analysis found no known violations within its rules.
- Unit and integration tests passed against their configured dependencies.
- A build artifact can be produced.
- Known migrations are syntactically valid.
- The change did not break the cases the team remembered to encode.
That is valuable. I do not want less of it.
But the pipeline does not automatically prove that production has the same schema, secrets, image, provider configuration, or network path. It does not prove that a backup can be restored. It does not prove that an alert reaches a person. It does not prove that a user can finish the journey that makes the product useful.
A green check is evidence. It is not a verdict.
I separate readiness into layers
When I review a system for a meaningful launch, I now separate the evidence into five layers.
1. Source readiness
Does the code model the important rules correctly? Are authentication, authorization, idempotency, validation, and error handling present? Are important operations atomic?
This is where tests and review are strongest.
2. Artifact readiness
Can I identify the exact source revision, container image, dependency lock, and migration set that will run? Can I prove the artifact being tested is the artifact being deployed?
"The main branch is green" is not enough when production is running an image built from another commit.
3. Runtime readiness
Are the services healthy with the real configuration shape? Are workers, schedulers, queues, caches, databases, and reverse proxies all participating? Can the system tolerate a process restart without losing or duplicating work?
A homepage returning 200 does not answer those questions.
4. Journey readiness
Can each important user role complete its critical path?
For commerce, that may mean discovery, checkout, payment confirmation, fulfilment, refunds, and notifications. For a financial product, it may mean consent, data refresh, analysis, clear limitations, and safe handoff to an authorised execution system.
The check should follow the journey, not merely the route list.
5. Operational readiness
Who notices failure? Who responds? Which dashboards, alerts, backups, runbooks, and rollback paths have actually been exercised?
If an alert rule exists but cannot reach anybody, it is configuration, not an operational control.
Capacity is not a feeling
One of the easiest readiness claims to make is also one of the hardest to prove: "It can handle ten thousand users."
A route count, database table count, test count, or healthy CPU graph does not establish capacity. The useful questions are more specific:
- Ten thousand registered users or ten thousand concurrent sessions?
- What traffic shape and peak duration?
- Which endpoints dominate database work?
- Which providers impose their own rate limits?
- What happens to queues during a downstream outage?
- How long does recovery take after the backlog begins draining?
Capacity needs a workload model, production-shaped tests, observations, and limits. Without those, the number is a target, not evidence.
Recovery must be demonstrated
Backups often appear in readiness documents as a checkbox. I prefer a harsher question: when was the last successful restore?
A backup file can exist and still be incomplete, corrupt, encrypted with a lost key, or incompatible with the current application. A restore drill turns a hopeful artifact into evidence.
The drill should record:
- The backup selected.
- Its integrity checks.
- The clean environment used for restoration.
- The time required.
- The amount of data lost relative to the failure point.
- The application checks performed against the restored state.
Until that happens, recovery remains an assumption.
Providers create a second system boundary
Many applications are only useful because another system participates: a payment provider, identity service, messaging network, storage service, or data supplier.
Mocks prove our interpretation of the provider contract. They do not prove the live credentials, callback URLs, account entitlements, webhook signatures, settlement states, or failure behaviour.
Provider journeys need controlled live verification. That does not mean using real customers as test probes. It means maintaining dedicated test identities, bounded amounts where money is involved, and an explicit cleanup process.
My preferred launch decision
I do not like a readiness report that ends with a long list and no decision. I want one of three outcomes:
- Go: the required evidence exists and the remaining risks have explicit owners.
- Conditional go: a narrow release is acceptable within stated limits and rollback conditions.
- No-go: one or more critical claims remain unproven or unsafe.
Unknown is not the same as failed, but it is also not passed.
That distinction prevents teams from converting the absence of bad news into evidence of safety.
The useful role of CI
CI should remain strict. It should catch regressions quickly, produce reproducible artifacts, and make the known rules difficult to violate.
It becomes dangerous only when the organisation asks it to certify things it never observed.
The green check means the code passed the checks we gave it. Production readiness begins when we list everything those checks could not see and collect evidence for the rest.