CodeOath
← All posts
CI/CD & DevOps60 min total · 17 parts

CI/CD Pipelines Explained: From git push to Production

Contents — Part 15 of 17: Rollbacks
Part 15 of 17 · ~2 min

Rollbacks

The uncomfortable truth about deployments: eventually, one of them will be bad, and how fast you can undo it — not whether you can prevent every bad deploy — is what actually limits the damage. A pipeline that can deploy forward flawlessly but has no fast, tested way to redeploy the previous known-good version is missing half of what "CI/CD" is supposed to provide.

A few concrete mechanisms, often combined:

  • Redeploy the previous artifact. If the previous build's artifact is still available (see the build-artifacts section — this is another reason to keep them, not just to promote one build through environments), rolling back can be exactly as automated as rolling forward: run the same deploy job, pointed at the previous artifact instead of the new one.
  • Revert the commit, let the pipeline redeploy normally. Simpler to reason about, but slower — it goes through the full pipeline again rather than just re-pointing at something already built and verified.
  • Flip a feature flag off (previous section) — instant, but only helps if the specific problem is isolated behind a flag rather than baked into every request the new version handles.
  • Database migrations need their own rollback story, and it's usually the hardest part — a schema change that's already run against production data isn't always cleanly reversible (a dropped column's data is genuinely gone), which is exactly why backward-compatible, staged migrations (add the new column, deploy code that writes to both, backfill, only then remove the old column in a later, separate deploy) are the safer default over a single migration that changes everything at once.

The practical test of a rollback plan isn't whether it exists on paper — it's whether it's been exercised. A rollback procedure nobody has actually run since it was written is a real risk in exactly the moment (a live production incident) when it needs to work correctly on the first try.