Kubernetes Deployment Strategies

Every deployment strategy is a different answer to one question: how much do you pay, in resources or complexity, to reduce the blast radius of a bad release?

Rolling update#

spec:
  replicas: 6
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 2          # up to 2 extra pods during the roll
      maxUnavailable: 0    # never drop below 6 ready
  minReadySeconds: 15      # a pod must stay ready this long before it counts
  progressDeadlineSeconds: 600
  revisionHistoryLimit: 5

maxUnavailable: 0 is the setting that matters. The default of 25% means a quarter of your capacity disappears during every deploy, which at peak traffic is how a routine release becomes an incident.

minReadySeconds guards against a pod that passes its readiness probe and then immediately crashes. Without it, Kubernetes counts it as available, moves on to the next pod, and cheerfully rolls a broken version across the fleet.

None of these strategies is genuinely zero-downtime without graceful shutdown, and this is the piece most often missing. When a pod is deleted, Kubernetes sends SIGTERM and removes the endpoint from the Service concurrently — the two are not ordered — so requests keep arriving for a second or two after the application has started shutting down. Set server.shutdown=graceful with a spring.lifecycle.timeout-per-shutdown-phase longer than your slowest request, and add a preStop hook that sleeps five seconds so endpoint removal has propagated before the JVM stops accepting work.

kubectl rollout status deployment/order-service --timeout=10m
kubectl rollout undo deployment/order-service            # back one revision
kubectl rollout history deployment/order-service

Rollback works because the old ReplicaSet is retained. revisionHistoryLimit controls how many, and setting it to zero — which people do to reduce clutter — removes your ability to roll back at all.

Blue-green#

kubectl apply -f green-deployment.yaml
kubectl rollout status deployment/order-service-green
./smoke-test.sh https://order-service-green.internal    # verify before any traffic
 
kubectl patch service order-service \
  -p '{"spec":{"selector":{"app":"order-service","version":"green"}}}'
 
# If anything looks wrong:
kubectl patch service order-service \
  -p '{"spec":{"selector":{"app":"order-service","version":"blue"}}}'

The strength is that rollback is instantaneous and complete — no waiting for pods to roll, no partial state. The cost is double resources during the window, and the switch is all-or-nothing: every user moves at once, so a bug affects everyone immediately rather than a small percentage.

Canary with automated analysis#

apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata: { name: order-service }
spec:
  replicas: 10
  strategy:
    canary:
      canaryService: order-service-canary
      stableService: order-service-stable
      trafficRouting:
        istio:
          virtualService: { name: order-service }
      steps:
        - setWeight: 5
        - pause: { duration: 5m }
        - analysis:
            templates: [{ templateName: success-rate }]
        - setWeight: 25
        - pause: { duration: 10m }
        - analysis:
            templates: [{ templateName: success-rate }, { templateName: latency }]
        - setWeight: 50
        - pause: { duration: 10m }
        - setWeight: 100
---
apiVersion: argoproj.io/v1alpha1
kind: AnalysisTemplate
metadata: { name: success-rate }
spec:
  metrics:
    - name: success-rate
      interval: 1m
      count: 5
      # Three consecutive failures abort the rollout and revert traffic.
      failureLimit: 3
      successCondition: result[0] >= 0.99
      provider:
        prometheus:
          address: http://prometheus:9090
          query: |
            sum(rate(http_server_requests_seconds_count{
              app="order-service", version="canary", status!~"5.."}[2m]))
            / sum(rate(http_server_requests_seconds_count{
              app="order-service", version="canary"}[2m]))

The analysis step is what makes canary genuinely valuable rather than just slower. Without it, a canary is a rolling update where someone is supposed to watch a dashboard — and at 3am nobody is. With it, the rollout queries Prometheus at each step and reverts automatically when the new version looks worse.

Note the canary must be separately labelled in metrics, or you are comparing the new version against an average that includes itself.

GitOps#

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata: { name: order-service, namespace: argocd }
spec:
  project: production
  source:
    repoURL: https://github.com/acme/k8s-manifests
    targetRevision: main
    path: overlays/production/order-service
  destination:
    server: https://kubernetes.default.svc
    namespace: production
  syncPolicy:
    automated:
      prune: true       # delete resources removed from Git
      selfHeal: true    # revert manual cluster changes
    syncOptions: [CreateNamespace=true]
    retry:
      limit: 5
      backoff: { duration: 5s, factor: 2, maxDuration: 3m }

The properties that make GitOps worth adopting: the cluster state is auditable through Git history, reproducible by pointing a new cluster at the same repo, and drift-resistant because selfHeal reverts a manual kubectl edit rather than leaving it as an undocumented difference.

The change that surprises teams is that CI no longer deploys. The pipeline builds an image and commits a new tag to the manifest repo; Argo CD notices and applies it. That means CI needs no cluster credentials at all, which is a meaningful security improvement.

Database migrations#

Every strategy above runs two versions simultaneously, so schema changes must work with both. Expand-and-contract, over several releases:

ALTER TABLE orders ADD COLUMN total_minor_units BIGINT;
-- Nullable. Old code ignores it, new code populates it.

UPDATE orders SET total_minor_units = ROUND(total * 100) WHERE total_minor_units IS NULL;

ALTER TABLE orders DROP COLUMN total;

A migration that renames or drops a column in the same release as the code change will break the old pods that are still running. This is the most common cause of errors during an otherwise correct zero-downtime deploy.

What to take away#

Default to rolling update with maxUnavailable: 0 and minReadySeconds. Reach for blue-green when instant rollback justifies double resources, and for canary with automated analysis when a bad release is expensive. Adopt GitOps so the cluster converges on Git and drift self-corrects — and keep every schema change backwards compatible, because two versions always run at once.