Service Mesh with Istio

A service mesh moves cross-cutting network concerns out of your application and into a proxy beside it. The application makes a plain HTTP call; the proxy adds mTLS, retries, timeouts, load balancing, telemetry and policy — in every language, without a library.

The architecture#

mTLS with two lines#

apiVersion: security.istio.io/v1
kind: PeerAuthentication
metadata:
  name: default
  namespace: production
spec:
  # Plaintext connections between meshed workloads are rejected outright.
  mtls:
    mode: STRICT

That is the whole configuration for mutual TLS across every service in the namespace. Certificates are issued per workload identity, rotated automatically every 24 hours, and never touched by application code. Doing the equivalent by hand — provisioning, distributing and rotating certificates for every service — is the kind of project that takes a quarter and is never quite finished.

Migrate with PERMISSIVE mode first, which accepts both plaintext and mTLS, then switch to STRICT once every workload is meshed.

Authorization#

apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: payment-access
  namespace: production
spec:
  selector:
    matchLabels: { app: payment-service }
  action: ALLOW
  rules:
    - from:
        - source:
            # Cryptographic identity, not an IP range or a network segment.
            principals: ['cluster.local/ns/production/sa/order-service']
      to:
        - operation:
            methods: ['POST']
            paths: ['/v2/charges']
      when:
        - key: request.auth.claims[scope]
          values: ['payments.write']

This is zero-trust made concrete. A compromised analytics pod cannot call the payment endpoint no matter what network access it has, because it cannot present the order service's certificate.

Note the default: with no AuthorizationPolicy, everything is allowed. Add a deny-all policy in the namespace first, then allow specific paths — otherwise you have written documentation rather than enforcement.

Traffic management#

apiVersion: networking.istio.io/v1
kind: VirtualService
metadata: { name: order-service }
spec:
  hosts: ['order-service']
  http:
    # Header-based routing first: internal testers get v2 deterministically.
    - match:
        - headers:
            x-canary: { exact: 'true' }
      route:
        - destination: { host: order-service, subset: v2 }
    # Everyone else is split by weight.
    - route:
        - destination: { host: order-service, subset: v1 }
          weight: 90
        - destination: { host: order-service, subset: v2 }
          weight: 10
      timeout: 3s
      retries:
        attempts: 2
        perTryTimeout: 1s
        retryOn: '5xx,reset,connect-failure'
---
apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata: { name: order-service }
spec:
  host: order-service
  trafficPolicy:
    connectionPool:
      tcp: { maxConnections: 100 }
      http: { http2MaxRequests: 1000, maxRequestsPerConnection: 10 }
    outlierDetection:
      consecutive5xxErrors: 5
      interval: 10s
      baseEjectionTime: 30s
      # Never eject more than half the pool, or a widespread issue
      # removes every instance and guarantees total failure.
      maxEjectionPercent: 50
  subsets:
    - name: v1
      labels: { version: v1 }
    - name: v2
      labels: { version: v2 }

outlierDetection is the mesh's circuit breaker, and it works at a different granularity from Resilience4j: it ejects individual instances that are misbehaving, rather than stopping calls to the service as a whole. One bad pod is removed from load balancing while the rest keep serving — something an in-process library cannot do because it does not know which instance it reached.

maxEjectionPercent is the safety valve. Without it, a bug affecting every instance ejects every instance, and a degraded service becomes a completely unavailable one.

Fault injection#

spec:
  http:
    - fault:
        delay:
          percentage: { value: 10 }
          fixedDelay: 5s
        abort:
          percentage: { value: 5 }
          httpStatus: 503
      route:
        - destination: { host: payment-service }

Inject latency and errors without changing any code, then verify your callers actually degrade the way they were designed to. This is the cheapest chaos engineering available — and running it in staging before an incident is considerably more pleasant than discovering during one that a fallback was never wired up.

Progressive delivery#

Flagger automates canary analysis on top of the mesh:

apiVersion: flagger.app/v1beta1
kind: Canary
metadata: { name: order-service }
spec:
  targetRef: { apiVersion: apps/v1, kind: Deployment, name: order-service }
  service: { port: 80 }
  analysis:
    interval: 1m
    threshold: 5
    maxWeight: 50
    stepWeight: 10
    metrics:
      - name: request-success-rate
        thresholdRange: { min: 99 }
        interval: 1m
      - name: request-duration
        thresholdRange: { max: 500 }
        interval: 1m

Traffic shifts 10% at a time; at each step Flagger queries the mesh's own metrics and rolls back automatically if success rate or latency breaches the threshold. Nobody has to watch a dashboard during a deploy, which is the actual value — automated rollback happens at 3am too.

Whether to adopt one#

The honest assessment: a mesh solves problems you only have at a certain scale, and adds problems immediately.

It is worth it when you need mTLS everywhere for a compliance requirement, when you have enough services that per-service resilience configuration has visibly drifted, when you run a polyglot estate where a Java library helps only some of it, or when you want progressive delivery with automatic rollback.

It is not worth it for five Java services in one cluster. Library-based resilience plus TLS at the ingress is simpler, easier to debug, and has no control plane to upgrade. Adopting a mesh prematurely mostly buys you a new category of incident where the application and the proxy disagree about what should have happened.

If you do adopt one, budget for the operational learning: reading Envoy access logs, understanding why a request was rejected by a policy, and upgrading the control plane are all skills a team has to acquire.

What to take away#

A mesh gives you automatic mTLS, identity-based authorization, per-instance outlier ejection and traffic shifting without touching application code. That is genuinely valuable at scale and genuinely overkill below it. Adopt it for a specific problem you have, not because the architecture diagram looks more modern with one.