Microservices Security Patterns

Perimeter security assumes an inside and an outside. A microservices deployment has neither in any meaningful sense — dozens of workloads talking over a network where one compromise should not become all of them.

Layers#

The double validation is not redundancy. The gateway's job is to reject obviously bad traffic cheaply so it never consumes downstream capacity. The service's job is to be correct regardless of how the request arrived — through the gateway, from another service, or from something that reached the pod network directly.

Token propagation#

@Bean
RestClient.Builder serviceClient() {
    return RestClient.builder().requestInterceptor((request, body, execution) -> {
        // Forward the caller's token so the downstream service can make its own
        // authorisation decision about this specific user.
        var auth = SecurityContextHolder.getContext().getAuthentication();
        if (auth instanceof JwtAuthenticationToken jwt) {
            request.getHeaders().setBearerAuth(jwt.getToken().getTokenValue());
        }
        request.getHeaders().add("X-Correlation-Id",
                Objects.toString(MDC.get("correlationId"), ""));
        return execution.execute(request, body);
    });
}

Forwarding the user's token keeps the user's identity — and therefore their permissions — accurate all the way down. The alternative pattern, where each service calls the next with its own service credentials, loses the user entirely and means the payment service cannot check that this user owns this order.

Two things to watch. Token lifetime must exceed the longest call chain, or a valid request fails mid-way. And the token now travels to more places, so each service must validate the audience — a token intended for the order service should not be accepted by a service it was not issued for.

Where a chain is long or crosses trust boundaries, token exchange (RFC 8693) is cleaner: a service exchanges the user's token for a narrower one scoped to the next hop, so a compromised downstream cannot replay a broadly-scoped credential.

Workload identity and mTLS#

apiVersion: security.istio.io/v1
kind: PeerAuthentication
metadata: { name: default, namespace: production }
spec:
  mtls: { mode: STRICT }        # plaintext connections are rejected
---
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata: { name: payment-access, namespace: production }
spec:
  selector:
    matchLabels: { app: payment-service }
  action: ALLOW
  rules:
    - from:
        - source:
            # Only the order service may call payments — by cryptographic
            # identity, not by IP or network segment.
            principals: ['cluster.local/ns/production/sa/order-service']
      to:
        - operation:
            methods: ['POST']
            paths: ['/v2/charges']

This is the practical form of "never trust the network". The order service proves who it is with a certificate rotated automatically by the mesh, and the policy names that identity. A compromised analytics pod cannot call the payment endpoint no matter what network access it has.

Without a mesh, SPIFFE/SPIRE provides the same thing directly: each workload is attested by node and workload selectors and issued a short-lived SVID identifying it as spiffe://acme.com/ns/production/sa/order-service.

Service credentials#

spring:
  security:
    oauth2:
      client:
        registration:
          inventory-api:
            client-id: ${INVENTORY_CLIENT_ID}
            client-secret: ${INVENTORY_CLIENT_SECRET}
            authorization-grant-type: client_credentials
            # Narrow. This service reads stock; it cannot write orders.
            scope: 'inventory.read'

The scoping is the point. A service compromised through a dependency vulnerability can do exactly what its credentials permit and nothing more. One shared client with broad scopes turns any single compromise into full estate access.

API keys for external consumers#

Treat an API key like a password, because that is what it is:

@Service
public class ApiKeyService {
 
    private final PasswordEncoder encoder;
 
    public IssuedKey issue(String tenantId, Set<String> scopes) {
        String raw = "ak_" + Base64.getUrlEncoder().withoutPadding()
                .encodeToString(secureRandomBytes(32));
 
        // Store the HASH. A database leak must not yield usable keys, and it
        // means we can never show the key again after issuance.
        repository.save(new ApiKey(
                tenantId, encoder.encode(raw), prefixOf(raw), scopes,
                Instant.now(), Instant.now().plus(Duration.ofDays(365))));
 
        return new IssuedKey(raw, "shown once — store it now");
    }
}

Store a short non-secret prefix alongside the hash so you can look up a key by prefix rather than hashing against every row. Give keys an expiry and a rotation window where both old and new work, or rotation becomes an outage and nobody does it.

Secrets#

Kubernetes Secret objects are base64-encoded, which is an encoding, not encryption. Anyone with read access to the namespace — or to an etcd backup — can decode them.

Three improvements, in increasing order of rigour. Enable etcd encryption at rest so a disk or backup leak does not expose secrets. Use the External Secrets Operator to sync from Vault or a cloud secret manager, so the source of truth has rotation and audit. Best of all, use dynamic secrets: Vault issues a database credential with a short lease, valid only for this workload, and revokes it automatically — a leaked credential is worthless within the hour.

Whatever the mechanism, no secret belongs in a container image, an environment variable baked into a manifest in git, or a log line.

What to take away#

Validate at the edge and again in every service. Forward verifiable tokens rather than trusting headers. Give each workload a cryptographic identity with mTLS and authorise by that identity, not by network position. Scope every credential narrowly, hash API keys, and move secrets to a manager that can rotate them.