Docker aur Kubernetes — Java AI app ko deploy karna

Spring Boot AI application deploy karna 90% waisa hi hai jaise koi bhi Spring Boot app deploy karna — AI wala kaam to kisi aur ke hardware par ho raha hai. Jo teen cheezein sach mein alag hain, wo hain: API key ko safely rakhna, health probes jo slow model dependency ko samajh sakein, aur ek aisi service ke liye sahi resource sizing jo zyadatar sirf wait karti hai.

Dockerfile — pehle sahi tareeka seekh lijiye#

Sabse aam galti ye hai:

FROM eclipse-temurin:21-jre
COPY target/app.jar app.jar
ENTRYPOINT ["java", "-jar", "app.jar"]

Chalega, lekin har chhote code change par poora 60 MB ka fat JAR dobara push hota hai, kyunki wo ek hi layer hai. Spring Boot ka layered JAR isi ke liye bana hai — dependencies ek layer, aapka code doosri layer:

# --- build ---
FROM eclipse-temurin:21-jdk AS build
WORKDIR /workspace
COPY . .
RUN ./mvnw -q -DskipTests package && \
    java -Djarmode=layertools -jar target/*.jar extract --destination extracted
 
# --- run ---
FROM eclipse-temurin:21-jre AS runtime
WORKDIR /app
COPY --from=build /workspace/extracted/dependencies/ ./
COPY --from=build /workspace/extracted/spring-boot-loader/ ./
COPY --from=build /workspace/extracted/snapshot-dependencies/ ./
COPY --from=build /workspace/extracted/application/ ./
 
RUN useradd --create-home --shell /bin/false spring
USER spring
 
ENV JAVA_TOOL_OPTIONS="-XX:MaxRAMPercentage=75 -XX:+UseSerialGC"
EXPOSE 8080
ENTRYPOINT ["java", "org.springframework.boot.loader.launch.JarLauncher"]

Yahan teen cheezein dhyaan dene layak hain:

1. Do stages. JDK sirf build ke liye, final image mein sirf JRE. Image aadhi ho jaati hai.
2. Layers alag. Dependencies wali layer tab tak cached rehti hai jab tak pom.xml na badle. Roz ke code change par sirf chhoti application layer push hoti hai.
3. Root nahi. Container ke andar root banke chalna bina wajah ka risk hai.

MaxRAMPercentage isliye hai kyunki JVM ko container ki memory limit ka pata hona chahiye — warna wo host ki poori RAM dekhkar heap decide karega aur pod OOMKilled hoga.

Kubernetes — API key Secret se aayegi#

Pehle Secret banaiye:

kubectl create secret generic model-credentials \
  --from-literal=openai-api-key="$OPENAI_API_KEY"

Phir usse pod mein environment variable ki tarah mount kijiye:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: ai-service
spec:
  replicas: 2
  selector:
    matchLabels:
      app: ai-service
  template:
    metadata:
      labels:
        app: ai-service
    spec:
      containers:
        - name: ai-service
          image: registry.example.com/ai-service:1.4.0
          ports:
            - containerPort: 8080
          env:
            - name: OPENAI_API_KEY
              valueFrom:
                secretKeyRef:
                  name: model-credentials
                  key: openai-api-key
          resources:
            requests:
              cpu: "250m"
              memory: "768Mi"
            limits:
              memory: "1Gi"

Application side par bas itna:

spring:
  ai:
    openai:
      api-key: ${OPENAI_API_KEY}

Dhyaan dijiye ki limits mein CPU nahi hai. CPU limit lagane se Kubernetes throttle karta hai, aur ek I/O-bound service ke liye wo aksar latency badha deta hai bina kuch bachaye. Memory limit zaroori hai, CPU limit aksar nahi.

Health probes — yahan sabse zyada galtiyan hoti hain#

Kubernetes do sawaal poochhta hai. Liveness: "kya ye pod zinda hai, ya isse maar ke naya banaun?" Readiness: "kya isko traffic bhej sakta hoon?"

Do alag sawaal hain, aur inka jawab bhi alag hona chahiye:

          livenessProbe:
            httpGet:
              path: /actuator/health/liveness
              port: 8080
            initialDelaySeconds: 30
            periodSeconds: 15
          readinessProbe:
            httpGet:
              path: /actuator/health/readiness
              port: 8080
            initialDelaySeconds: 10
            periodSeconds: 5

Ab asli baat: liveness probe kabhi bhi model provider ko check nahi karna chahiye. Socho kya hoga — OpenAI 10 minute ke liye down hua, aapka health check red hua, Kubernetes ne saare pods maar diye, naye pods bhi red aaye, aur ab aap crash-loop mein hain. Provider wapas aa bhi jaaye to aapki service khud bhi tooti padi hai. Model ka down hona aapke pod ka bimaar hona nahi hai.

Spring Boot mein ye endpoints on karne ke liye:

management:
  endpoint:
    health:
      probes:
        enabled: true
      show-details: never
  endpoints:
    web:
      exposure:
        include: health,info,prometheus

Agar aap chahte hain ki model ki halat kahin dikhe, to uske liye alag custom health indicator banaiye aur use readiness ya liveness group ke bahar rakhiye — dashboard ke liye, restart ke liye nahi.

Scaling — CPU par mat kijiye#

Ye wo jagah hai jahan AI service normal service se sach mein alag behave karti hai.

Ek normal API par load badhe to CPU chadhta hai, aur CPU-based autoscaling kaam kar jaati hai. AI service par load badhta hai to CPU 8% par baitha rehta hai kyunki saare threads bas model ka jawab wait kar rahe hain. CPU par scale karenge to autoscaler kabhi trigger hi nahi hoga, aur users timeout khaate rahenge.

Scale kijiye concurrency par — kitni requests ek saath chal rahi hain. Isko KEDA ya custom metrics se karte hain, aur base par HPA memory par:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: ai-service
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: ai-service
  minReplicas: 2
  maxReplicas: 10
  metrics:
    - type: Pods
      pods:
        metric:
          name: http_server_active_requests
        target:
          type: AverageValue
          averageValue: "20"

Aur ek baar Java 21 ke virtual threads on kar dijiye (spring.threads.virtual.enabled=true) — ek hi pod bahut zyada concurrent model calls sambhal lega, aur aapko utne replicas ki zaroorat hi nahi padegi.

Aage kya#

Application container mein hai, key safe hai, probes samajhdaar hain aur scaling sahi signal par hai. Ab agla sawaal architecture ka hai — AI capability poore microservices setup mein kahan baithegi aur uske failures se baaki system ko kaise bachaya jaaye.