Microservices architecture — AI service kahan baithegi

AI add karne ke liye poori architecture dobara likhne ki zaroorat nahi hai. Lekin do sawaal sach mein sochne layak hain: AI capability aapke microservices topology mein kahan baithegi, aur uske failure modes se baaki system ko kaise bachaya jaayega. Ye un patterns ka practical tour hai jo AI kaam se seedha takraate hain.

AI service alag kyun honi chahiye#

Ek normal order-service 30 millisecond mein jawab deti hai, uska failure rate lagbhag zero hai, aur har request ka cost bhi practically zero hai.

Ek model call ka profile poora ulta hai:

• Latency 2 se 20 second — normal se hazaar guna zyada.
• Failure aam baat hai — rate limit, timeout, provider outage.
• Cost har request par lagta hai, aur token ke hisaab se lagta hai.
• Throughput aapke haath mein nahi, provider ke quota mein hai.

In chaaron mein se koi bhi cheez aapki order-service par nahi lagti. Agar aap AI code ko usi service ke andar daal denge, to model ka slow hona us service ke threads kha jaayega aur order placement bhi girega — jiska AI se koi lena dena hi nahi tha.

Alag service ka matlab hai: alag scaling (concurrency par, CPU par nahi), alag circuit breaker, alag budget aur alag alerts. Aur sabse badi baat — model down hone par aapki core business chalti rahegi.

Topology — ek saada shakal#

                    ┌──────────────┐
   client  ───────▶ │ API Gateway  │
                    └──────┬───────┘
                           │
        ┌──────────────────┼──────────────────┐
        ▼                  ▼                  ▼
  order-service      catalog-service      ai-service
                                               │
                                               ├──▶ model provider (bahari)
                                               └──▶ vector store

Gateway par authentication, rate limiting aur routing hoti hai. ai-service ke paas apna vector store hai aur wahi ek jagah hai jahan se bahar model ko call jaati hai. Baaki services agar AI capability chahti hain to ai-service se maangti hain — model provider ko seedha koi call nahi karta.

Ye "sirf ek jagah se bahar jaata hai" wala rule bahut kaam ka hai. API key ek jagah, rate limiting ek jagah, cost tracking ek jagah, aur prompt ka version control bhi ek jagah.

Circuit breaker — AI mein ye optional nahi hai#

Circuit breaker ka idea ghar ke MCB jaisa hi hai. Jab dependency baar-baar fail kar rahi ho, to circuit "open" ho jaata hai aur aage ki calls turant fail ho jaati hain — bina 20 second wait kiye. Thodi der baad wo ek test call jaane deta hai; wo pass ho gayi to circuit wapas band.

Resilience4j ke saath Spring Boot mein:

@Service
public class AiAdvisorService {
 
    private final ChatClient chatClient;
 
    public AiAdvisorService(ChatClient chatClient) {
        this.chatClient = chatClient;
    }
 
    @CircuitBreaker(name = "modelProvider", fallbackMethod = "staticSuggestions")
    @TimeLimiter(name = "modelProvider")
    @Retry(name = "modelProvider")
    public CompletableFuture<List<String>> suggest(String query) {
        return CompletableFuture.supplyAsync(() ->
            chatClient.prompt().user(query).call().entity(new ParameterizedTypeReference<>() {}));
    }
 
    private CompletableFuture<List<String>> staticSuggestions(String query, Throwable cause) {
        return CompletableFuture.completedFuture(popularityRanker.topFor(query));
    }
}

Configuration:

resilience4j:
  circuitbreaker:
    instances:
      modelProvider:
        slidingWindowSize: 20
        failureRateThreshold: 50
        waitDurationInOpenState: 30s
        permittedNumberOfCallsInHalfOpenState: 3
  timelimiter:
    instances:
      modelProvider:
        timeoutDuration: 20s
  retry:
    instances:
      modelProvider:
        maxAttempts: 3
        waitDuration: 1s
        enableExponentialBackoff: true

Fallback ka matlab "error" nahi hota#

Circuit breaker ka asli fayda fallback mein hai, aur yahi wo jagah hai jo log sabse kam sochte hain. "Service unavailable" dikhana fallback nahi hai — wo sirf failure ko sundar shabdon mein likhna hai.

Achhe fallbacks aise dikhte hain:

Har case mein user ka kaam ho jaata hai. AI ne experience behtar banaya tha, wo experience ka poora aadhaar nahi tha — aur design mein bhi yahi dikhna chahiye.

Agent workflows aur saga pattern#

Ab thoda aage badhiye. Ek agent ek kaam nahi karta, wo kai kadam uthata hai:

1. Calendar mein slot dekha
2. Meeting book ki
3. Invite bheji
4. CRM update kiya

Ab step 4 fail ho gaya. Ab kya? Meeting book ho chuki hai, invite ja chuki hai, aur system aadhi halat mein hai. Ye ek distributed transaction jaisa hi problem hai — aur uska jawab saga pattern hai.

Saga kehta hai: har forward action ke saath uska compensating action pehle se likho.

public record AgentStep(String name, Runnable execute, Runnable compensate) {}
 
public void run(List<AgentStep> steps) {
    Deque<AgentStep> done = new ArrayDeque<>();
    try {
        for (AgentStep step : steps) {
            step.execute().run();
            done.push(step);
        }
    } catch (Exception failure) {
        // Ulte kram mein undo — jo aakhri hua wo pehle wapas.
        while (!done.isEmpty()) {
            done.pop().compensate().run();
        }
        throw failure;
    }
}

Agent design mein isse zyada kaam ki soch shaayad hi koi ho: agar aap kisi action ka undo nahi likh sakte, to agent ko wo action apne aap karne ki ijaazat mat dijiye. Us action ke liye insaan ki approval maangiye. Paisa transfer karna, email bhejna, record delete karna — ye sab isi category mein aate hain.

Aur bhi do cheezein jo aksar chhoot jaati hain#

Idempotency. Agent kabhi-kabhi wahi step dobara chala dega (retry, ya khud ka confusion). Har side-effect wale endpoint par ek idempotency key lijiye, taaki dobara call se duplicate booking na bane.

Har call par budget. Ek agent loop mein phas sakta hai. Per-request token cap aur maximum step count rakhiye, aur cross karte hi rok dijiye. Bina limit ke ek buggy agent raat bhar mein bada bill bana sakta hai.

Aage kya#

AI service ki jagah, uske around ki suraksha, aur multi-step kaam ka safe design — teenon clear hain. Ab agla topic hai streaming: jab jawab ek saath nahi, token-by-token aata hai, tab code ka shape kaisa hota hai.