Skip to content
JavaAgentic

Type at least two characters. Try “RAG”, “pgvector” or “tool calling”.

GenAI on AWS, Azure & GCP

Run generative AI on the major clouds from Java: Amazon Bedrock, Azure OpenAI and Google Vertex AI compared, with Spring AI and LangChain4j integration and how to choose.

Intermediate4 min readUpdated
On this page

Running generative AI on a major cloud gives you enterprise governance — data residency, private networking, unified identity and billing — that direct provider APIs do not. This tutorial compares Amazon Bedrock, Azure OpenAI and Google Vertex AI, and shows the Java integration, which is reassuringly uniform.

Key Takeaways

  • Cloud AI services add enterprise governance on top of foundation models.
  • Bedrock (multi-provider), Azure OpenAI (OpenAI models), Vertex AI (Google + others).
  • Default to the cloud you already run on — keep AI beside your data and identity.
  • Integration is a config change; the ChatModel abstraction keeps your code portable.

Why go through a cloud

Calling a provider API directly is simplest. Cloud AI services exist for the governance enterprises need:

  • Data residency — keep prompts and responses in a chosen region.
  • Private networking — no traffic over the public internet.
  • Unified identity — the cloud's IAM, not a separate API key to manage.
  • Consolidated billing and agreements — one vendor relationship, enterprise terms.
  • Compliance — the cloud's certifications extend to the AI service.

If none of these matter to you, a direct API is fine. If you are in an enterprise with data-governance requirements, they are often decisive. See AI regulations & compliance.

The three services

ServiceModelsBest for
Amazon BedrockAnthropic, Meta, Mistral, Amazon, othersAWS teams wanting model choice in one API
Azure OpenAIOpenAI models (GPT family)Azure/Microsoft-stack enterprises
Google Vertex AIGemini, plus othersGCP teams, strong multimodal

Amazon Bedrock

One API across many providers, within your AWS environment. Model choice without integrating each vendor separately.

Bedrock with Spring AI
// spring-ai-starter-model-bedrock-converse
// Uses your AWS credentials chain — no separate API key.
spring:
  ai:
    bedrock:
      aws:
        region: us-east-1
      converse:
        chat:
          options:
            model: anthropic.claude-sonnet-4-v1:0

Azure OpenAI

OpenAI models with Azure's enterprise wrapper — the same GPT models, governed by Azure.

spring:
  ai:
    azure:
      openai:
        api-key: ${AZURE_OPENAI_KEY}
        endpoint: ${AZURE_OPENAI_ENDPOINT}
        chat:
          options:
            deployment-name: gpt-4o-mini

Google Vertex AI

Gemini and other models on GCP, with strong multimodal support.

spring:
  ai:
    vertex:
      ai:
        gemini:
          project-id: ${GCP_PROJECT}
          location: us-central1
          chat:
            options:
              model: gemini-2.0-flash

Choosing a cloud

The decision is usually made for you by where you already run:

Default to the cloud you already run on — AI belongs beside your data and identity.

Cost and quotas

Cloud AI services bill per token like direct APIs, sometimes with different pricing and enterprise discounts. They also have quotas and rate limits you must request increases for ahead of a launch — default quotas are often lower than you expect, and hitting them in production is a scramble. Plan capacity and request quota increases early. Track cost per feature as always; see Spring AI observability.

Deployment considerations

  • Credentials — use the cloud's IAM roles (instance profiles, workload identity) rather than static keys where possible. See Docker & Kubernetes.
  • Regions — pick regions for data residency and latency; not every model is available in every region.
  • Private endpoints — route through private networking for sensitive workloads.
  • Fallback — a model or region outage still happens; keep a circuit breaker and consider a fallback provider.

Next

You have completed Phase 4 — the model-side knowledge that makes you an AI engineer, not just an integrator.

Frequently Asked Questions

What is the difference between using OpenAI directly and Azure OpenAI?
Azure OpenAI serves OpenAI models through Microsoft Azure, giving you Azure's enterprise controls — private networking, regional data residency, Azure identity and billing, and enterprise agreements. The models are the same; the difference is the governance, compliance and integration wrapper. Enterprises often prefer it for exactly those controls.
What is Amazon Bedrock?
A managed AWS service that provides access to multiple foundation models — from Anthropic, Meta, Mistral, Amazon and others — through one API, with AWS's security, networking and billing. It suits teams already on AWS who want model choice without integrating each provider separately, and want their data to stay within their AWS environment.
Which cloud should I use for generative AI?
Usually the one you already run on, because keeping AI beside your data and identity infrastructure avoids cross-cloud complexity, data egress and a separate security review. Bedrock for AWS, Azure OpenAI for Azure, Vertex AI for GCP. Only choose against your existing cloud if it lacks a specific model or capability you genuinely need.
Do the cloud AI services work with Spring AI and LangChain4j?
Yes. Both frameworks have modules for Amazon Bedrock, Azure OpenAI and Google Vertex AI. Because your code depends on the ChatModel abstraction, using a cloud provider is a dependency and configuration change, and your application logic is unchanged — the same portability that applies to direct provider APIs.

Related tutorials