Skip to content
JavaAgentic

Type at least two characters. Try “RAG”, “pgvector” or “tool calling”.

Foundation Models Deep Dive

Understand the foundation models you build on: GPT, Claude, Gemini, Llama and Mistral families, how they differ, and a practical framework for choosing a model for your Java application.

Intermediate4 min readUpdated
On this page

You build on foundation models the way you build on a database engine — you rarely make one, but choosing and understanding the one you use matters. This tutorial surveys the major families, how they differ, and a practical framework for choosing that does not rely on leaderboards.

Key Takeaways

  • Foundation models are the broad, adaptable base you build on via prompting and retrieval, not retraining.
  • The families differ in strengths, context length, latency and price — not in a single ranking.
  • Test on your own task. Benchmarks are a rough filter; your inputs are the real benchmark.
  • Because your code uses an abstraction, evaluating several models is cheap — do it.

The major families

FamilyProviderTypically strong atAccess
GPTOpenAIGeneral reasoning, tool use, ecosystemHosted API
ClaudeAnthropicLong context, code, careful instruction-followingHosted API
GeminiGoogleMultimodal, long context, Google Cloud integrationHosted API
LlamaMetaOpen weights, self-hostable, customisableOpen weight
MistralMistral AIEfficient open-weight models, strong for sizeOpen + hosted

These shift constantly — new versions ship monthly and the relative strengths move. Treat any specific claim about "the best model" as perishable, and re-evaluate periodically.

Hosted vs open-weight

The first branch in choosing a model is whether you call an API or run the model yourself.

Hosted (GPT, Claude, Gemini): highest capability, no infrastructure, pay per call. The default for most applications, and where the frontier lives.

Open-weight (Llama, Mistral, and others): you run it, so you get privacy, no per-call fee, and customisation — at the cost of operating inference infrastructure and a generally lower capability ceiling. See Spring AI with Ollama and small language models.

What actually differs between models

Beyond the marketing, the dimensions that affect your application:

  • Reasoning depth — how well it handles multi-step problems. The clearest differentiator on hard tasks.
  • Context length — how much it can consider at once, from tens of thousands to millions of tokens. Matters for long documents and large retrieval sets. See tokenization and context windows.
  • Instruction following — how reliably it obeys complex, constraint-heavy prompts.
  • Tool use — how well it selects and calls tools, which matters enormously for agents.
  • Latency — time to first token and total generation time, which shapes user experience.
  • Price — per-token cost, which at scale is a real budget line.
  • Multimodality — whether it handles images, audio, and how well.

A framework for choosing

Do not choose from a leaderboard. Choose like this:

Evaluate candidates on your task
// Because your code depends on ChatModel, swapping models for evaluation is a
// one-line change. Run your real task through each and compare on the axes you
// care about — not on a general benchmark.
List<ChatModel> candidates = List.of(gpt4oMini, claudeSonnet, gemini);
 
for (ChatModel model : candidates) {
    EvalResult result = evaluate(model, yourRealTaskSet);
    log.info("model={} accuracy={} p95LatencyMs={} costPer1k={}",
            model, result.accuracy(), result.p95Latency(), result.costPer1000());
}

Model versions and deprecation

Providers version and deprecate models. A model you built on can change behaviour with a new version, or be retired. Protect yourself:

  • Pin model versions explicitly rather than using a floating "latest" alias, so behaviour does not shift under you.
  • Keep a regression eval so you can test a new version before adopting it. See LLMOps for generative AI.
  • Design for portability — depend on the ChatModel abstraction, not a provider SDK, so a deprecation is a config change, not a rewrite.

Understanding what is under the hood

You do not need to build a transformer to use one well, but understanding the architecture explains a lot of model behaviour — why context length is limited, why tokens cost what they do, why models hallucinate. That is the next tutorial.

Next

Frequently Asked Questions

What is a foundation model?
A large model trained on broad data that can be adapted to many tasks — the base you build applications on rather than training from scratch. GPT, Claude, Gemini, Llama and Mistral are foundation model families. You typically use them through an API or run open-weight versions yourself, adapting behaviour through prompting and retrieval rather than retraining.
How do I choose between GPT, Claude, and Gemini?
Test them on your own task rather than trusting leaderboards. They have different strengths — some are stronger at code, some at long-context reasoning, some at following complex instructions — and pricing and latency differ. Because your code depends on an abstraction like ChatModel, evaluating several on your real inputs is cheap, and that evaluation beats any general ranking.
Should I use open-weight models like Llama or Mistral?
Consider them when you need to run models yourself for privacy, cost at high volume, or customisation. Open-weight models you can host give you control and no per-call fee, at the cost of operating inference infrastructure and generally lower ceiling than the frontier hosted models. Many teams use hosted models for hard tasks and open-weight ones for high-volume simple tasks.
Do model benchmarks predict real-world performance?
Only loosely. Benchmarks measure specific capabilities on specific datasets, which may not match your task, and models can be tuned to score well on popular benchmarks. Treat them as a rough filter, then evaluate the shortlist on your own representative inputs. Your task is the only benchmark that matters for your decision.

Related tutorials