AI & Java Glossary

38 terms defined for people who write Java for a living — no linear algebra assumed, no marketing language.

Core AI

Chain-of-thought#: Prompting the model to work through intermediate reasoning steps before answering. It improves accuracy on multi-step problems at the cost of latency and tokens.
Context window#: The maximum number of tokens a model can consider at once, covering the system prompt, conversation history, retrieved documents and the response. Exceeding it causes truncation or an API error.
Few-shot prompting#: Including a handful of worked input/output examples in the prompt so the model infers the pattern. Usually more effective than describing the desired format in prose, especially for classification and structured extraction.
Fine-tuning#: Continuing to train a pre-trained model on your own examples so it adapts to a task, format or tone. It teaches behaviour, not facts — for facts, use retrieval. LoRA and QLoRA make it affordable by training small adapter matrices instead of the full model.
Hallucination#: Fluent, confident output that is factually wrong. It is a consequence of how language models generate text, not a bug to be patched — it is mitigated with retrieval grounding, citations, output validation and human review, never eliminated.
Structured output#: Constraining a model to return JSON matching a schema so it can be deserialised directly into a Java record. Both Spring AI and LangChain4j do this by deriving the schema from your type and validating the response.
System prompt#: Instructions given to the model with higher priority than user input, defining role, tone, constraints and output format. In Spring AI it is set with ChatClient.prompt().system(...) or a defaultSystem on the builder.
Temperature#: A sampling parameter controlling randomness. Near 0 the model picks the most likely token every time, producing repeatable output — the right setting for extraction, classification and tool calling. Higher values increase variety, useful for brainstorming and copywriting.
Token#: The unit a language model actually reads and bills for — roughly three-quarters of an English word. Code and non-Latin scripts tokenize less efficiently, which is why a 500-line Java file can cost more tokens than you expect.

Retrieval

Chunking#: Splitting a document into smaller passages before embedding it. Chunk size trades recall against precision: large chunks retrieve more context but dilute the signal, small chunks are precise but may lose the surrounding meaning.
Embedding#: A fixed-length vector of floating-point numbers representing the meaning of a piece of text. Texts with similar meaning produce vectors that are close together, which is what makes semantic search possible.
GraphRAG#: Retrieval over a knowledge graph rather than a flat set of chunks. Because it can traverse relationships, it answers multi-hop questions that vector search alone cannot connect.
HNSW#: Hierarchical Navigable Small World — the graph index most vector databases use for approximate nearest-neighbour search. It trades a small amount of recall for a very large speed-up over exhaustive comparison.
Hybrid search#: Combining keyword (BM25) and vector search, then fusing the rankings. It recovers exact-match cases — product codes, error numbers, rare acronyms — that pure semantic search misses.
RAG (Retrieval-Augmented Generation)#: A pattern that retrieves relevant documents from a knowledge base and injects them into the prompt so the model answers from your data rather than from memory. It is the standard fix for hallucination and stale training data, and needs no model retraining.
Re-ranking#: A second retrieval pass in which a cross-encoder model scores each candidate passage against the query directly. Slower than vector similarity but far more accurate, so it is typically applied to the top 20-50 results only.
Vector store#: A database optimised for approximate nearest-neighbour search over embeddings. Examples include pgvector, Qdrant, Milvus, Weaviate and Redis. Spring AI and LangChain4j both abstract over them so the store can be swapped without changing application code.

Agents

Agent memory#: Everything an agent carries between steps or sessions: the working context of the current task, an episodic log of past interactions, and semantic memory of learned facts, typically recalled through vector search.
Agentic AI#: A system in which a language model decides which actions to take, executes them through tools, observes the results and repeats until a goal is met. The distinguishing feature is control flow: in generative AI your code decides what happens next, in agentic AI the model does. Read the tutorial →
Human-in-the-loop (HITL)#: A design in which an agent pauses for human approval before consequential actions — sending an email, merging a pull request, issuing a refund. The standard way to deploy autonomy without accepting unbounded risk.
Multi-agent system#: An architecture where several specialised agents collaborate, usually coordinated by an orchestrator that routes work and merges results. Useful when tasks need genuinely different tools or system prompts; costly when a single agent would do.
ReAct#: An agent loop that interleaves Reasoning and Acting: the model writes a thought, chooses a tool, sees the observation, and repeats. It is the default architecture behind most tool-using agents, including LangChain4j AI Services with @Tool methods.
Tool calling#: A model capability where, instead of answering in prose, the model emits a structured request to invoke a named function with typed arguments. Your application executes the function and returns the result, which the model uses to continue. Also called function calling.

Java & Spring

Advisor#: A Spring AI interceptor that wraps a model call to add behaviour — injecting retrieved documents, appending chat memory, logging or redacting. Conceptually the same idea as a servlet filter or a Spring AOP advice.
AiServices#: The LangChain4j facility that turns a plain Java interface into an LLM-backed implementation: annotations declare the prompt, the return type drives output parsing, and @Tool methods become callable tools.
ChatClient#: Spring AI's fluent API for talking to a model: chatClient.prompt().system(...).user(...).call().content(). It handles prompt assembly, advisors, tool dispatch and output conversion.
LangChain4j#: A framework-neutral Java library for LLM applications, with a wide integration surface and a declarative AiServices style where you define an interface and the library implements it against a model. Read the tutorial →
pgvector#: A PostgreSQL extension adding a vector column type with similarity operators and index support. For teams already running Postgres it removes the need for a separate vector database at small and medium scale.
Server-Sent Events (SSE)#: A one-way streaming protocol over plain HTTP, and the standard way to stream tokens from a Spring Boot endpoint to a browser. Simpler than WebSockets when only the server needs to push.
Spring AI#: A Spring project providing portable abstractions over chat models, embedding models, vector stores and tool calling, with Boot auto-configuration, Micrometer observability and the usual Spring testing support. Read the tutorial →
Virtual threads#: Lightweight threads introduced in Java 21 that make blocking I/O cheap. They suit LLM workloads well, where a request spends almost all its time waiting on a remote model rather than using CPU.

Operations

Evaluation set#: A fixed collection of inputs with known-good outputs, run on every prompt or model change. Without one you cannot tell whether an edit improved the system or merely changed it.
Guardrails#: Deterministic checks around a model call: validating input, constraining output against a schema, filtering unsafe content and enforcing business rules. Guardrails are ordinary code — they are what makes a probabilistic component safe to put in a production path.
LLMOps#: The operational practice around LLM features: versioning prompts like code, evaluating changes against a regression set, tracing calls, tracking token cost, and rolling out model upgrades behind canaries.
Prompt injection#: An attack where instructions hidden in untrusted content — a web page, a PDF, an email — are read by the model and followed as if they came from the operator. It is the top entry in the OWASP LLM Top 10 and cannot be fully solved by prompt wording alone.
Quantization#: Storing model weights at lower numeric precision (8-bit or 4-bit instead of 16-bit) to shrink memory use and speed up inference, with a small accuracy cost. GGUF, GPTQ and AWQ are common formats.
Semantic cache#: A cache keyed by embedding similarity rather than exact string match, so "how do I reset my password" hits the entry stored for "password reset steps". Cuts cost and latency on repetitive traffic, at the risk of returning a near-miss answer.
Time to first token (TTFT)#: How long a model takes to emit its first token. For interactive features it matters more than total generation time, because streaming lets the user start reading immediately.