Skip to content
JavaAgentic

Type at least two characters. Try “RAG”, “pgvector” or “tool calling”.

Guardrails & Safety Systems

Build guardrails around LLMs: input filtering, output validation against schemas and rules, content moderation, jailbreak defense and layered safety — deterministic controls in Java.

Advanced4 min readUpdated
On this page

A language model is a probabilistic component; guardrails are the deterministic code that makes it safe to deploy. They wrap the model call — checking input on the way in and output on the way out — so that however the model behaves, harmful results are caught. This tutorial builds a layered guardrail system.

Key Takeaways

  • Guardrails are deterministic code, not prompts — that is what makes them reliable.
  • Validate input (before the model) and output (before the user or an action).
  • Defense is layered: no single check is sufficient against adversarial input.
  • Prompts steer; code enforces. Rely on code for anything that must not escape.

The guardrail architecture

Guardrails wrap the model: filter input, validate output, and reject or fall back on failure.

Input filtering

Check input before spending a model call and before untrusted text reaches the prompt:

Input guardrails
public InputCheck checkInput(String input) {
    // Size bound — an oversized input is a cost and a potential DoS.
    if (input.length() > MAX_INPUT_CHARS) {
        return InputCheck.reject("input too long");
    }
    // Known malicious patterns (jailbreak signatures, injection markers).
    if (jailbreakDetector.looksLikeAttack(input)) {
        return InputCheck.flag("possible injection");   // flag, don't just pass
    }
    // Optional: a moderation model for disallowed content categories.
    if (moderation.isDisallowed(input)) {
        return InputCheck.reject("content policy");
    }
    return InputCheck.ok();
}

Output validation

The most important layer, because it catches problems regardless of how they arose — jailbreak, injection, or the model simply going wrong:

Output guardrails
public OutputCheck validateOutput(String output, Context context) {
    // 1. Schema/shape — use structured output, then verify.
    if (!matchesExpectedShape(output)) {
        return OutputCheck.retry("malformed output");
    }
    // 2. Scope — did it stay on-topic and in-policy?
    if (isOffScope(output, context)) {
        return OutputCheck.block("out of scope");
    }
    // 3. Content — no disallowed material, no leaked secrets.
    if (containsDisallowed(output) || leaksSecrets(output, context)) {
        return OutputCheck.block("policy violation");
    }
    // 4. Business rules — domain-specific constraints.
    if (!satisfiesBusinessRules(output, context)) {
        return OutputCheck.block("business rule");
    }
    return OutputCheck.ok();
}

Content moderation

For user-facing applications, a moderation layer screens both input and output for disallowed content categories:

// A moderation model (a dedicated classifier) checks content against policy
// categories. Cheaper and faster than the main model, run on both directions.
ModerationResult result = moderationModel.classify(text);
if (result.flagged()) {
    return handlePolicyViolation(result.categories());
}

This protects users from harmful output and protects you from generating it — both matter for a public product.

Jailbreak defense

Jailbreaks try to make the model ignore its instructions. Defend in layers, because any single layer is beatable:

  1. Input filtering — catch known jailbreak patterns.
  2. A firm system prompt — clear scope and refusal instructions steer the model. See prompt engineering.
  3. Output validation — catch violations regardless of how the input was crafted.
  4. Action gating — consequential actions authorized in code, immune to what the model says.

The key insight: you are not trying to make the model unjailbreakable (you cannot), but to make a successful jailbreak harmless — it produces output that your validation blocks and actions your code refuses.

Layered safety in practice

The full pipeline
public Response handleSafely(String input, Context context) {
    // Layer 1: input.
    InputCheck in = checkInput(input);
    if (in.rejected()) return Response.refused(in.reason());
 
    // Layer 2: the model, with a firm system prompt.
    String output = model.chat(SAFE_SYSTEM_PROMPT, input);
 
    // Layer 3: output validation.
    OutputCheck out = validateOutput(output, context);
    if (out.blocked()) return Response.refused(out.reason());
    if (out.needsRetry()) return handleSafely(input, context);   // bounded retries
 
    // Layer 4: actions (if any) are separately authorized in code.
    return Response.ok(output);
}

Balancing safety and usefulness

Guardrails that are too aggressive frustrate legitimate users — a support bot that refuses half of reasonable questions is useless, however safe. Tune the balance with your evaluation set: measure both the harmful-output rate (should be near zero) and the false-refusal rate (should be low), and adjust to keep both acceptable. Safety that destroys usefulness is its own kind of failure.

Next

Frequently Asked Questions

What are guardrails in an LLM application?
Deterministic checks around a model call — validating input before it reaches the model, and validating output before it reaches the user or triggers an action. Guardrails are ordinary code, not prompts, which is what makes them reliable: a probabilistic model wrapped in deterministic checks becomes safe enough to put in a production path.
How do I validate LLM output?
Check it against your requirements before acting on it: does it match the expected schema, stay within scope, contain no disallowed content, and satisfy business rules? Use structured output to constrain the shape, then validate the values in code. Never let raw model output trigger a consequential action without passing these checks.
What is a jailbreak and how do I defend against it?
A jailbreak is an input crafted to make the model ignore its instructions and produce restricted output. You defend with layers: input filtering to catch known patterns, a firm system prompt, output validation to catch violations regardless of how they arose, and gating consequential actions in code. No single layer is sufficient, which is why defense is layered.
Should guardrails be prompts or code?
Both, but rely on code. A system prompt asking the model to behave is a request it may not honor, especially under adversarial input. Deterministic code that validates output is enforcement. Use prompts to steer the model toward good behaviour, and code to guarantee that bad output does not escape or cause harm.

Related tutorials