DevAgentic — Autonomous DevOps Agent
An agent that investigates incidents, analyses logs, reviews pull requests and proposes remediation — with every write action behind a human approval gate. The project where autonomy meets consequences.
Stack at a glance
- Backend
- Spring Boot 3.3+Spring AILangChain4jJava 21
- AI
- Claude Sonnet (code analysis)GPT-4o-mini (routing)Ollama
- Data
- PostgreSQLpgvectorElasticsearchRedis
- Ops
- KubernetesPrometheusLokiKafkaOpenTelemetry
On this page
AgenticHR was an assistant. This is an agent — it decides its own investigation path, and some of its tools touch production. That difference is the entire lesson.
Architecture
Module 1 — The investigation agent
An alert fires. The agent gathers evidence and produces a hypothesis. It cannot change anything.
package com.javaagentic.devagentic.investigate;
import java.time.Duration;
import java.time.Instant;
import org.springframework.ai.tool.annotation.Tool;
import org.springframework.ai.tool.annotation.ToolParam;
import org.springframework.stereotype.Component;
/**
* Read-only by construction. Nothing in this class mutates state, which means
* the investigation agent can be given a generous step budget without risk.
*/
@Component
public class IncidentInvestigationTools {
private final LogSearchClient logs;
private final MetricsClient metrics;
private final DeploymentHistory deployments;
private final KubernetesClient kubernetes;
// ... constructor omitted
@Tool(description = """
Search application logs for a service within a time window.
Returns up to 50 matching lines, newest first, with timestamps.
Use a specific query — searching for "error" alone returns noise.
""")
public String searchLogs(
@ToolParam(description = "Service name, e.g. checkout-api") String service,
@ToolParam(description = "Search query, e.g. 'connection refused'") String query,
@ToolParam(description = "How far back to search, in minutes (max 1440)") int minutes) {
// Bound the window. An unbounded log query against a busy cluster is a
// denial of service against your own observability stack.
int windowMinutes = Math.min(Math.max(minutes, 1), 1440);
var results = logs.search(service, query, Duration.ofMinutes(windowMinutes), 50);
return results.isEmpty()
? "NO_MATCHES: no log lines matched in the last %d minutes.".formatted(windowMinutes)
: results.formatted();
}
@Tool(description = """
Get recent deployments for a service. Use this early — a change in
deployment history shortly before an incident is the most common
root cause by a wide margin.
""")
public String recentDeployments(
@ToolParam(description = "Service name") String service,
@ToolParam(description = "How many hours of history (max 168)") int hours) {
return deployments.since(service, Instant.now().minus(
Duration.ofHours(Math.min(Math.max(hours, 1), 168)))).formatted();
}
@Tool(description = """
Get pod status, restart counts and recent events for a service in
Kubernetes. Returns CrashLoopBackOff, OOMKilled and similar states.
""")
public String podHealth(@ToolParam(description = "Service name") String service) {
return kubernetes.describePods(service).formatted();
}
@Tool(description = """
Query a Prometheus metric over a time window. Returns a summary with
min, max, mean and the time of the peak.
""")
public String queryMetric(
@ToolParam(description = "PromQL query") String promql,
@ToolParam(description = "Window in minutes (max 1440)") int minutes) {
return metrics.rangeSummary(promql, Duration.ofMinutes(Math.min(minutes, 1440)));
}
}The agent itself
package com.javaagentic.devagentic.investigate;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.prompt.ChatOptions;
import org.springframework.stereotype.Service;
@Service
public class InvestigationAgent {
private static final String SYSTEM = """
You are a site reliability engineer investigating a production incident.
Method:
1. Check recent deployments first. Most incidents follow a change.
2. Then pod health, then logs, then metrics.
3. Form a hypothesis only when the evidence supports it.
4. Stop as soon as you can explain the incident. Do not keep digging
for completeness.
Output format:
SUMMARY: one sentence.
EVIDENCE: the specific tool results that support your conclusion.
HYPOTHESIS: the most likely cause, with your confidence (high/medium/low).
RECOMMENDED ACTION: what a human should do next.
Rules:
- Never claim to have checked something you did not check.
- If the evidence is inconclusive, say so and state what you would
need to look at next. An honest "I don't know" is more useful than
a confident wrong answer during an incident.
- You have read-only access. You cannot change anything, and you must
not imply that you have.
""";
private final ChatClient chatClient;
private final TrajectoryRecorder trajectory;
public InvestigationAgent(ChatClient.Builder builder,
IncidentInvestigationTools tools,
TrajectoryRecorder trajectory) {
this.chatClient = builder
.defaultSystem(SYSTEM)
.defaultTools(tools)
.defaultAdvisors(trajectory.advisor())
.build();
this.trajectory = trajectory;
}
public Investigation investigate(Alert alert) {
String runId = trajectory.beginRun(alert.id());
String report = chatClient.prompt()
.user(u -> u.text("""
Alert: {name}
Service: {service}
Fired at: {firedAt}
Description: {description}
Investigate and report.
""")
.param("name", alert.name())
.param("service", alert.service())
.param("firedAt", alert.firedAt().toString())
.param("description", alert.description()))
// A strong model for reasoning over messy evidence. This is the
// one place in the system where model quality genuinely shows.
.options(ChatOptions.builder().temperature(0.2).build())
.call()
.content();
return new Investigation(runId, alert.id(), report, trajectory.stepsFor(runId));
}
}Module 2 — Budgets and the trajectory log
Two pieces of infrastructure that everything else depends on.
package com.javaagentic.devagentic.guard;
/**
* A hard stop on a single agent run.
*
* Agents fail in a specific way: they retry a failing tool with slight
* variations, indefinitely, each attempt re-sending the whole conversation.
* Cost grows quadratically with trajectory length. A budget converts an
* unbounded incident into a bounded one.
*/
public class RunBudget {
private final int maxSteps;
private final long maxTokens;
private int steps;
private long tokens;
public RunBudget(int maxSteps, long maxTokens) {
this.maxSteps = maxSteps;
this.maxTokens = maxTokens;
}
public synchronized void recordStep(long tokensUsed) {
steps++;
tokens += tokensUsed;
if (steps > maxSteps) {
throw new BudgetExceededException(
"step limit reached (%d). The agent did not converge.".formatted(maxSteps));
}
if (tokens > maxTokens) {
throw new BudgetExceededException(
"token budget reached (%d).".formatted(maxTokens));
}
}
}/**
* Records every prompt, tool call and observation for a run.
*
* This is not optional infrastructure. When an agent reaches a wrong
* conclusion, the trajectory is the only artefact that explains why — the
* final answer tells you nothing about which of twelve tool calls returned
* misleading data.
*/
@Component
public class TrajectoryRecorder {
private final TrajectoryRepository repository;
public String beginRun(String correlationId) {
return repository.create(correlationId).id();
}
public void recordStep(String runId, String tool, String args, String result, long durationMs) {
repository.append(runId, new Step(tool, args, truncate(result, 4000), durationMs));
}
/** Results can be enormous; store a bounded prefix rather than blowing up the row. */
private String truncate(String value, int max) {
return value.length() <= max ? value : value.substring(0, max) + "…[truncated]";
}
}Module 3 — The pull request review agent
The most immediately useful module, and low risk: its only write action is posting a comment.
package com.javaagentic.devagentic.review;
import java.util.List;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.prompt.ChatOptions;
import org.springframework.stereotype.Service;
@Service
public class PullRequestReviewer {
public record Finding(
String file,
int line,
Severity severity,
String issue,
String suggestion) {
public enum Severity { BLOCKER, MAJOR, MINOR, NITPICK }
}
public record Review(String summary, List<Finding> findings) {}
private static final String SYSTEM = """
You review Java pull requests.
Report only issues you can point at a specific line for:
- Correctness bugs: null handling, off-by-one, wrong boundary, resource leaks
- Concurrency: shared mutable state, missing synchronisation
- Security: injection, missing authorisation, secrets in code
- Spring-specific: missing @Transactional, incorrect bean scope,
blocking calls on reactive threads
Do not report:
- Formatting or style (a formatter handles that)
- Preferences with no defect behind them
- Speculation about code you cannot see in the diff
An empty findings list is a perfectly good review. Reviewers who
always find something teach people to ignore them.
""";
private final ChatClient chatClient;
public PullRequestReviewer(ChatClient.Builder builder) {
this.chatClient = builder.defaultSystem(SYSTEM).build();
}
public Review review(PullRequest pr) {
return chatClient.prompt()
.user(u -> u.text("""
Pull request: {title}
Description: {description}
<diff>
{diff}
</diff>
""")
.param("title", pr.title())
.param("description", pr.description())
.param("diff", pr.unifiedDiff()))
.options(ChatOptions.builder().temperature(0.1).build())
.call()
.entity(Review.class);
}
}Module 4 — The approval gate
Where proposals become actions, and the only place they can.
package com.javaagentic.devagentic.remediate;
import org.springframework.stereotype.Service;
@Service
public class RemediationGate {
private final ProposalRepository proposals;
private final RunbookExecutor executor;
private final NotificationService notifications;
private final AuditLog audit;
/**
* The agent's only way to affect the world: create a proposal. It cannot
* approve, and there is no code path from an agent to executeApproved().
*/
public Proposal propose(String runId, RemediationAction action, String rationale) {
Proposal proposal = proposals.save(Proposal.pending(runId, action, rationale));
notifications.toOncall(proposal);
audit.record("proposal.created", runId, action.describe());
return proposal;
}
/** Called only from an authenticated human approval, never from an agent. */
public ExecutionResult executeApproved(String proposalId, String approverId) {
Proposal proposal = proposals.require(proposalId);
if (!proposal.isPending()) {
throw new IllegalStateException("proposal is not pending: " + proposal.status());
}
// Four-eyes: the approver must not be whoever raised the incident.
if (approverId.equals(proposal.raisedBy())) {
throw new IllegalStateException("a proposal cannot be approved by its raiser");
}
// Proposals go stale. A remediation reasoned about an hour ago may be
// actively harmful now.
if (proposal.isOlderThan(java.time.Duration.ofMinutes(30))) {
proposals.expire(proposalId);
throw new IllegalStateException("proposal expired; re-run the investigation");
}
audit.record("proposal.approved", proposal.runId(), approverId);
ExecutionResult result = executor.execute(proposal.action());
proposals.markExecuted(proposalId, result);
return result;
}
}Four guards, each earned from a real failure mode: status check (no double execution), four-eyes (no self-approval), staleness (no acting on outdated reasoning), audit (no unexplained changes).
Build order
- PR review bot. Highest value, lowest risk — its worst case is a bad comment.
- Read-only investigation tools, one at a time, tested individually before the agent sees them.
- The investigation agent with a small step budget and full trajectory logging.
- Trajectory review UI. You cannot improve what you cannot see.
- Proposals — display only. The agent suggests; nothing executes. Run it this way for weeks.
- The approval gate, once you have evidence the proposals are sound.
- Runbook execution, for a small set of reversible actions only.
What this project teaches
- The read/write split as an architectural boundary, not a convention
- Budgets and step caps as the only real defence against runaway loops
- Trajectory logging as the debugger for non-deterministic systems
- Why "propose, don't act" is the default for anything touching production
- Model selection per task: a strong model for evidence reasoning, a cheap one for routing
Next: FinAgentic, which adds multi-agent orchestration and a regulatory dimension.