Detection Engineering for AI Agents: Rules That Catch a Hijacked Run.

Why your existing rules miss this

A hijacked agent does not trigger impossible-travel alerts, does not brute-force a login, and does not run whoami. It authenticates correctly as itself, calls APIs it is authorized to call, and does so through the same code path as every legitimate run. Every atomic action looks fine. Only the pattern is wrong.

Meanwhile the timeline is compressed. The Hugging Face intrusion involved thousands of actions across disposable sandboxes, a volume and pace no human operator produces. Detection tuned to human dwell time fires after it matters.

This post is the detection layer that sits on top of the harness controls: eight rules that catch agent-specific compromise, written against telemetry you should already be emitting.

The telemetry contract

Detections are only as good as the events beneath them. From the harness and gateway, per action:

{
  "ts": "2026-07-26T03:14:22Z",
  "session_id": "sess_01J...",
  "agent_id": "invoice-processor",
  "principal": "svc:ap-automation",
  "on_behalf_of": "u_4821",
  "action_type": "tool_call",
  "tool": "payments.execute",
  "decision": "denied",
  "reason": "counterparty_not_allowlisted",
  "destination": "api.internal.acme.com",
  "args_digest": "sha256:...",
  "result_bytes": 1841,
  "input_sources": ["doc_88231", "doc_88244"],
  "turn": 17,
  "latency_ms": 240
}

Two fields carry most of the detection value. decision distinguishes what the agent attempted from what it achieved, which is the difference between a signal and noise. input_sources ties an action back to the content that may have steered it, which is what makes an alert investigable in minutes rather than hours. These are the same records that feed your audit trail, so the marginal cost is close to zero.

Eight detections

1. Denied-action burst. The highest-signal rule you can write. A legitimate agent rarely gets denied; it was built to work within its permissions. A run accumulating denials is either broken or being steered.

SELECT session_id, agent_id, count(*) AS denials,
       array_agg(DISTINCT reason) AS reasons
FROM agent_actions
WHERE decision = 'denied' AND ts > now() - interval '5 minutes'
GROUP BY session_id, agent_id
HAVING count(*) >= 5

Start at five in five minutes and tune down. When this fires, the reasons array usually tells you the story immediately.

2. First-seen tool or destination for an agent. Agents have narrow, stable action vocabularies. An invoice processor that has called four tools for six months and suddenly calls a fifth is worth a look, even if that tool is technically permitted.

SELECT a.session_id, a.agent_id, a.tool, a.destination
FROM agent_actions a
LEFT JOIN agent_baseline b
  ON b.agent_id = a.agent_id AND b.tool = a.tool AND b.destination = a.destination
WHERE a.ts > now() - interval '5 minutes' AND b.agent_id IS NULL

Maintain agent_baseline from a trailing 30-day window, and require a tool to appear on several distinct days before it counts as normal, so one anomalous run does not teach the baseline to accept itself.

3. Rate anomaly against a per-agent baseline. Actions per minute, compared to that agent's own history rather than a global threshold. A batch reconciliation agent legitimately runs at 200 actions per minute; a customer-support agent doing that is compromised. Global thresholds are why teams turn these alerts off.

4. Retrieval breadth (the enumeration signature). A normal question touches a handful of documents. An agent instructed to "summarize everything you can access" touches hundreds. Distinct doc_id count per session against that agent's p99 is one of the cleanest exfiltration-staging detections available, and it also catches over-broad retrieval permissions before an attacker does.

5. Output volume and encoding anomalies. Exfiltration through an allowed channel looks like a normal call with an abnormal payload. Alert on outbound args_digest sizes well above the tool's historical distribution, and on base64-shaped or hex-shaped blobs in arguments to tools that never legitimately carry them.

6. Credential minting anomalies. With a broker minting short-lived tokens, mint rate is a direct proxy for agent activity, and mints for a host the session has never used before are a strong lateral-movement signal. A token minted for one session appearing from a different source is a critical alert, not an informational one.

7. Instruction-shaped text in retrieved content. Scan retrieved chunks and tool results for imperative patterns aimed at the model: "ignore previous instructions," "you are now," "send the following to," hidden-text markers, zero-width characters, or invisible-styling tricks in HTML. This is a lossy, bypassable detection and it is still worth running, because it catches unsophisticated attempts and, more importantly, it flags the source document so you can quarantine it and find siblings. Log the hit against input_sources rather than blocking on it, so a false positive never breaks a legitimate run.

8. Honeytokens in the corpus. Plant documents that no legitimate query should ever retrieve: a fake "Executive Compensation Q3" file, a fake credentials note, each with a unique unmistakable string. Any retrieval of one, or any appearance of its canary string in an output or an outbound payload, is a high-confidence alert with essentially no false-positive rate. This is the cheapest detection on the list and often the first to fire in a real incident.

Baselines per agent, not per fleet

The single most common reason agent detections get muted is a global threshold applied across agents with wildly different legitimate behavior. Build the baseline per agent_id, and rebuild it on a schedule from a trailing window that excludes any session already flagged as an incident. Otherwise your baseline slowly learns to accept the attack.

New agents have no history, which is exactly when they are least understood. Run them in a stricter mode for their first weeks: tighter denial thresholds, alert on every first-seen tool, lower rate ceilings.

Detection as code

These rules are software and deserve the same treatment: version controlled, code reviewed, and tested against recorded traffic.

Keep a replay corpus of two kinds of sessions: known-good runs (regression tests, so a tuning change does not silently disable a rule) and known-bad runs, drawn from incidents, tabletop exercises, and your own red-team runs. Every real incident should contribute a session to this corpus, the same way it contributes a case to the eval gate. A rule that no longer fires on last quarter's incident has regressed, and only a test tells you.

Response, partially automated

Detection without a response path is a dashboard. Match the automation to confidence:

Freezing preserves state for forensics while stopping new actions, which is why it is the default automated response rather than termination. Auto-freeze needs an unfreeze path that a human can use quickly, or your first false positive will teach the team to disable it.

Where to start this week

Two rules and a honeytoken get you most of the value in an afternoon. Emit the action log with decision and input_sources. Write the denied-action burst query and point it at a channel a human reads. Plant three honeytoken documents in the corpus and alert on retrieval or on their canary strings appearing anywhere.

Then add per-agent baselines once you have a few weeks of history to build them from. The remaining rules are refinements; those three catch the incidents that actually happen.