← All work
Security & Compliance · Prototype · 2026

Agent Risk Graph

Design for a static scanner that proves agent exfiltration paths with CodeQL, checks them against the Rule of Two, then shows a flow-control fix blocking them.

Agent Risk Graph
Year
2026
Status
Prototype
Category
Security & Compliance
Role
Architect & Lead

Architecture

Four pure stages over files, with no server or UI runtime. Extract finds agents, tools, handoffs and approval nodes in LangGraph, OpenAI Agents SDK, CrewAI and MCP code and configs. Prove runs CodeQL (or Opengrep on code CodeQL may not be used on) and turns every SARIF source-to-sink path into a graph edge. Graph and Policy build an Agent Capability Graph and check the Rule of Two, toxic flows and missing human approval deterministically, with small fine-tuned classifiers labelling only what the rules cannot settle and an LLM reserved for the uncertain band. Output is SARIF 2.1.0, graph.json and a static SVG trust-boundary map.

Case study

Agent Risk Graph

Status: design and evaluation plan, September 2026. Build scheduled; no results yet. Every number on this page is either a published third-party figure (linked, dated and labelled as published) or a target. Nothing here is a measurement of this tool.

Agent Risk Graph is the rebuild of the earlier scanner. It names every path by which an agent can read, write, call, trigger or leak. It proves each path with a real dataflow engine, checks it against the Rule of Two on a capability graph, and then demonstrates it: a live exploit on a local demo app, which a flow-control policy then blocks.

[[toc]]

The problem

An agent's risk is not in any one file. It is spread across tools, memory, retrievers, outbound APIs, the permissions each tool carries, and whether a human approval step sits on the path. A tool defined in one module is handed to an agent in another. A retriever pulls in text nobody on the team wrote. An MCP server ships only a name, a description and a JSON schema, so the code behind it is never visible.

The dangerous case is a combination. One agent reads something untrusted, such as an email or a fetched page. The same agent can read something private, such as notes or secrets. And it can act outward, by sending HTTP, writing state or running code. None of these capabilities is a bug on its own. Together, with no approval in between, they are an exfiltration path.

The field data says this is common. Exposed by Design audited 414 production MCP servers and found 68 vulnerabilities, with 91.8% of the servers lacking OAuth (published, arXiv 2608.00150, 2026-07-31). VIPER-MCP reports 106 zero-days and 67 CVEs across 39,884 MCP repos (published, arXiv 2605.21392, v2 2026-08-12).

Why the usual approach is out of date

Prompt-level defences do not hold. "The Attacker Moves Second" broke 12 in-band defences with adaptive attacks at above 90% attack success rate (published, arXiv 2510.09023, 2025-10-10; USENIX Security 2026). A scanner that recommends "add a guard prompt" is recommending something that has already been shown to fail.

Flow control holds up much better. CaMeL completes 77% of AgentDojo tasks with provable security (published, arXiv 2503.18813). FIDES puts information-flow labels and a pre-execution policy into Microsoft's agent framework, shipped as experimental in agent-framework-core 1.3.0 and later (published, Microsoft devblog, 2026-05-20). A June 2026 adaptive evaluation of these out-of-band defences took Progent from 25.8% to 4.2% attack success, and 2.6% under adaptive attack, though its authors note this is a single 7B model (published, arXiv 2606.26479, 2026-06-25).

The engines caught up. CodeQL 2.24.1 added an experimental py/prompt-injection query with models for the OpenAI Agents SDK and openai (changelog, 2026-02-05). CodeQL 2.26.0 made Python taint more precise through container contents and self.attr, and added prompt-injection sinks for more GenAI SDKs (changelog, 2026-07-08). Pattern matching on names is no longer the best available evidence.

The policy lens is now written down. Meta's Agents Rule of Two says an agent holding untrusted input, sensitive data and external action at once needs a human in the loop (Meta AI, 2025-10-31). The OWASP Top 10 for Agentic Applications 2026 (2025-12-09) and the OWASP GenAI LLM Top 10 2026 (2026-08-04, Excessive Agency now #3) give the taxonomy.

So the design is neuro-symbolic. A deterministic engine proves. A graph and a fixed policy decide. Small models label only what the rules cannot settle, and an LLM sees only the uncertain remainder.

Architecture

flowchart LR
  R["Repo + MCP configs"] --> X["Extract
agents, tools, handoffs, approval nodes"]
  R --> E["Prove
CodeQL 2.27.1 + models-as-data
or Opengrep 1.30 intra-file"]
  X --> RL{"Effect rules
(code-backed tools)"}
  RL -->|opaque tool| LY["Fine-tuned Laya
one question per effect"]
  LY -->|below confidence gate| J["LLM judge
(fails closed)"]
  X -->|descriptions, prompts| PI["Injection classifier"]
  E -->|SARIF paths| G["Agent Capability Graph"]
  RL --> G
  LY --> G
  J --> G
  PI --> G
  G --> P{"Policy
Rule of Two / toxic flow / missing approval"}
  P --> T["Triage cascade"]
  T --> O["SARIF + graph.json + SVG map"]
  O -.->|predicts| D["Proof harness
inbox assistant + canary"]
  • Extract uses the Python standard library ast and a JSON parse. Framework adapters find LangGraph nodes, edges and interrupt() calls, OpenAI Agents SDK Agent(tools=, handoffs=), CrewAI agents and tasks, FastMCP @tool handlers, mcp.json server entries, memory stores and retrievers.
  • Prove runs CodeQL with our models-as-data YAML adding sources and sinks for LangGraph tools, FastMCP handlers, retrievers and memory. Each SARIF path becomes a FLOWS_TO edge with file and line steps. There is no co-occurrence or substring matching.
  • Graph and Policy compute each agent's reachable capability set through its tools, handoffs and sub-agents, then run three checks: the Rule of Two, toxic flow and missing approval. Findings map to OWASP ASI 2026, the LLM Top 10 2026 and MITRE ATLAS v2026.09, for example AML.T0086, Exfiltration via AI Agent Tool Invocation.

CodeQL's licence allows it on open-source code, academic research and demonstrations, not on private code without GitHub Advanced Security. So the public benchmarks use CodeQL, and the path for private code is Opengrep, which is LGPL-2.1 but only follows taint within a file.

Event flow

One scan, from the command line to the report:

sequenceDiagram
  participant CLI as acg scan
  participant X as Extract
  participant Q as CodeQL
  participant G as Capability graph
  participant P as Policy
  participant R as Report
  CLI->>X: parse agents, tools, handoffs, approvals
  CLI->>Q: database create + analyze
  Q-->>G: SARIF paths (source to sink, file:line)
  X-->>G: nodes + effect labels
  G->>P: reachable capability set per agent
  P->>P: Rule of Two, toxic flow, missing approval
  P-->>R: findings with ASI / ATLAS IDs
  R-->>CLI: findings.sarif, graph.json, map.svg

And the runtime path the scan is meant to predict, on the deliberately vulnerable "inbox assistant" demo:

sequenceDiagram
  participant A as Attacker email
  participant T as read_email tool
  participant L as Agent LLM
  participant N as Private notes
  participant S as send_http tool
  participant C as Local canary
  A->>T: indirect injection in body
  T->>L: untrusted text enters context
  L->>N: read secret
  alt undefended
    L->>S: send_http(secret)
    S->>C: secret received
  else approval gate or FIDES policy
    L->>S: send_http(secret)
    S-->>L: denied before execution
  end

How each decision is made

Decision Method Instead of
Is there a flow from untrusted input to a privileged sink? CodeQL taint paths, or Opengrep intra-file Names appearing in the same file
What can each tool do? (reads private, writes state, network egress, executes code, irreversible) Rules first, from CodeQL sink kinds on tools whose code is in the repo. Opaque tools go to a fine-tuned Laya classifier, one binary question per effect. Below the confidence gate, an LLM judge An LLM reading every tool
Is this finding real and exploitable? Cascade: the Rule-of-Two filter, then a fine-tuned Laya ranker, then an LLM only for the uncertain band, then a grounding check that the LLM's reasoning cites real source spans One LLM agent run per alert
Does this text carry an injection? The capability graph decides whether it matters. A dedicated injection classifier (PIGuard, fine-tuned on MCP tool-poisoning) scores the text An LLM judging each string
Is the agent allowed to do this? Deterministic policy code A model's opinion

The rules tier is high-precision but not complete. Dynamic dispatch, getattr, eval and calls through SDKs without models can hide an effect, so its negatives are hand-audited and treated as weaker than its positives.

The classifier labels follow one rule: a safe label never loosens policy. Each effect fails closed on its own. A confident "dangerous" answer can add a capability and tighten the Rule of Two check. A "safe" answer below the confidence gate never removes one. The capability stays assumed present and the tool goes to the LLM judge. If the judge is unavailable or its budget is spent, the answer is still "present". Uncertain means dangerous.

The confidence gate is set per effect from a risk-coverage curve on held-out labels, not a fixed threshold. Laya's base checkpoints are near chance zero-shot on typed decisions (the English base scores 0.362 against a 0.461 majority baseline, published by the vendor on the model card), so fine-tuning is required. The code-backed tools supply free training labels. Long MCP schemas are summarised to fit the 1,024-token checkpoint.

On injection text, the evidence favours a graph-first design. In Mozilla.ai's independent benchmark, PIGuard scored F1 0.86 on BIPIA email and 0.91 on tables, but no detector was reliable on function-calling, where the best F1 was 0.5 (published, Mozilla.ai, 2025-11-06). So the graph check is the primary control and text detection is secondary. PIGuard's weights licence is checked before it ships.

Code walkthrough (design sketches)

These are design sketches of the key interfaces, written for this page. They are not excerpts from a codebase.

Design sketch 1: the policy check. Only proven flows count, and an approval node on the path clears the finding.

UNTRUSTED, PRIVATE = "UNTRUSTED_SOURCE", "READ_PRIVATE"
ACTIONS = {"EGRESS", "WRITE_STATE", "CODE_EXEC"}

@dataclass(frozen=True)
class Finding:
    rule: str
    agent: str
    steps: tuple[str, ...]  # SARIF path steps, "file:line"

def check_agent(agent: str, graph: CapabilityGraph, flows: list[Flow]) -> list[Finding]:
    caps = graph.reachable_capabilities(agent)  # tools, handoffs, sub-agents
    out = []
    if UNTRUSTED in caps and PRIVATE in caps and caps & ACTIONS \
            and not graph.every_action_path_approved(agent):
        out.append(Finding("rule-of-two", agent, ()))
    for f in flows:  # from CodeQL/Opengrep SARIF, never co-occurrence
        if (graph.effect(f.source) == UNTRUSTED
                and graph.effect(f.sink) in ACTIONS
                and graph.owner(f.sink) == agent
                and not graph.approval_on(f)):
            out.append(Finding("toxic-flow", agent, f.steps))
    return out

Design sketch 2: the capability labeller. Rules first, then Laya, then the judge, failing closed per effect.

EFFECTS = ("reads_private", "writes_state", "network_egress",
           "executes_code", "irreversible")

def label_tool(tool: Tool, rules: Rules, laya: Laya, judge: Judge,
               tau: dict[str, float]) -> dict[str, bool]:
    """True means the capability is assumed present."""
    labels = {}
    state = summarise(tool.name, tool.description, tool.schema, max_tokens=1024)
    for effect in EFFECTS:
        proven = rules.effect(tool, effect)  # True/False from sink kinds, None if opaque
        if proven is not None:
            labels[effect] = proven
            continue
        p_present, confidence = laya.ask(state, effect)  # two-option choice
        if confidence >= tau[effect]:
            labels[effect] = p_present >= 0.5
        else:  # a low-confidence "safe" never loosens policy
            labels[effect] = judge.ask(state, effect, on_error=True)
    return labels

Design sketch 3: the CLI output shape. One finding as it lands in graph.json, with a SARIF twin for code scanning.

# acg scan <path> [--engine codeql|opengrep] [--judge mlx|api|off]
class FindingOut(TypedDict):
    rule: Literal["rule-of-two", "toxic-flow", "missing-approval"]
    agent: str
    source: str          # e.g. "read_email" (UNTRUSTED_SOURCE)
    sink: str            # e.g. "send_http" (EGRESS)
    in_scope: list[str]  # e.g. ["notes_store"] (READ_PRIVATE)
    steps: list[str]     # "file:line" per SARIF path step
    approval_on_path: bool
    taxonomy: dict[str, list[str]]  # {"owasp_asi": [...], "atlas": ["AML.T0086"]}
    triage: Literal["confirmed", "rejected", "needs_human"]
    decided_by: Literal["rules", "laya", "llm"]

How it will be measured

Everything runs on one M2 Max laptop. Tool versions, commit SHAs and model checkpoints are pinned in a lock file, every chart is drawn from CSV, and public scorers are used unmodified.

Benchmarks.

  • RealVuln (26 Python repos, 796 labels including 120 false-positive traps). Precision, recall and F3 with the paper's scorer. Published reference points: Semgrep F3 17.7 and Claude Sonnet 4.6 F3 51.7 (published, arXiv 2604.13764, 2026-04-15).
  • OWASP BenchmarkPython v0.1. The standard scorecard: true-positive rate, false-positive rate and Youden index per CWE.
  • Agentproof's 18 workflows. Agreement with Agentproof's own missing-approval check, plus runtime. Its paper reports 27% structural defects and 55% human-gate violations (published, arXiv 2603.20356, 2026-03-20).
  • Agent Audit GT v2.2. Recall on its 236 positives only, since the set has just 2 negatives.
  • Our own labelled set, about 60 small programs, 30 positive and 30 negative, published openly and labelled as author-built. The negatives are written first, to trap co-occurrence heuristics.

The headline experiment: classifiers against an LLM. CodeQL runs on OWASP Benchmark Java plus 10 to 20 public agent repos, and at least 500 alerts are hand-labelled. Arm A is an LLM triage agent on every alert. Arm B is the cascade. Success means B's F1 is within A's 95% bootstrap interval or better, with at least 70% fewer LLM calls and lower wall time. The side tables cover tool-effect labelling (Laya against an LLM judge, with hidden-body CodeQL labels as ground truth, headlined by the false-"safe" rate on egress and exec) and injection detection (PIGuard fine-tuned against Prompt-Guard-2 and an LLM judge, recall at 1% false-positive rate or less).

Load and throughput charts. A corpus of 300 open-source agent repos with pinned commits.

  • Throughput: x axis concurrency workers (1, 2, 4, 8, 12), y axes repos per minute and p50/p95 per-repo latency, with peak memory annotated.
  • Stage breakdown: stacked latency per repo for extract, database build, query, policy and judge.
  • Policy scaling: x axis graph nodes on a log scale from 10 to 5,000, y axis policy latency. Agentproof reports verification under a second up to 5,000 nodes (published, same paper).
  • Cost: x axis API dollars spent, y axis triage precision gained, local model against API.

AgentFlow's median 14.17 s and p95 163.54 s per project (published, arXiv 2607.01640, 2026-07-02) appear as a labelled reference only. It is a different corpus and its code is not released.

Proof of capability

One end-to-end story on the inbox assistant, which reads email (untrusted), reads a private notes store, and has send_http.

  1. Scan. acg scan demo/inbox_agent should report one Rule of Two violation and one toxic flow, from read_email through the LLM context into the arguments of send_http, with the notes store in scope, CodeQL path steps, ATLAS AML.T0086, and no approval node on the path.
  2. Exploit. A corpus of 100 indirect-injection emails, including an adaptive loop where the attacker sees refusals, runs against a local model. Success means the secret reaches a localhost canary.
  3. Fix. Add a LangGraph interrupt() before send_http, or a FIDES confidentiality policy. Re-scan and the finding should clear.
  4. Re-run. The same corpus runs against the undefended, approval-gated and FIDES versions. The page will report attack success rate and task utility before and after, and check that the scan flagged exactly the path the successful attacks used.

The canary log, the finding that predicted it and the clean re-scan will sit side by side, each linked to its CSV.

What changed from the earlier scanner

  • Proof replaces pattern matching. The earlier scanner built its own AST graph and scored threats by co-occurrence, and every attack it matched was reported at full confidence. Now every flow is a CodeQL or Opengrep SARIF path, and a regression test checks that two names in unrelated files produce nothing.
  • The unit of risk is the agent. Findings come from an agent's combined capabilities under the Rule of Two, not from individual nodes matching ATLAS patterns.
  • The taxonomy is current. The scanner now maps findings to OWASP ASI 2026, the LLM Top 10 2026 and ATLAS v2026.09, which also corrects an old mapping: AML.T0040 is AI Model Inference API Access, not model inversion.
  • Models only where judgment is needed. Rules and small fine-tuned classifiers handle bounded labels. An LLM sees the uncertain band, and never has the last word on policy.
  • Smaller, and measured against its predecessor. The React, FastAPI, Docker and Terraform stack gives way to SARIF, JSON and a static SVG. The old card's figures are retired until new benchmarks exist, and the earlier scanner runs as a baseline on every benchmark.

Sources

Tech stack

CodeQLOpengrepPythonLayaPIGuardMCP

Other 2026 work

Developer Tools
TeamFlow
2026-09-17
Security & Compliance
Provely
2026-09-05
AI & Machine Learning
Heart
2026-08-14