Provely

Provely

An agent calls POST /v1/refunds, reads 200 OK, and reports "I refunded the customer". The refund may still be pending, no money may have moved, or the retry may have charged the customer twice. Or the model wrapped its arguments in a code fence, the JSON never parsed, and the refund never ran at all.

A successful tool call is not proof of a completed business outcome. Provely is the defensive layer that closes that gap, live at provely.sh.

Two lines

import { Provely } from '@provely/sdk';
new Provely();

Provely finds the agent's framework by itself (LangChain, LangGraph, LlamaIndex, OpenAI Agents, Vercel AI, the OpenAI SDK and the Anthropic SDK, in TypeScript and Python) and loads its plugin. There is no guard call to write and no adapter to import. It puts one deterministic check on each model answer and tool call, and when one fails it makes the fix itself:

Every fix writes one decision-log line of ids and codes, and guard_mode: observe is the default. The SDKs are in private preview. No language model takes part in any decision. Rules are optional; teams that want to audit or share the checks can compile and sign them into .provely/rules.yaml. The site publishes its measured coverage per failure mode and per framework, including what still fails.

When a check is ambiguous, and learning from the log

Some cases are not clear-cut: a tool result that starts with "Error" can be "Error rate is 0% for the last hour." For those, Provely asks the agent's own model, through the client the agent already holds, a fixed question with a closed set of answers. The judge only runs when the deterministic answer is unclear, has a 3-second timeout and a budget of 5 questions per run, and with no answer the check does what it did before. The prompt and inputs never leave the process.

Learning is always on. A shadow mode asks the judge on 1% of results, within its own budget and capped per day, and changes no decision. The learner (provely rules learn) groups the decision log by fingerprint and proposes rules in three scopes: one app, one organisation, or every customer. A rule that only tightens can apply by itself; one that widens waits for a person. Only fingerprints and counts leave an app, and a general rule needs the same fingerprint in at least 3 organisations plus a review by the Provely team.

Before the agent runs

provely scan reads a repository and lists its agents, tools, prompts and Agent Skills, then groups the new side-effecting calls into a review that a person confirms item by item; there is no accept-all. The scanner reads the documented tool forms of each framework, now including the Claude Agent SDK's tool() definitions and the built-in tools an agent enables by name, such as Bash, Write or Edit.

Proof of the outcome

For actions with side effects, Provely turns the task into a deterministic outcome:

1. A completion contract that states what "done" means.
2. A verdict that rests on evidence from the real systems, never on a language model's judgement.
3. A signed receipt that anyone holding the trusted key can check offline.

Signed integrations cover 16 providers, including Stripe, Slack, Linear, HubSpot, Google Drive and OpenAI. An integration compiler maintains them from schemas, documents and tests. Agents get Provely as MCP tools and Agent Skills, and provely init writes the guidance, hooks and CI gate for hosts such as Claude Code, Codex and Cursor.