Agent Verify Docs

Every scan produces one of three verdicts:

• VERIFIED — sufficient execution context was assessed and the required trust controls passed.
• NOT VERIFIED — sufficient execution context was assessed and one or more security or protocol controls failed.
• NOT ASSESSED — the submitted content did not contain enough agent execution context (tools, permissions, actions) to make a verification decision. This is not a security failure — it means Agent Verify could not evaluate the input.

Findings are split into two categories: A — A2SPA Protocol Compliance (signature, nonce, timestamp, fail-closed, and scope evidence for execution-time authorization) and B — General Agent Security (secrets, tool access, human approval gates, audit logging, rate limiting, prompt injection surface, and related runtime risk).

Two ways to run a scan:

1. Sign in and use the web scanner to paste or upload agent code/config for a private report.
2. Install the CLI and scan from your terminal or CI pipeline:

npm install -g agentverify
agentverify scan ./agents --key av_your_key

Get an API key from Dashboard → API access after signing in.

The dashboard scanner runs the same @agentverify/scanner engine as the CLI and API against pasted code or an uploaded file, then saves a private report to your account. Reports created this way are client-submitted: they are useful for iterating on your own agent, but they are not server-attested evidence in the same sense as a report saved through an authenticated CLI/API scan. See Limitations.

agentverify scan [dir] --key <api-key>
agentverify scan --file <file> --key <api-key>
agentverify --help
agentverify --version

Common flags:

• --key, -k — API key (or set AGENTVERIFY_API_KEY)
• --file, -f — scan a single file instead of a directory
• --json — machine-readable output
• --markdown — Markdown summary, useful for PR comments
• --ci — concise output with strict exit codes
• --allow-not-assessed — in CI mode, do not fail solely because content is NOT_ASSESSED

CI exit codes:

Package: agentverify on the public npm registry.

npm install -g agentverify
# or run without installing
npx agentverify scan . --key av_your_key

Requires Node.js 18 or later. The package ships both CommonJS and ESM builds and TypeScript types; it has no dependency on this monorepo — it works standalone in any project.

Sign up with email/password or Google. Forgot your password? Use Forgot password? on the sign-in form — if an account exists for that email, a reset link is sent. For account-security reasons, the response is identical whether or not the email is registered.

API keys are generated per-account from Dashboard → API access. Regenerating a key immediately revokes the previous one.

A workspace (also called an organization) is a shared context multiple people can belong to, with its own members, roles, audit log, and integration settings — separate from your personal account, which continues to work exactly as before if you never create one. Scans, reports, and API keys stay tied to your own account either way; a workspace adds a shared view on top: who else can see what, and a record of what happened.

Why a team uses one: a solo account has no concept of teammates. A workspace lets a security lead invite the rest of the team, see every member's scan activity in one audit log, and (as an Owner or Admin) enforce who can invite others, change policies, or configure integrations — without sharing a single login.

Roles

Every role check happens server-side against the role actually stored for your account in that workspace — the dashboard only hides buttons you can't use, it never grants anything on its own.

Creating a workspace

From Dashboard → Workspace, enter a name and select Create workspace. You become its Owner immediately. There's no limit on how many workspaces you can belong to, and no migration or setup required — your existing scans and reports are unaffected.

Managing members

An Owner or Admin invites a teammate by email from the Members tab and assigns their role. The invited person must already have an Agent Verify account with that email — invites don't currently create an account for someone who hasn't signed up. Roles can be changed or a member removed at any time (except the Owner, who can't be demoted or removed through this screen — a workspace always keeps its Owner).

API keys and workspaces

Your API key (from Dashboard → API access) is tied to your personal account, not to a specific workspace — there is no separate workspace-owned key today. What a workspace does add: a scan request can optionally include an organizationId, and if your role in that workspace has scan permission, that scan is attributed to the workspace and appears in its Audit Log. This is the real, current mechanism for connecting scan activity to a team — workspace-issued/rotatable API keys are not implemented yet.

Audit log

The Audit Log tab is a server-written record of what actually happened in the workspace — never something a client can fabricate or backdate. Every entry the log can show today is one of: a scan completing (with its verdict), an attestation being issued, a policy being applied to a scan, a member being added or removed, a role being changed, or a webhook being created or disabled. It's searchable and filterable by action type directly in the tab.

Integrations

The Integrations tab lists the real ways to connect Agent Verify to your workflow: GitHub Actions and the CLI for CI/CD, and the REST API for custom tooling — all available today. Webhooks are the newest addition; see the next section for exactly what that means right now.

A webhook lets Agent Verify notify another application automatically when something happens in your workspace — instead of that application having to repeatedly ask "did anything change yet?", Agent Verify would push an event to a URL you provide, the moment it happens. Owners and Admins configure webhooks from Dashboard → Workspace → Webhooks.

How delivery works

When one of the events below happens, Agent Verify inserts a durable delivery record (Cloudflare D1) for every active webhook subscribed to it, then a Cron Trigger sweeps for due deliveries roughly every minute and sends a real signed POST. This is not fire-and-forget: every attempt's outcome (HTTP status, response snippet, error, timestamp) is written back to that same durable record, which is what powers the delivery history and retry described below.

Supported events

A webhook can be subscribed to any of these seven event types — the exact set the API accepts, no more:

Payload & signature

This is exactly what your endpoint receives once delivery is live in production:

POST <your endpoint URL>
Content-Type: application/json
Agent-Verify-Signature: t=1735689600,v1=5f4dcc3b5aa765d61d8327deb882cf99...

{
  "eventId": "3fa2c1e0-...",
  "eventType": "VERIFICATION_FAILED",
  "organizationId": "org_abc123",
  "timestamp": "2026-09-02T13:49:12.000Z",
  "data": { "scanId": "REPORT-qhhic9fbkv", "verdict": "NOT_VERIFIED", "score": 30, "fileName": "agent.ts" }
}

The Agent-Verify-Signature header carries t=<unix-seconds>,v1=<hex-encoded HMAC-SHA256> — the same shape Stripe uses for its own webhooks. The signature is computed over ${timestamp}.${rawRequestBody} using your webhook's secret (shown once, at creation — store it like any other credential). To verify a delivery:

const [t, v1] = signatureHeader.split(',').map(p => p.split('=')[1])
const expected = hmacSha256Hex(webhookSecret, `${t}.${rawBody}`)
if (!timingSafeEqual(v1, expected)) reject('invalid signature')
if (Math.abs(nowInSeconds() - Number(t)) > 300) reject('too old — possible replay')

Replay/freshness protection: implemented in the reference verifier — a signature older than 5 minutes is rejected as expired, so a captured, valid delivery can't be replayed indefinitely.

Expected HTTP behavior, retries & dead-letter

Your endpoint should verify the signature, then respond quickly with a 2xx status to acknowledge receipt. Anything else (a non-2xx status, a timeout, or a connection error) is treated as a failed attempt and retried with exponential backoff: 1 minute, 5 minutes, 30 minutes, 2 hours, then 12 hours. After 6 total attempts with no 2xx response, the delivery becomes a dead letter and stops retrying automatically.

Delivery history & manual retry

Every webhook in the Workspace → Webhooks tab has an expandable delivery history: status (Delivered / Retrying / Failed), attempt count, the last HTTP status or error, and (while retrying) when the next attempt is scheduled. A dead-lettered delivery can be manually retried — an Owner or Admin's explicit decision gives it a full fresh attempt cycle, not just one more try. Delivery history never includes the event payload or the signing secret, only attempt/outcome metadata.

Disabling a webhook

An Owner or Admin can disable a webhook at any time from the Webhooks tab — it stops being active immediately, including any deliveries already queued and waiting for their next retry (disabling is checked again right before every attempt, not just when a delivery is first queued). There is currently no way to permanently delete a webhook or rotate its secret; if a secret is compromised, disable that webhook and create a new one.

Security recommendations

• Always verify Agent-Verify-Signature before trusting a payload — never process an unsigned or incorrectly-signed request.
• Reject anything outside the 5-minute timestamp window, even with a valid signature.
• Use an HTTPS endpoint. Plain http:// is technically accepted, but a real destination should not accept unencrypted webhook traffic.
• Treat your webhook secret like any other credential — it's shown once, at creation, and never displayed again.
• A webhook endpoint can never point at a private/internal address (localhost, RFC1918 ranges, cloud metadata endpoints) — Agent Verify rejects those at creation time.

Authenticated scan endpoint (used by the CLI and SDK):

POST https://agentverify-api.agentverify.workers.dev/v1/scan
Authorization: Bearer av_your_key
Content-Type: application/json

{ "content": "<agent code or config>", "fileName": "agent.ts" }

Responses include the full scan result (verdict, riskScore, findings, categoryScores, bom, threatCategories, metadata) plus reportId, saved, and reportUrl when a report was persisted. A 429 response means your plan's monthly scan quota (10 scans/month on Free, 100/month on Pro) has been reached. A 401 means the key is missing, malformed, disabled, or revoked.

The Worker also serves a live SVG badge at GET /v1/badge/:reportId for saved reports, and the billing endpoints used by the dashboard (/v1/billing/status, /checkout, /portal) which require a signed-in Firebase session, not an API key.

Run Agent Verify as a required PR check:

- uses: AI-Blockchain-Ventures/agentverify@v1
  with:
    path: ./agents
    api-key: ${{ secrets.AGENTVERIFY_API_KEY }}
    format: text

Or call the CLI directly in any workflow:

- run: npx --yes agentverify scan ./agents --ci
  env:
    AGENTVERIFY_API_KEY: ${{ secrets.AGENTVERIFY_API_KEY }}

Add the job to branch protection so a non-zero exit blocks merge. Full reference: docs/github-action.md.

Every result carries schemaVersion, a stable machine-safe verdict (VERIFIED / NOT_VERIFIED / NOT_ASSESSED), a riskScore (0–100), riskLevel, a confidence score, category scores for Protocol Compliance and Security Controls, a findings array, a Runtime Bill of Materials, and scan metadata (scanner version, timestamp, duration, detected language/framework).

The score starts at 100 and is reduced by weighted findings: −20 per critical finding, −10 per high, −5 per medium (low findings don't subtract points but still appear in the report). Broad/wildcard tool or permission access caps the score at 65 regardless of other findings. A VERIFIED verdict additionally requires zero critical findings, at most one high finding, no broad permissions, and either full A2SPA execution-authorization evidence (signature, nonce, fail-closed) or a complete security-controls baseline (no hardcoded secrets, audit logging, rate limiting, and human approval gates present).

This is a heuristic, evidence-based score, not a guarantee. See Limitations.

Each finding includes a stable code, severity (critical/high/medium/low), category (A or B), a plain-language description of what was found and why it matters, a line number when locatable, a plain-English "what an agent could do because of this" note, a recommended fix, an evidence type, and — where relevant — mappings to OWASP LLM Top 10, NIST AI RMF, and SOC 2 controls. Findings that reference a detected secret show a redacted preview (e.g. api_key: "sk_l****" (redacted)) — the live value is never included in a report. Named, higher-confidence patterns are detected for OpenAI/Anthropic/AWS/GitHub/Stripe/Slack keys, private key blocks, and database URLs with embedded credentials, in addition to a generic key/password/token pattern.

Every finding also carries an evidence type so a heuristic guess is never presented as an established fact:

• Definite — a concrete pattern was matched (a literal wildcard permission, a literal secret literal).
• Heuristic — inferred from the absence of an expected pattern, or contextual keyword co-occurrence. Can under- or over-detect if a control is implemented in a way the scanner doesn't recognize.
• Informational — a neutral observation, not itself good or bad.

Capabilities answer "what could this agent actually do if compromised?" — read/write/delete files, execute commands, deploy software, access/modify databases, send email/messages, transfer money, issue refunds, touch crypto wallets, reach cloud infrastructure, create/delete users, or call external APIs. Each capability is derived only from concrete evidence in the submitted content, with its own confidence level — never guessed from a tool name alone.

MCP (Model Context Protocol) detection identifies MCP server configuration, known server/tool categories (filesystem, shell, database, GitHub, Slack, email, browser automation, payments, docker, memory), and builds an Agent → MCP Server → Tool → Potential Action chain when there is real evidence. An MCP integration with an unidentified server is reported as exactly that — "Unknown MCP server" — never invented.

Potential Blast Radius surfaces dangerous combinations of capabilities that matter more than any single permission — e.g. filesystem write + shell execution, or a payment capability with no visible destination constraint or human approval step. This never claims an exploit exists, only that the combination could allow the stated impact if the agent is compromised or misused.

Agent Verify runs 44 distinct, independently-tested verification checks — not a marketing round number. Each check has a stable ID (e.g. AV-SECRET-003, AV-MCP-005), a category, severity, detection type, and its own regression test asserting it actually fires. Every check the scanner can produce is cross-checked in both directions against a real test suite: no catalog entry exists without a working code path, and no working code path exists without a catalog entry.

Beyond the 44 checks, the scanner separately runs 19 capability detectors (what an agent can do), 10 MCP tool classifiers, 8 capability-chain ("blast radius") combination rules, and 14 threat-category assessments — these are classification/context systems, not pass/fail checks, and are counted separately so the 44 stays an honest number of real, distinct checks rather than an inflated one.

Findings are tagged with one of 11 technical security categories (Identity, Permissions, Tools, MCP, Execution Authorization, Secrets, Runtime, Network, Dependencies, Auditability, Human Oversight) — this is the ground truth every finding, check, and report is built from. For a non-technical audience, those 11 categories group into 7 executive families, derived from the same tags (never assigned separately, so the two views can never contradict each other):

• Identity & Access — Identity, Permissions
• Tools & Capabilities — Tools, MCP
• Execution Security — Execution Authorization
• Secrets & Data — Secrets
• Runtime & Network — Runtime, Network
• Supply Chain — Dependencies
• Oversight & Auditability — Auditability, Human Oversight

Every report is built from one canonical scan result — the same verdict, score, and findings, rendered six different ways for six different audiences, all from the exact same evidence, each in its own component (apps/web/src/components/report/views/). Switch between them with the tabs at the top of any report:

• Executive — a 60-second read for a CEO, CISO, CTO, buyer, or investor: an explicit deployment recommendation (DEPLOY / DEPLOY WITH CONDITIONS / DO NOT DEPLOY / NOT ASSESSED, derived only from the verdict and critical/high finding counts — never a separately-invented judgment), the specific deployment blockers when it isn't a clean DEPLOY, top risks, what to fix first, and a posture trend (Improving / Degrading / Unchanged) computed from your real scan history — shown honestly as "No prior scan available" rather than a guess when there is nothing to compare against.
• Security — answers "where can this agent hurt us?": attack surface, capabilities/privileges, MCP/tool exposure, capability chains (real evidence-based combinations, surfaced as "top attack paths" — never a fabricated one), credential/secret exposure, human-approval and authorization gaps, and a prioritized remediation plan.
• Developer — action-oriented, for every finding: rule/check ID, severity, confidence, file/line (explicitly "Not available" when the content doesn't make one locatable, never guessed), technical reason, exact remediation, example fix code when one is real, how to verify the fix (re-scan and confirm that rule ID no longer fires), and a machine-readable remediation metadata block for tooling.
• Compliance — a real evidence-mapping engine, not a re-render of finding labels. See Compliance Mapping below.
• AI / JSON — the stable, schema-versioned machine-readable report for CI/CD, SIEM, and API clients.
• Full Technical — essentially everything Agent Verify knows about the scan in one place, without exposing the detection engine's internal implementation.

All six are proven distinct from the same evidence object by a real automated test (apps/web/test/sixViewContract.test.mjs), which server-renders every view and asserts each one shows the fields specific to it and none of the fields that belong only to another view.

A report page shows the verdict, score, category breakdown, threat-category assessment, prioritized fix plan, individual findings with evidence, compliance mapping, and the Runtime Bill of Materials. Reports created via the CLI/API are stored separately from dashboard (browser) reports so their provenance (source: cli vs dashboard) is always visible on the report itself.

Reports are private by default. An owner can explicitly toggle a report to Public from Share settings, which makes it reachable at its canonical URL without signing in; toggling back to Private immediately revokes that access. Anyone who can edit the report can regenerate its content by re-scanning, which produces a new report ID and URL — the old link is unaffected. Password-protected sharing is not implemented yet; the UI will not claim it works until a server can verify a password before releasing report content.

agentverify scan ./agents --json > results.json

Produces { results: [...], summary: { total, verified, notVerified, notAssessed, errors } } for CI systems, dashboards, or custom tooling.

The AI/JSON report view is validated against a documented JSON Schema (packages/scanner/schema/report.schema.json, draft-07) checked into the repository, including negative tests that confirm it actually rejects invalid data. Field names match the scanner's real output exactly: schemaVersion, reportId (referred to as scanId), verdict, riskScore, scoreFormula, findings (each with a stable code), securityCategories, capabilities (consequential capabilities), mcpExposures (MCP servers/tools), capabilityChains, securityControlsDetected, bom (runtime bill of materials — the closest field to a general permissions/tools summary), notDetermined, and metadata.scannerVersion (also referred to as "ruleset version" — the scanner ships as one versioned unit with its detection rules, so there is no separately-versioned ruleset artifact). A comparisonSummary is included only when you supply a previous scan to compare against — a single scan has nothing to compare on its own.

The Compliance view is a real evidence-mapping engine (apps/web/src/lib/complianceMapping.ts) between canonical scan evidence and framework controls — OWASP LLM Top 10, NIST AI Risk Management Framework (GOVERN/MAP/MEASURE/MANAGE function citations), and SOC 2 Trust Services Criteria — not a re-render of finding labels. Every control gets exactly one of four statuses, and the wording "compliant", "certified", or "passed compliance" is never used anywhere in this view (enforced by a real test):

• GAP_IDENTIFIED — a real finding fired in this scan and its own citation implicates this control. Always traceable to that exact finding.
• EVIDENCE_OBSERVED — a narrow, explicitly-documented case: the scanner ran its complete check set for a security category and found nothing wrong (e.g. all 12 secret-detection checks found no exposed credential). Real, checked, negative-result evidence — always shown at medium confidence, never high, because a clean static scan still isn't a guarantee.
• NOT_ASSESSED — this scan produced no evidence either way for a real, known control. This is not the same as passing, and is shown explicitly rather than omitted.
• NOT_APPLICABLE — reserved for a control that is structurally impossible for static code analysis to ever assess, when one is genuinely determinable as such.

A control can never become GAP_IDENTIFIED or EVIDENCE_OBSERVED merely because a tag string exists somewhere — both require real backing evidence (a firing finding, or a genuinely strong category status), proven by a dedicated regression test (apps/web/test/complianceMapping.test.mjs) that specifically checks an unrelated strong category can never leak evidence onto a different control. This does not constitute a compliance certification of any kind.

Every report can carry a SHA-256 hash of its canonical evidence fields (verdict, score, findings, capabilities, MCP exposures, BOM, and related scan output — not presentation/sharing state like public/private visibility, which is expected to change after a scan). Recomputing the hash from the currently-stored evidence and comparing it to the stored hash tells you whether that data has changed since it was hashed.

What this proves: the evidence fields are byte-identical to what was hashed at scan time. What this does not prove: that the scan itself was accurate (a wrong scan hashes just as cleanly as a correct one), and it is not a cryptographic signature or a third-party-verifiable attestation — there is no private key or external timestamping authority involved. Anyone with write access to the underlying data store could recompute and overwrite the hash alongside the data, so this defends against accidental corruption or a partial/naive tamper, not a fully compromised backend acting deliberately. Agent Verify does not claim reports are "immutable" anywhere in the product — this is hash verification, described exactly as what it is.

Scan-duration numbers are measured, not asserted: packages/scanner/benchmark/benchmark.mjs runs the real scan() function 30 times against four representative synthetic fixture sizes (small ~200B, medium ~8KB, large ~100KB, very-large ~2.3MB — past the scanner's internal truncation limit) and reports median, p95, fastest, and slowest wall-clock time. As measured: a small agent file scans in a fraction of a millisecond; a 100KB file in single-digit milliseconds; a 2.3MB file (larger than any realistic single agent module, and past the point the scanner truncates for safety) in roughly 100–125ms. The scan computation itself is not the bottleneck in any realistic use of the product — network transfer and the CLI/API round-trip dominate end-to-end time, and those depend on your network conditions, not on the scanner.

A2SPA (execution-time authorization) findings check whether an agent's consequential actions are protected by: a cryptographic signature over the execution request, a per-request nonce (replay protection), a timestamp/expiry window, fail-closed behavior when verification fails, and a scoped (non-wildcard) permission model. Agent Verify detects whether this evidence is present in the submitted code — it does not itself sign or verify production traffic.

Each report carries an overall A2SPA status, deliberately conservative — mentioning "A2SPA" by name in a comment or string is never treated as evidence; only an actual signature/verification code pattern counts:

• Detected — all five controls show real code-pattern evidence.
• Partially detected — some but not all controls show evidence.
• Not detected — none of the five controls show evidence.
• Cannot determine — there wasn't enough execution context to assess anything.

When you scan the same agent again (matched by agent name, falling back to file name, within your own account), the report shows what changed since the previous scan: score delta, verdict change, new vs. resolved findings, capability and MCP exposure changes, and risk-relevant configuration changes. Findings are matched by their stable code only — never by title text or evidence content — so two unrelated findings are never falsely merged, and the same finding is still recognized as unchanged even if its evidence shifted slightly.

Reports don't only list what's wrong. A "Security Controls Detected" section lists real positive signals already present in the code (scoped permissions, human approval gates, audit logging, rate limiting, and others) — never awarded without matching evidence. Every report also includes a "What Agent Verify Could Not Determine" section stating the specific limits of static analysis for that scan (runtime-only behavior, dynamically granted permissions, external policy, production network ACLs, and similar).

Where detectable from the submitted content, each report shows: detected language, framework (LangChain, AutoGen, CrewAI, OpenAI, Anthropic, LlamaIndex, Haystack), platform, agent name, tool access level, credential exposure, memory persistence, audit logging, human approval gates, rate limiting, prompt-injection surface, and delegation scope. Fields Agent Verify cannot determine from the submitted content are shown as Unknown or Not Detected — never guessed.

The CLI scans .js, .ts, .py, .json, .yaml, .yml, .md, .mjs, and .cjs files, skipping node_modules, .git, build output, and coverage directories. The web scanner accepts pasted text or a single uploaded file of the same types.

• Agent Verify performs static analysis of submitted content. It does not execute your agent, and it cannot see behavior that only exists at runtime or in code that wasn't submitted.
• Findings are evidence-based and heuristic. A clean report is not a guarantee of safety, and a finding is not proof of exploitation — both should be read as signals, not deterministic facts.
• NOT ASSESSED means there wasn't enough agent execution context to issue a verdict — it must never be read as, or presented as, VERIFIED.
• Dashboard (browser) scans are self-reported: the report is written by the signed-in browser, not attested by the server. CLI/API scans are computed and saved server-side.
• Detected secrets are redacted in every report; Agent Verify does not store or display live credential values.

• "API key required" — pass --key or set AGENTVERIFY_API_KEY.
• "Invalid or unauthorized Agent Verify API key" — the key is missing, malformed, disabled, or was regenerated (which revokes the old key). Generate a fresh key from the dashboard.
• HTTP 429 / "Monthly scan quota exceeded" — you've used your plan's scans for the current month. Upgrade to Pro or wait for the next billing month.
• No agent files found — the directory has no files with a supported extension; scan a specific file with --file.
• Exit code 3 in CI — this is an execution error (key/network/file), not a failed agent. Fix the pipeline configuration, not the agent.

Does Agent Verify run my agent's code? No. Analysis is static; your code is inspected, not executed.

Can a report be forged or edited after it's created? A report's verdict, score, and findings are immutable once saved — only visibility (public/private) can change afterward.

What happens to scans over my plan's limit? The API returns 429 and no scan is performed or counted against your quota.

Is the scanner open source? The web app, CLI, and Worker API are open source. The detection engine (@agentverify/scanner) is proprietary and lives only in the private build behind the API — the published CLI package is a pure HTTP client and does not bundle it.

Does the CLI send my code anywhere? Yes — each scanned file's content is sent to the Agent Verify API over HTTPS to be analyzed server-side; nothing is scanned locally by the CLI itself. Don't scan files containing production secrets you aren't comfortable transmitting.