Agent Verify

Documentation

Agent Verify Docs

Agent Verify inspects an AI agent or agent package for identity, permissions, tools, execution controls, secrets, runtime configuration, and dependency risk, then issues a VERIFIED or NOT VERIFIED result with evidence. This page documents the web scanner, CLI, npm package, and API as they actually exist today — every command and endpoint here is real and testable.

Overview

Every scan produces one of three verdicts:

  • VERIFIED — sufficient execution context was assessed and the required trust controls passed.
  • NOT VERIFIED — sufficient execution context was assessed and one or more security or protocol controls failed.
  • NOT ASSESSED — the submitted content did not contain enough agent execution context (tools, permissions, actions) to make a verification decision. This is not a security failure — it means Agent Verify could not evaluate the input.

Findings are split into two categories: A — A2SPA Protocol Compliance (signature, nonce, timestamp, fail-closed, and scope evidence for execution-time authorization) and B — General Agent Security (secrets, tool access, human approval gates, audit logging, rate limiting, prompt injection surface, and related runtime risk).

Quick Start

Two ways to run a scan:

  1. Sign in and use the web scanner to paste or upload agent code/config for a private report.
  2. Install the CLI and scan from your terminal or CI pipeline:
npm install -g agentverify
agentverify scan ./agents --key av_your_key

Get an API key from Dashboard → API access after signing in.

Web Scanner

The dashboard scanner runs the same @agentverify/scanner engine as the CLI and API against pasted code or an uploaded file, then saves a private report to your account. The scan runs on Agent Verify's server (the same /v1/scan endpoint the CLI and API use), and the server writes the report, so dashboard, CLI and API reports carry the same evidence. Pro and Team plans also get Fix & re-check on the report: edit the code (or let AI fix it) and re-check it instantly without using a monthly scan. See Limitations.

CLI

agentverify scan [dir] --key <api-key>
agentverify scan --file <file> --key <api-key>
agentverify --help
agentverify --version

Common flags:

  • --key, -k — API key (or set AGENTVERIFY_API_KEY)
  • --file, -f — scan a single file instead of a directory
  • --json — machine-readable output
  • --markdown — Markdown summary, useful for PR comments
  • --ci — concise output with strict exit codes
  • --allow-not-assessed — in CI mode, do not fail solely because content is NOT_ASSESSED
  • --profile <id> — assess a skill directory against an assessment profile instead of running the standard scan; see OWASP Agentic Skills Profile

CI exit codes:

CodeMeaning
0All scanned files verified
1NOT VERIFIED — a real security/verification failure
2NOT ASSESSED — insufficient evidence for a verdict
3Execution error — bad key, network failure, missing file (not a security finding)

npm Installation

Package: agentverify on the public npm registry.

npm install -g agentverify
# or run without installing
npx agentverify scan . --key av_your_key

Requires Node.js 18 or later. The package ships both CommonJS and ESM builds and TypeScript types; it has no dependency on this monorepo — it works standalone in any project.

Authentication

Sign up with email/password or Google. Forgot your password? Use Forgot password? on the sign-in form — if an account exists for that email, a reset link is sent. For account-security reasons, the response is identical whether or not the email is registered.

API keys are generated per-account from Dashboard → API access. Regenerating a key immediately revokes the previous one.

Workspaces

A workspace (also called an organization) is a shared context multiple people can belong to, with its own members, roles, audit log, and integration settings — separate from your personal account, which continues to work exactly as before if you never create one. Scans, reports, and API keys stay tied to your own account either way; a workspace adds a shared view on top: who else can see what, and a record of what happened.

Why a team uses one: a solo account has no concept of teammates. A workspace lets a security lead invite the rest of the team, see every member's scan activity in one audit log, and (as an Owner or Admin) enforce who can invite others, change policies, or configure integrations — without sharing a single login.

Roles

RoleScan, view & share reportsInvite / manage membersPolicies, integrations & webhooksBilling & delete workspace
OwnerYesYes, incl. changing rolesYesYes
AdminYesYes, cannot change rolesYesNo
MemberYesView onlyView onlyNo
ViewerView onlyView onlyView onlyNo

Every role check happens server-side against the role actually stored for your account in that workspace — the dashboard only hides buttons you can't use, it never grants anything on its own.

Creating a workspace

From Dashboard → Workspace, enter a name and select Create workspace. You become its Owner immediately. There's no limit on how many workspaces you can belong to, and no migration or setup required — your existing scans and reports are unaffected.

Managing members

An Owner or Admin invites a teammate by email from the Members tab and assigns their role. The invited person must already have an Agent Verify account with that email — invites don't currently create an account for someone who hasn't signed up. Roles can be changed or a member removed at any time (except the Owner, who can't be demoted or removed through this screen — a workspace always keeps its Owner).

API keys and workspaces

Your API key (from Dashboard → API access) is tied to your personal account, not to a specific workspace — there is no separate workspace-owned key today. What a workspace does add: a scan request can optionally include an organizationId, and if your role in that workspace has scan permission, that scan is attributed to the workspace and appears in its Audit Log. This is the real, current mechanism for connecting scan activity to a team — workspace-issued/rotatable API keys are not implemented yet.

Audit log

The Audit Log tab is a server-written record of what actually happened in the workspace — never something a client can fabricate or backdate. Every entry the log can show today is one of: a scan completing (with its verdict), an attestation being issued, a policy being applied to a scan, a member being added or removed, a role being changed, or a webhook being created or disabled. It's searchable and filterable by action type directly in the tab.

Integrations

The Integrations tab lists the real ways to connect Agent Verify to your workflow: GitHub Actions and the CLI for CI/CD, and the REST API for custom tooling — all available today. Webhooks are the newest addition; see the next section for exactly what that means right now.

Webhooks

A webhook lets Agent Verify notify another application automatically when something happens in your workspace — instead of that application having to repeatedly ask "did anything change yet?", Agent Verify would push an event to a URL you provide, the moment it happens. Owners and Admins configure webhooks from Dashboard → Workspace → Webhooks.

Live since 2.0

Automatic delivery runs in production: a scheduled job sends due deliveries about once a minute, with retries as described below. Every event is also visible in the Audit Log.

How delivery works

When one of the events below happens, Agent Verify inserts a durable delivery record (Cloudflare D1) for every active webhook subscribed to it, then a Cron Trigger sweeps for due deliveries roughly every minute and sends a real signed POST. This is not fire-and-forget: every attempt's outcome (HTTP status, response snippet, error, timestamp) is written back to that same durable record, which is what powers the delivery history and retry described below.

Supported events

A webhook can be subscribed to any of these seven event types — the exact set the API accepts, no more:

EventFires when
SCAN_COMPLETEDA scan attributed to the workspace finishes, regardless of verdict.
VERIFICATION_PASSEDThat scan's verdict was VERIFIED.
VERIFICATION_FAILEDThat scan's verdict was NOT_VERIFIED.
ATTESTATION_ISSUEDA signed attestation was produced for the scan.
POLICY_APPLIEDA policy profile was evaluated against the scan.
MEMBER_ADDEDSomeone was invited into the workspace.
ROLE_CHANGEDA member's role was changed.

Payload & signature

This is exactly what your endpoint receives once delivery is live in production:

POST <your endpoint URL>
Content-Type: application/json
Agent-Verify-Signature: t=1735689600,v1=5f4dcc3b5aa765d61d8327deb882cf99...

{
  "eventId": "3fa2c1e0-...",
  "eventType": "VERIFICATION_FAILED",
  "organizationId": "org_abc123",
  "timestamp": "2026-09-02T13:49:12.000Z",
  "data": { "scanId": "REPORT-qhhic9fbkv", "verdict": "NOT_VERIFIED", "score": 30, "fileName": "agent.ts" }
}

The Agent-Verify-Signature header carries t=<unix-seconds>,v1=<hex-encoded HMAC-SHA256> — the same shape Stripe uses for its own webhooks. The signature is computed over ${timestamp}.${rawRequestBody} using your webhook's secret (shown once, at creation — store it like any other credential). To verify a delivery:

const [t, v1] = signatureHeader.split(',').map(p => p.split('=')[1])
const expected = hmacSha256Hex(webhookSecret, `${t}.${rawBody}`)
if (!timingSafeEqual(v1, expected)) reject('invalid signature')
if (Math.abs(nowInSeconds() - Number(t)) > 300) reject('too old — possible replay')

Replay/freshness protection: implemented in the reference verifier — a signature older than 5 minutes is rejected as expired, so a captured, valid delivery can't be replayed indefinitely.

Expected HTTP behavior, retries & dead-letter

Your endpoint should verify the signature, then respond quickly with a 2xx status to acknowledge receipt. Anything else (a non-2xx status, a timeout, or a connection error) is treated as a failed attempt and retried with exponential backoff: 1 minute, 5 minutes, 30 minutes, 2 hours, then 12 hours. After 6 total attempts with no 2xx response, the delivery becomes a dead letter and stops retrying automatically.

Delivery history & manual retry

Every webhook in the Workspace → Webhooks tab has an expandable delivery history: status (Delivered / Retrying / Failed), attempt count, the last HTTP status or error, and (while retrying) when the next attempt is scheduled. A dead-lettered delivery can be manually retried — an Owner or Admin's explicit decision gives it a full fresh attempt cycle, not just one more try. Delivery history never includes the event payload or the signing secret, only attempt/outcome metadata.

Disabling a webhook

An Owner or Admin can disable a webhook at any time from the Webhooks tab — it stops being active immediately, including any deliveries already queued and waiting for their next retry (disabling is checked again right before every attempt, not just when a delivery is first queued). There is currently no way to permanently delete a webhook or rotate its secret; if a secret is compromised, disable that webhook and create a new one.

Security recommendations

  • Always verify Agent-Verify-Signature before trusting a payload — never process an unsigned or incorrectly-signed request.
  • Reject anything outside the 5-minute timestamp window, even with a valid signature.
  • Use an HTTPS endpoint. Plain http:// is technically accepted, but a real destination should not accept unencrypted webhook traffic.
  • Treat your webhook secret like any other credential — it's shown once, at creation, and never displayed again.
  • A webhook endpoint can never point at a private/internal address (localhost, RFC1918 ranges, cloud metadata endpoints) — Agent Verify rejects those at creation time.

API

Authenticated scan endpoint (used by the CLI and SDK):

POST https://agentverify-api.agentverify.workers.dev/v1/scan
Authorization: Bearer av_your_key
Content-Type: application/json

{ "content": "<agent code or config>", "fileName": "agent.ts" }

Responses include the full scan result (verdict, riskScore, findings, categoryScores, bom, threatCategories, metadata) plus reportId, saved, and reportUrl when a report was persisted. A 429 response means your plan's monthly scan quota (10 scans/month on Free, 100/month on Pro) has been reached. A 401 means the key is missing, malformed, disabled, or revoked.

The Worker also serves a live SVG badge at GET /v1/badge/:reportId for saved reports, and the billing endpoints used by the dashboard (/v1/billing/status, /checkout, /portal) which require a signed-in Firebase session, not an API key.

OWASP Agentic Skills Profile

Assess an agent skill package against the OWASP Agentic Skills Top 10. Profile owasp-agentic-skills-2026 implements three controls — AST02 Supply Chain Compromise, AST03 Over-Privileged Skills, and AST04 Insecure Metadata. AST01 and AST05–AST10 are always returned as NOT_ASSESSED, which is not a pass. See a sample assessment.

Assessed against the OWASP Agentic Skills Top 10 (CC-BY-SA-4.0) at a pinned upstream commit. This is not an OWASP certification or endorsement.

Statuses. Each control, and each of its checks, is exactly one of:

  • EVIDENCE_OBSERVED — evidence supporting the expected security property was observed.
  • GAP_IDENTIFIED — evidence of a security gap was observed.
  • NOT_ASSESSED — there was not enough evidence to reach a determination.

Versions. 1.0.0-alpha.1 is the version of the OWASP profile mapping, and 1.1.0 is the assessment schema. Neither is the Agent Verify product version.

CLI — scans a directory and prints the assessment (--json prints the service response unmodified). A GAP_IDENTIFIED result is a normal result and exits 0; execution errors exit 3. The CLI does not request signing.

agentverify scan ./my-skill --profile owasp-agentic-skills-2026 --key av_your_key

API — the same /v1/scan endpoint with a profile and the package files. Add "attest": true for a signed assessment.

POST https://agentverify-api.agentverify.workers.dev/v1/scan
Authorization: Bearer av_your_key
Content-Type: application/json

{
  "profile": "owasp-agentic-skills-2026",
  "files": [{ "path": "SKILL.md", "content": "..." }],
  "attest": true
}

A 200 response is { profile, assessment, attestation, saved: false }. attestation is null unless you asked for signing; a signed response also carries bundleVersion ("1.0.0"). Profile assessments are not saved as reports.

Signing. A signature proves this exact assessment was issued by Agent Verify and has not been altered. It does not independently prove that every finding is correct or complete. The signing key is published on the signing key page and at GET /v1/profile-attestation/public-key; verify against that key, not the key inside the bundle.

Metering. A completed profile assessment uses one scan from your monthly quota, signed or unsigned. A request rejected before assessment (malformed, too large, unsafe package, over quota) uses none. If the assessment completes but signing then fails, that one scan is still used, and a retry is a new request.

Errors.

StatusMeaning
400 / 413The request could not be read, or the package is too large.
401Missing, malformed, or revoked credentials.
422The package, or the assessment, could not be admitted — for example an unsafe file path. A signed request that cannot be signed is also refused here rather than returned unsigned.
429Monthly scan quota reached, or too many profile assessments this hour.
503Signing is currently unavailable. Retry, or request an unsigned assessment.
500Internal failure. The response never includes internal detail.

CI/CD & GitHub Actions

Run Agent Verify as a required PR check:

- uses: AI-Blockchain-Ventures/agentverify@v1.4.0
  with:
    path: ./agents
    api-key: ${{ secrets.AGENTVERIFY_API_KEY }}
    format: text

Or call the CLI directly in any workflow:

- run: npx --yes agentverify scan ./agents --ci
  env:
    AGENTVERIFY_API_KEY: ${{ secrets.AGENTVERIFY_API_KEY }}

Add the job to branch protection so a non-zero exit blocks merge. Full reference: docs/github-action.md.

Scan Results

Every result carries schemaVersion, a stable machine-safe verdict (VERIFIED / NOT_VERIFIED / NOT_ASSESSED), a riskScore (0–100), riskLevel, a confidence score, category scores for Protocol Compliance and Security Controls, a findings array, a Runtime Bill of Materials, and scan metadata (scanner version, timestamp, duration, detected language/framework).

Security Score

The score starts at 100 and is reduced by weighted findings: −20 per critical finding, −10 per high, −5 per medium (low findings don't subtract points but still appear in the report). Broad/wildcard tool or permission access caps the score at 65 regardless of other findings. A VERIFIED verdict additionally requires zero critical findings, at most one high finding, no broad permissions, and request-authorization checks that match what the agent can do. A read-only agent (it reads and answers, but changes nothing) needs nothing extra: missing request signing is shown as a low-severity suggestion. An agent that takes actions (edits files or data, sends messages or emails, runs commands) needs signed, replay-protected, fail-closed requests or a human approval step. A high-impact agent (moves money, issues refunds, deploys, deletes users, or holds broad permissions) needs signed requests, or human approval plus an audit log. Any signing method counts (HMAC, JWT, A2SPA or your own); A2SPA is not required.

This is a heuristic, evidence-based score, not a guarantee. See Limitations.

Findings

Each finding includes a stable code, severity (critical/high/medium/low), category (A or B), a plain-language description of what was found and why it matters, a line number when locatable, a plain-English "what an agent could do because of this" note, a recommended fix, an evidence type, and — where relevant — mappings to OWASP LLM Top 10, NIST AI RMF, and SOC 2 controls. Findings that reference a detected secret show a redacted preview (e.g. api_key: "sk_l****" (redacted)) — the live value is never included in a report. Named, higher-confidence patterns are detected for OpenAI/Anthropic/AWS/GitHub/Stripe/Slack keys, private key blocks, and database URLs with embedded credentials, in addition to a generic key/password/token pattern.

Every finding also carries an evidence type so a heuristic guess is never presented as an established fact:

  • Definite — a concrete pattern was matched (a literal wildcard permission, a literal secret literal).
  • Heuristic — inferred from the absence of an expected pattern, or contextual keyword co-occurrence. Can under- or over-detect if a control is implemented in a way the scanner doesn't recognize.
  • Informational — a neutral observation, not itself good or bad.

Capabilities, MCP, and Blast Radius

Capabilities answer "what could this agent actually do if compromised?" — read/write/delete files, execute commands, deploy software, access/modify databases, send email/messages, transfer money, issue refunds, touch crypto wallets, reach cloud infrastructure, create/delete users, or call external APIs. Each capability is derived only from concrete evidence in the submitted content, with its own confidence level — never guessed from a tool name alone.

MCP (Model Context Protocol) detection identifies MCP server configuration, known server/tool categories (filesystem, shell, database, GitHub, Slack, email, browser automation, payments, docker, memory), and builds an Agent → MCP Server → Tool → Potential Action chain when there is real evidence. An MCP integration with an unidentified server is reported as exactly that — "Unknown MCP server" — never invented.

Potential Blast Radius surfaces dangerous combinations of capabilities that matter more than any single permission — e.g. filesystem write + shell execution, or a payment capability with no visible destination constraint or human approval step. This never claims an exploit exists, only that the combination could allow the stated impact if the agent is compromised or misused.

Verification Check Catalog

Agent Verify runs 124 distinct, independently-tested verification checks (48 of them new in v2.0 — MCP tool poisoning, hidden characters, tool shadowing, risky MCP server setups, payments and deletions without limits or approval, personal data sent to models, retrieval without a per-customer filter, AI agents in CI, and framework packs for LangChain, LangGraph, CrewAI, AutoGen, LlamaIndex, the OpenAI Agents SDK, the Claude Agent SDK and Claude Code, the Vercel AI SDK, Semantic Kernel, smolagents and Google ADK). — not a marketing round number. Each check has a stable ID (e.g. AV-SECRET-003, AV-MCP-005), a category, severity, detection type, and its own regression test asserting it actually fires. Every check the scanner can produce is cross-checked in both directions against a real test suite: no catalog entry exists without a working code path, and no working code path exists without a catalog entry. Every check has its own help page with what it means and how to fix it: see the check library.

CategoryChecks
Runtime29
Secrets26
MCP18
Permissions12
Human oversight10
Network7
Tools6
Dependencies5
Execution authorization4
Identity4
Auditability3

Beyond the 124 checks, the scanner separately runs 19 capability detectors (what an agent can do), 10 MCP tool classifiers, 8 capability-chain ("blast radius") combination rules, and 14 threat-category assessments — these are classification/context systems, not pass/fail checks, and are counted separately so the 124 stays an honest number of real, distinct checks rather than an inflated one.

Risk Taxonomy

Findings are tagged with one of 11 technical security categories (Identity, Permissions, Tools, MCP, Execution Authorization, Secrets, Runtime, Network, Dependencies, Auditability, Human Oversight) — this is the ground truth every finding, check, and report is built from. For a non-technical audience, those 11 categories group into 7 executive families, derived from the same tags (never assigned separately, so the two views can never contradict each other):

  • Identity & Access — Identity, Permissions
  • Tools & Capabilities — Tools, MCP
  • Execution Security — Execution Authorization
  • Secrets & Data — Secrets
  • Runtime & Network — Runtime, Network
  • Supply Chain — Dependencies
  • Oversight & Auditability — Auditability, Human Oversight

Report Types

Every report is built from one canonical scan result — the same verdict, score, and findings, rendered six different ways for six different audiences, all from the exact same evidence, each in its own component (apps/web/src/components/report/views/). Switch between them with the tabs at the top of any report:

  • Executive — a 60-second read for a CEO, CISO, CTO, buyer, or investor: an explicit deployment recommendation (DEPLOY / DEPLOY WITH CONDITIONS / DO NOT DEPLOY / NOT ASSESSED, derived only from the verdict and critical/high finding counts — never a separately-invented judgment), the specific deployment blockers when it isn't a clean DEPLOY, top risks, what to fix first, and a posture trend (Improving / Degrading / Unchanged) computed from your real scan history — shown honestly as "No prior scan available" rather than a guess when there is nothing to compare against.
  • Security — answers "where can this agent hurt us?": attack surface, capabilities/privileges, MCP/tool exposure, capability chains (real evidence-based combinations, surfaced as "top attack paths" — never a fabricated one), credential/secret exposure, human-approval and authorization gaps, and a prioritized remediation plan.
  • Developer — action-oriented, for every finding: rule/check ID, severity, confidence, file/line (explicitly "Not available" when the content doesn't make one locatable, never guessed), technical reason, exact remediation, example fix code when one is real, how to verify the fix (re-scan and confirm that rule ID no longer fires), and a machine-readable remediation metadata block for tooling.
  • Compliance — a real evidence-mapping engine, not a re-render of finding labels. See Compliance Mapping below.
  • AI / JSON — the stable, schema-versioned machine-readable report for CI/CD, SIEM, and API clients.
  • Full Technical — essentially everything Agent Verify knows about the scan in one place, without exposing the detection engine's internal implementation.

All six are proven distinct from the same evidence object by a real automated test (apps/web/test/sixViewContract.test.mjs), which server-renders every view and asserts each one shows the fields specific to it and none of the fields that belong only to another view.

Reports

A report page shows the verdict, score, category breakdown, threat-category assessment, prioritized fix plan, individual findings with evidence, compliance mapping, and the Runtime Bill of Materials. Reports created via the CLI/API are stored separately from dashboard (browser) reports so their provenance (source: cli vs dashboard) is always visible on the report itself.

Sharing Reports

Reports are private by default. The owner makes a report public with Make public & copy link at the top of the report, which makes it reachable at its canonical URL without signing in; Make private immediately revokes that access. Anyone who can edit the report can regenerate its content by re-scanning, which produces a new report ID and URL — the old link is unaffected. Password-protected sharing is not implemented yet; the UI will not claim it works until a server can verify a password before releasing report content.

JSON Output

agentverify scan ./agents --json > results.json

Produces { results: [...], summary: { total, verified, notVerified, notAssessed, errors } } for CI systems, dashboards, or custom tooling.

JSON Schema

The AI/JSON report view is validated against a documented JSON Schema (packages/scanner/schema/report.schema.json, draft-07) checked into the repository, including negative tests that confirm it actually rejects invalid data. Field names match the scanner's real output exactly: schemaVersion, reportId (referred to as scanId), verdict, riskScore, scoreFormula, findings (each with a stable code), securityCategories, capabilities (consequential capabilities), mcpExposures (MCP servers/tools), capabilityChains, securityControlsDetected, bom (runtime bill of materials — the closest field to a general permissions/tools summary), notDetermined, and metadata.scannerVersion (also referred to as "ruleset version" — the scanner ships as one versioned unit with its detection rules, so there is no separately-versioned ruleset artifact). A comparisonSummary is included only when you supply a previous scan to compare against — a single scan has nothing to compare on its own.

Compliance Mapping

The Compliance view is a real evidence-mapping engine (apps/web/src/lib/complianceMapping.ts) between canonical scan evidence and framework controls — OWASP LLM Top 10, NIST AI Risk Management Framework (GOVERN/MAP/MEASURE/MANAGE function citations), and SOC 2 Trust Services Criteria — not a re-render of finding labels. Every control gets exactly one of four statuses, and the wording "compliant", "certified", or "passed compliance" is never used anywhere in this view (enforced by a real test):

  • GAP_IDENTIFIED — a real finding fired in this scan and its own citation implicates this control. Always traceable to that exact finding.
  • EVIDENCE_OBSERVED — a narrow, explicitly-documented case: the scanner ran its complete check set for a security category and found nothing wrong (e.g. all 12 secret-detection checks found no exposed credential). Real, checked, negative-result evidence — always shown at medium confidence, never high, because a clean static scan still isn't a guarantee.
  • NOT_ASSESSED — this scan produced no evidence either way for a real, known control. This is not the same as passing, and is shown explicitly rather than omitted.
  • NOT_APPLICABLE — reserved for a control that is structurally impossible for static code analysis to ever assess, when one is genuinely determinable as such.

A control can never become GAP_IDENTIFIED or EVIDENCE_OBSERVED merely because a tag string exists somewhere — both require real backing evidence (a firing finding, or a genuinely strong category status), proven by a dedicated regression test (apps/web/test/complianceMapping.test.mjs) that specifically checks an unrelated strong category can never leak evidence onto a different control. This does not constitute a compliance certification of any kind.

Report Integrity

Every report can carry a SHA-256 hash of its canonical evidence fields (verdict, score, findings, capabilities, MCP exposures, BOM, and related scan output — not presentation/sharing state like public/private visibility, which is expected to change after a scan). Recomputing the hash from the currently-stored evidence and comparing it to the stored hash tells you whether that data has changed since it was hashed.

What this proves: the evidence fields are byte-identical to what was hashed at scan time. What this does not prove: that the scan itself was accurate (a wrong scan hashes just as cleanly as a correct one), and it is not a cryptographic signature or a third-party-verifiable attestation — there is no private key or external timestamping authority involved. Anyone with write access to the underlying data store could recompute and overwrite the hash alongside the data, so this defends against accidental corruption or a partial/naive tamper, not a fully compromised backend acting deliberately. Agent Verify does not claim reports are "immutable" anywhere in the product — this is hash verification, described exactly as what it is.

Benchmark Methodology

Scan-duration numbers are measured, not asserted: packages/scanner/benchmark/benchmark.mjs runs the real scan() function 30 times against four representative synthetic fixture sizes (small ~200B, medium ~8KB, large ~100KB, very-large ~2.3MB — past the scanner's internal truncation limit) and reports median, p95, fastest, and slowest wall-clock time. As measured: a small agent file scans in a fraction of a millisecond; a 100KB file in single-digit milliseconds; a 2.3MB file (larger than any realistic single agent module, and past the point the scanner truncates for safety) in roughly 100–125ms. The scan computation itself is not the bottleneck in any realistic use of the product — network transfer and the CLI/API round-trip dominate end-to-end time, and those depend on your network conditions, not on the scanner.

A2SPA Checks

A2SPA (execution-time authorization) findings check whether an agent's consequential actions are protected by: a cryptographic signature over the execution request, a per-request nonce (replay protection), a timestamp/expiry window, fail-closed behavior when verification fails, and a scoped (non-wildcard) permission model. Agent Verify detects whether this evidence is present in the submitted code — it does not itself sign or verify production traffic.

Each report carries an overall A2SPA status, deliberately conservative — mentioning "A2SPA" by name in a comment or string is never treated as evidence; only an actual signature/verification code pattern counts:

  • Detected — all five controls show real code-pattern evidence.
  • Partially detected — some but not all controls show evidence.
  • Not detected — none of the five controls show evidence.
  • Cannot determine — there wasn't enough execution context to assess anything.

Scan-to-Scan Comparison

When you scan the same agent again (matched by agent name, falling back to file name, within your own account), the report shows what changed since the previous scan: score delta, verdict change, new vs. resolved findings, capability and MCP exposure changes, and risk-relevant configuration changes. Findings are matched by their stable code only — never by title text or evidence content — so two unrelated findings are never falsely merged, and the same finding is still recognized as unchanged even if its evidence shifted slightly.

Security Controls Detected

Reports don't only list what's wrong. A "Security Controls Detected" section lists real positive signals already present in the code (scoped permissions, human approval gates, audit logging, rate limiting, and others) — never awarded without matching evidence. Every report also includes a "What Agent Verify Could Not Determine" section stating the specific limits of static analysis for that scan (runtime-only behavior, dynamically granted permissions, external policy, production network ACLs, and similar).

Runtime Bill of Materials

Where detectable from the submitted content, each report shows: detected language, framework (LangChain, AutoGen, CrewAI, OpenAI, Anthropic, LlamaIndex, Haystack), platform, agent name, tool access level, credential exposure, memory persistence, audit logging, human approval gates, rate limiting, prompt-injection surface, and delegation scope. Fields Agent Verify cannot determine from the submitted content are shown as Unknown or Not Detected — never guessed.

Supported Formats

The CLI scans .js, .ts, .py, .json, .yaml, .yml, .md, .mjs, and .cjs files, skipping node_modules, .git, build output, and coverage directories. The web scanner accepts pasted text or a single uploaded file of the same types.

Limitations

  • Agent Verify performs static analysis of submitted content. It does not execute your agent, and it cannot see behavior that only exists at runtime or in code that wasn't submitted.
  • Findings are evidence-based and heuristic. A clean report is not a guarantee of safety, and a finding is not proof of exploitation — both should be read as signals, not deterministic facts.
  • NOT ASSESSED means there wasn't enough agent execution context to issue a verdict — it must never be read as, or presented as, VERIFIED.
  • Dashboard, CLI and API scans all run and are saved on the server. Fix & re-check results are not saved as reports until you save them as a new scan.
  • Detected secrets are redacted in every report; Agent Verify does not store or display live credential values.

Troubleshooting

  • "API key required" — pass --key or set AGENTVERIFY_API_KEY.
  • "Invalid or unauthorized Agent Verify API key" — the key is missing, malformed, disabled, or was regenerated (which revokes the old key). Generate a fresh key from the dashboard.
  • HTTP 429 / "Monthly scan quota exceeded" — you've used your plan's scans for the current month. Upgrade to Pro or wait for the next billing month.
  • No agent files found — the directory has no files with a supported extension; scan a specific file with --file.
  • Exit code 3 in CI — this is an execution error (key/network/file), not a failed agent. Fix the pipeline configuration, not the agent.

FAQ

Does Agent Verify run my agent's code? No. Analysis is static; your code is inspected, not executed.

Can a report be forged or edited after it's created? The database rules stop anyone, including the report's owner, from changing a saved report's verdict, score, or findings; only visibility (public/private) can change afterward. Each report also carries a SHA-256 hash of its evidence, so a changed report no longer matches its hash. Someone with direct administrator access to the database could still edit a report and recompute the hash, so the hash is a tamper check, not a signature.

What happens to scans over my plan's limit? The API returns 429 and no scan is performed or counted against your quota.

Is the scanner open source? The web app, CLI, and Worker API are open source. The detection engine (@agentverify/scanner) is proprietary and lives only in the private build behind the API — the published CLI package is a pure HTTP client and does not bundle it.

Is an OWASP Agentic Skills assessment a certification? No. It is an assessment against the OWASP Agentic Skills Top 10, using evidence Agent Verify observed in your package. Controls that are not implemented are shown as not assessed, which is not a pass, and the result is not an OWASP certification or endorsement.

Does the CLI send my code anywhere? Yes — each scanned file's content is sent to the Agent Verify API over HTTPS to be analyzed server-side; nothing is scanned locally by the CLI itself. Don't scan files containing production secrets you aren't comfortable transmitting.