# Sentinel Audit
**A repository audit that verifies its own findings — and keeps the ones it disproves.**
[](https://github.com/Burtson-Labs/sentinel-audit/actions/workflows/ci.yml)
[](LICENSE)
[](docs/profiles.md)
[](package.json)
[](package.json)
A Burtson Labs open-source project
---
Most security reviews — human or AI — hand you a list of assertions. Sentinel hands
you a list of *checked* assertions, each labelled with how it was checked and what
happened: `proof-confirmed`, `pattern-confirmed`, `plausible`, `refuted`, or
`triaged-out`. Where it can, it writes a runnable proof script, executes it against
your real code, and saves the script next to the report so you can re-run it
yourself.
> A refuted finding is a feature. Sentinel keeps them in the report, because "we
> checked this and your protection works" is information you paid for too — and
> because a tool that only ever reports problems cannot be trusted about the
> problems it reports.
```
proof-confirmed 4 (an executed proof demonstrated it)
pattern-confirmed 19 (the pattern re-matched; exploitability unproven)
plausible 5 (reasoned, unproven)
refuted 1 (an executed check disproved it)
triaged out 1 (noise, with reasons)
score 5.4/10 (acceptable / moderate) · security 5.7/10 · gate PASS-WITH-CONDITIONS
confirmed share of live findings: 82% · proven by execution: 14%
```
- **Nothing says "confirmed" unless something ran — and the two ways of running
are not the same badge.** A generated script that exercised your real modules and
returned a verdict earns `proof-confirmed`. Re-reading the artefact from disk and
re-establishing the factual claim earns `pattern-confirmed`: the pattern is real,
the exploit is unproven. Everything else stays `plausible`, and the report says
so next to the finding.
- **The confidence number is derived, not authored.** It comes out of the
verification record. A model cannot set it, and neither can a reviewer.
- **Zero runtime dependencies.** A tool whose pitch is "your supply chain is the
attack surface" should not arrive with one. The CLI parses its own arguments.
- **The model pass is additive and never load-bearing.** With no provider
reachable the scan completes, and every artefact states that it did not run
along with the coverage cost.
## Quickstart
```bash
# Node >= 22.6 required — the proof runner uses native TypeScript type stripping
# from npm: installs the `sentinel` command
npm i -g @burtson-labs/sentinel-audit
sentinel scan ../some-repo --out ./results
# or from source
git clone https://github.com/Burtson-Labs/sentinel-audit.git
cd sentinel-audit && pnpm install && pnpm build
# audit any repository
node dist/cli.js scan ../some-repo --out ./results
# pick a standards vocabulary and the artefacts you want
node dist/cli.js scan ../some-repo --profile cwe-top-25 --format md,html,json,sarif
# deterministic only: no model calls, seconds rather than minutes
node dist/cli.js scan ../some-repo --no-llm
```
Exit codes: `0` clean · `1` findings at or above the conditional threshold · `2`
the profile gate blocked.
From a source checkout, `npm link` puts `sentinel` on your `PATH` if you want the
short command.
Proofs execute the audited repository's code, so they run in a Docker or Podman
container with no network, a read-only filesystem, no Linux capabilities and an
empty environment. With no container runtime running, Sentinel skips proofs rather
than run them on your machine, and the report says so. See
[SECURITY.md](SECURITY.md#running-against-untrusted-code).
### Artefacts
| File | What it is |
|---|---|
| `REPORT.md` | Findings with evidence, verification record, standards mapping and a fix plan each |
| `REPORT.html` | Self-contained page, dark/light, filterable by status. No network requests at all |
| `findings/*.json` | One file per finding. Superset of the common consultant finding schema — see below |
| `findings/index.json` | Manifest with counts, provenance, and the schema-validation result |
| `report.sarif` | SARIF 2.1.0 for GitHub code scanning. Refuted findings travel as `suppressions`, not deletions |
| `CONFIDENCE.md` | Scored assessment: overall, security posture, gate decision, risk matrix, remediation phases |
| `COVERAGE.md` | What was **not** examined and why — derived from what actually ran |
| `proofs/*.mjs` | The generated proof scripts. Re-runnable, unchanged, by hand |
| `scan-context.json` | The run record: provenance, collector results, gaps. Summaries only — never a copy of your source |
## Why this is different
| | Consultant-style review (human or LLM) | sentinel-audit |
|---|---|---|
| **Finding status** | Asserted. You cannot tell a demonstration from a suspicion | Every finding carries `status` + a `verification` block stating method, checks run, and result — and a demonstration (`proof-confirmed`) never wears the same badge as a re-matched pattern (`pattern-confirmed`) |
| **Exploitability** | "Assessed by code reading" | A generated script runs the real code and returns a verdict; behavioural claims cannot reach any confirmed status without one |
| **Wrong findings** | Quietly dropped, or shipped as real | `refuted`, with the proof that disproved them, kept in the report |
| **Scanner noise** | Either all of it or none of it | Triaged out with a reason *and* a cited `file:line`. An uncited dismissal is rejected by the pipeline |
| **Confidence number** | Authored by the reviewer | Derived from the verification record. A model cannot set it |
| **Standards mapping** | One firm's private catalogue | Selectable profile: `owasp-asvs`, `cwe-top-25`, `generic-enterprise`, or your own JSON |
| **Dependency advisories** | Whatever `npm audit` printed | Cross-checked against the **installed** version (pnpm store included); an advisory that does not apply is refuted |
| **Coverage claims** | A paragraph of prose | A table derived from which collectors ran and which tools were missing |
| **Remediation** | A recommendation | A structured fix plan, and for the mechanical subset, `sentinel fix` drives an agent to implement it on a branch and open a PR |
| **Reproducibility** | Re-read everything | Stable fingerprints; a second run over an unchanged tree produces identical ids |
### The verification ladder
A finding earns its status; it is never assigned one. There are two ways for a
check to agree with a claim, they are worth very different amounts, and they get
different names — because calling both of them "confirmed" sells the cheap one at
the price of the expensive one.
- **`proof-confirmed`** — a generated script exercised your real modules and
returned a `vulnerable` verdict. The behaviour was *demonstrated*. This is the
only status that claims exploitability, and it is **required** for any claim
about behaviour.
- **`pattern-confirmed`** — a static or lexical assertion was re-established from
disk at report time, independently of the rule pass: the construct *is* at that
line, the policy *is* absent from every file it could live in, the vulnerable
version *is* the one installed. That makes the **pattern** a verified fact about
your code. It does **not** establish that the pattern can be exploited. A
behavioural claim can never reach this status — it stays `plausible`.
- **`plausible`** — reasoned from reading. Honest and still useful, but you should
confirm it before funding the fix. The schema *forbids* promoting a code-read
finding to either confirmed status, which is why model-proposed findings top out
here.
- **`refuted`** — an executed check disproved it. Kept, with the reason.
- **`triaged-out`** — suppressed as noise, with a reason and a cited location.
The CLI summary, `REPORT.md`, `REPORT.html`, `CONFIDENCE.md`, SARIF (`precision`
`very-high` vs `high`) and `findings/*.json` all count the two separately, and the
derived confidence number is materially lower for a pattern than for a proof.
`verification.state` repeats the precise status inside the verification block;
`verification.class` folds it back to the coarse `confirmed | plausible | refuted |
triaged-out` vocabulary, so a consumer that only wants "confirmed vs not" — or one
written before this split — keeps working unchanged. `findings/index.json` and the
SARIF run properties carry `proofConfirmed`, `patternConfirmed` **and** their sum as
`confirmed` for the same reason.
Two asymmetries keep the verdicts honest:
1. **Stub asymmetry.** Real application modules often cannot load outside their
bundler. The proof harness substitutes inert objects for unresolvable imports
and records what it replaced. A `safe` verdict under stubbing is still
meaningful — a stub cannot *add* escaping. A `vulnerable` verdict under
stubbing is downgraded to inconclusive, because one of the stubs might have
been the sanitiser.
2. **Proofs can only strengthen factual claims.** If a data-flow proof fails to
reproduce a write that re-assertion already established from the source, the
finding stands. A proof that cannot load a module is never treated as a
refutation.
## CLI
```
sentinel scan 