conformly-scan

conformly-scan

Find the AI systems in your codebase and report them to your Conformly inventory.

npx conformly-scan --dry-run          # see what it finds, send nothing
npx conformly-scan --key=conf_xxx     # report findings to your workspace

Why an agent instead of integrations

Compliance platforms normally collect evidence through dozens of OAuth integrations, which means handing them credentials to your cloud, your code host and your identity provider — and getting that reviewed by your security team before you can even start.

This agent inverts it. It runs where your code already is, reads only what it needs to identify AI systems, and posts a list of findings. No OAuth app, no credentials leave your machine, nothing to review.

What is sent

Type	Example	What it is
ai.dependency	npm:openai	An AI library in a manifest, with its version range
ai.model	openai:gpt-4o	A model id quoted in your code
ai.service	aws:bedrock	A managed AI service your code actually calls
env.key	OPENAI_API_KEY	The name of an AI-provider env var
repo.doc	acme/api:MODEL_CARD.md	Path, size and SHA-256 of a governance doc
repo.meta	acme/api	Commit count and merge ratio over 90 days

What is never sent

• Secret values. Not from .env, not from anywhere. The env scanner splits each line at the first = and discards the right-hand side immediately — the value never reaches a variable that outlives the loop iteration.
• File contents. Governance documents are reported as path + size + SHA-256. The hash proves a document existed and detects that it changed; it cannot be turned back into your text.
• Source code. Files are read to match patterns. Only which file matched is reported, never a line of it.
• Customer data. Never touched.

Two independent guards enforce this: no scanner collects a secret in the first place, and redact.ts drops any finding that looks credential-shaped before the request is built. The server runs the same check on arrival — neither side trusts the other to be the only guard.

Run --dry-run first. Everything printed is everything that would be sent.

Mentions are not usage

A file that talks about Bedrock — a docs page, a provider dropdown, a vendor list — is not a file that uses Bedrock. A discovery report full of false positives is worse than no report: you stop trusting every finding, including the true ones.

So the patterns demand shapes that only real usage produces. A model id must sit inside a string literal, the way you actually pass one to an SDK. A service must show an SDK import path or an API hostname, never a bare word. In practice:

client = boto3.client("bedrock-runtime")   # reported
# we evaluated bedrock and sagemaker       # not reported

Findings carry a confidence: high for dependencies and service wiring (nobody installs openai by accident), medium for a quoted model id (a test fixture quotes one too).

Options

Flag	Meaning
--key=conf_xxx	Your API key — Settings → Integrations → API keys
--path=.	Directory to scan. Default: current directory
--dry-run	Print findings and exit. Nothing transmitted. No key needed
--json[=file]	Write the exact payload to a file (default conformly-scan.json)
--api=URL	Self-hosted instance. Default https://getconformly.com
--help / --version

Exit codes: 0 success · 1 transient failure (network, rate limit) · 2 your input was wrong (missing/invalid key, read-only org).

Keeping it continuous

A scan is a snapshot. Your inventory drifts the moment someone adds a dependency, so run it in CI:

# .github/workflows/conformly.yml
name: Conformly AI discovery
on:
  push:
    branches: [main]
  schedule:
    - cron: "0 6 * * 1" # Mondays, 06:00 UTC

jobs:
  scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0 # repo.meta needs history for the 90-day signals
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - run: npx conformly-scan --key=${{ secrets.CONFORMLY_API_KEY }}

The discovery.scan-recent control turns green when a scan has run in the last 30 days, and discovery.no-shadow-ai stays green only while every finding is reconciled with a declared system.

What it does not do

It reads the repository you point it at. It does not enumerate your cloud account, your SaaS estate, or employee devices — AI reached through a browser leaves no trace in your code. Connect an integration for those.