GitHub App

• name: Byspec, head_sha, external_id: <run_id>, details_url: <api>/runs/<run_id>/export?sig=….
• conclusion: success for exit 0, failure for exit 1, neutral for exit 2 (UNVERIFIABLE with fail-on-unverifiable off is success; the tenant can opt into failure later), action_required when spec_changed_in_diff is true with --frozen-spec and nothing else failed (the human changed the spec in this PR; that is allowed but must be seen).
• output.title: <n> pass, <n> fail, <n> unverifiable, <n> skipped; output.summary: the v1 github renderer; output.text: the agent renderer (short and plain).
• Annotations: one per non-PASS, non-SKIPPED criterion at path = source.file, start_line = end_line = source.line, annotation_level: failure for FAIL, warning for UNVERIFIABLE, title = <id>: <title>, message = the failing check and its message plus up to 5 evidence lines. Sent in batches of 50 (GitHub's limit per update).
• Export link: GET /runs/{run_id}/export?sig=<hmac-sha256(run_id + expiry, ExportSigningSecret)>&exp=<unix> builds the packet (Section 6) on first request, stores it at the run prefix, and 302s to a 15-minute presigned URL. Anyone with the link inside the expiry can download; the link is only ever placed on the check run, which inherits the repo's visibility.

Source: docs/byspec-v2-spec.md § 9. GitHub check run

fetch.yml (egress allowed): clone clone_url at head_sha with the installation token (depth 1 plus base_sha fetch for --changed), run pnpm install --frozen-lockfile (or npm ci when no pnpm lockfile) when a lockfile exists, install @byspec-dev/cli@<pinned> into .byspec-tool/, then tar --zstd the checkout (including node_modules and .git) to tenants/<tenant>/cache/<sha256(lockfile)>.tar.zst when a lockfile exists and to tenants/<tenant>/runs/<run_id>/workspace.tar.zst always.

verify.yml (no egress in isolated): download workspace.tar.zst from S3 (gateway endpoint), extract, run

BYSPEC_ACTOR=ci:byspec-app .byspec-tool/node_modules/.bin/byspec verify --changed --base <base_sha> --head <head_sha> --frozen-spec --no-ledger --provider bedrock --format json --out report.json

with BYSPEC_REGION set, then upload report.json, .byspec/evidence/<run_id>/**, and byspec report --format md to the run prefix. verify exit codes 0, 1, 2 are success for the build (the verdict is in the report); 3 and 4 fail the build so the pipeline's Catch reports them. The Verify project's service role has no s3:PutObject outside tenants/<tenant>/runs/<run_id>/ and no network path other than the endpoints listed in 8.1; the checked-out code can read its own workspace and talk to Bedrock, nothing else.

Source: docs/byspec-v2-spec.md § 8.4 Sandbox stages (buildspecs)

The CLI is how a developer or an agent verifies locally. The control plane is how a team gets a ledger nobody has to operate: a private GitHub App verifies every pull request in an isolated sandbox, signs the verdicts with KMS into a per-tenant hash chain in DynamoDB, keeps evidence in S3 under SSE-KMS, and posts a Byspec check run with one annotation per failing criterion at the line of its criterion block, plus a link to the evidence packet. The full design is docs/byspec-v2-spec.md, itself a Byspec spec.

How a run works: the webhook creates a queued check run and starts a Step Functions execution. Stage one (CodeBuild, with egress) clones the head commit with a short-lived installation token, installs dependencies, and tars the workspace to S3. Stage two (CodeBuild, no egress in isolated stages) restores that tarball and runs byspec verify --changed --frozen-spec --no-ledger --provider bedrock with the single-file CLI bundle shipped by the deployment, then uploads the report, the evidence, and the parsed spec files. An ingest function validates the report, writes the run and per-criterion verdicts, appends signed entries to the tenant's chain in one DynamoDB transaction per chunk, and completes the check run. The tenant is the GitHub installation (gh-<installation id>); Clerk organisations, Stripe billing, Resend digests, and a dashboard are v2.1 and attach to that model without a migration.

What the sandbox can reach: its own workspace, S3 (the run's own prefix only), CloudWatch Logs, and Bedrock. In prod (isolated) it runs in a VPC with no NAT gateway and only those interface endpoints, so command and test checks cannot reach the internet; the endpoints cost roughly $30 per month while idle. In dev (open) the same project runs without a VPC, which is cheaper and fine for trying it out. Neither stage runs in privileged mode.

Deploying your own (needs AWS credentials in the default chain and Bedrock access to the configured model ids in us-east-1):

pnpm build && pnpm build:bundle           # the sandbox runs packages/cli/dist-bundle/byspec.mjs
pnpm exec sst install
pnpm exec sst deploy --stage dev          # prints the API url

Then open <api>/github/setup in a browser: it shows the app manifest and a single button that registers a private GitHub App on your account (or ?org=<name> for an organisation) with the webhook and callback pointing at this deployment. The callback stores the app's credentials in SSM under /byspec/<stage>/github/*, so no redeploy is needed, and links to the install page; install the app on the repositories you want verified. Every pull request from then on gets a Byspec check run. Exit code 0 is success, 1 is failure, 2 is neutral, and a spec file changed inside the PR with nothing else failing is action_required. The check run's details link downloads the evidence packet through a signed, expiring URL.

Source: README.md § Cloud control plane (v2, preview)