Byspec

Did the code satisfy the spec?

Byspec takes a human-approved spec, gathers deterministic evidence from the repository, asks an LLM judge only about what a rule cannot settle, and records every verdict in a hash-chained, signed ledger. It is the "validate" step that spec-driven tools leave optional.

PASS SPEC-001 Criterion blocks are extracted from Markdown (1 check, 0.5s) PASS LED-004 Ledger writes are append-only (3 checks, judge 0.92) FAIL CHK-002 A failing grep produces FAIL (2 checks, 0.4s) 1 grep: 0 match(es) for "export function verifyChain" in src/**/*.ts 2 pass, 1 fail, 0 unverifiable, 0 skipped → exit 1 seq 175 prev 587f3e3c… hash 68d2bc82… sig ecdsa-p256-sha256

Real output from Byspec verifying its own specification. How it works.

Command line

npm i -g @byspec-dev/cli
byspec init
byspec lint specs/
byspec verify

Runs locally, in any CI, and inside a coding agent's loop. Node 22 or later. Install options · CLI reference

GitHub App

Every pull request verified in an isolated sandbox, with one annotation per failing criterion at the line of its criterion block and a downloadable evidence packet. Early access.

Request early access How the app works

GitHub Action

- uses: adamchensley/byspec/action@v1
  with:
    spec: specs/product.spec.md

The CLI in your own CI, with your own AWS credentials for the judge. Action reference

Rules before models

A criterion is verified by deterministic checks whenever possible: files, greps, exported symbols, schemas, commands, tests, HTTP calls. The judge runs only when every check passes and the criterion asks for one. A criterion with no checks is UNVERIFIABLE, never PASS.

Information hiding

The judge sees the criterion, its rubric, and the evidence the criterion declares. It never sees the diff, the repository, or an undeclared file, so it cannot be persuaded by the code it is verifying.

The ledger is the product

Verdicts are append-only, hash-chained, signed, and reproducible. A packet exported from the ledger verifies offline with no access to the repository or the cloud. Byspec's own repository has verified itself this way since its first commit.

A criterion is a fenced block in any Markdown file

criterion
id: LED-004
title: Ledger writes are append-only
ears: The ledger module shall append entries without truncating, rewriting, or deleting existing entries.
pattern: ubiquitous
priority: must
verify:
  - type: grep
    path: "packages/core/src/ledger/**/*.ts"
    pattern: "writeFile\\(|truncate\\(|unlink\\(|rm\\("
    expect: absent
  - type: test
    pattern: packages/core/test/ledger.test.ts
  - type: judge
    rubric: |
      PASS only if every write to the ledger file uses an append-mode handle
      and no code path rewrites, truncates, or deletes ledger.jsonl.
    evidence:
      - type: file
        path: packages/core/src/ledger/append.ts
      - type: check_output
        index: 1
    min_confidence: 0.8

PRDs, design docs and tickets carry their acceptance criteria inline; the prose around them is ignored. Each criterion is one EARS sentence with a single "shall", and a clarity gate scores it before anything runs. The spec format