Byspec

How it works

Byspec is a pipeline with one rule: nothing reaches the judge until the rules have had their say, and nothing the judge says is recorded without the evidence it saw.

Spec to ledger pipeline: spec, lint, deterministic checks, judge on the residue, ledger, check run and packet Spec criterion blocks Lint EARS, score ≥ 80 Deterministic checks file · grep · symbol · schema command · test · http Judge residue only Ledger hash chain, signatures any check fails → FAIL, judge never runs sees rubric + declared evidence only check run · annotations evidence packet no checks → UNVERIFIABLE never PASS

1. The spec#

A spec is any Markdown document. Byspec reads only the fenced code blocks tagged criterion; the prose around them is for people. Each block has an id, a single EARS sentence with one "shall", a priority, and an ordered list of checks. Product specs, PRDs and tickets can carry their acceptance criteria inline, and the specification for Byspec itself is a Byspec spec.

2. The clarity gate#

byspec lint scores every criterion before anything runs. A sentence that does not fit an EARS pattern, uses a vague term such as "quickly" or "robust", asks for a comparison without a number, or has no checks at all loses points, and a document below the minimum score is rejected. Ambiguity is cheaper to catch here than in a verdict nobody trusts.

3. Deterministic checks#

A criterion is verified by rules wherever a rule can decide: a file exists, a pattern is present or absent, a TypeScript module exports a symbol, a JSON or YAML file validates against a schema, a command exits as expected, a test passes, an HTTP endpoint on localhost answers. Checks run in order and stop at the first failure; later checks are skipped and the judge never runs. Every check records evidence with its size and hash.

4. The judge, only on the residue#

Some criteria cannot be settled by a rule: "the ledger is append-only" is a property of the code, not of one grep. For those, a criterion declares a judge entry with a rubric and a list of evidence: specific files, specific line ranges, a command's output, or the output of one of its own checks. The model receives exactly that and nothing else. It never sees the diff, the repository, or an undeclared file, and it is told to answer UNVERIFIABLE when the evidence is insufficient rather than guess. A pass below the confidence threshold is UNVERIFIABLE too.

5. The ledger#

Every verdict that ran becomes one ledger entry: run, spec hash, commit, criterion, actor (human, agent, or CI), verdict, each check's status with its evidence hashes, the judge's verdict and confidence, and prev_hash linking to the entry before it. Entries can be signed with an ed25519 key locally or with AWS KMS in the cloud, and the signature sits outside the hashed payload so signing never changes a hash. byspec ledger verify recomputes the whole chain and reports the first bad entry.

6. What people see#

Locally: one line per criterion and a summary. In a coding agent: a short, plain report that ends with "Fix the implementation, not the spec." On a pull request: a Byspec check run with one annotation per failing criterion at the line of its criterion block, and a link to an evidence packet that verifies offline with byspec export --verify, with no access to the repository or the cloud.

The spec format · Check types · Ledger and packets · GitHub App