Anatomy of a Rule

A rule is a directory. Which engine it belongs to decides what the directory holds, and for three of the four engines the whole rule is in there: nothing outside it is read, and deleting it deletes the rule. (The linter engine is the exception. It writes into ESLint’s, Ruff’s, RuboCop’s, or Stylelint’s own configuration and has no directory here.)

This page is the reference for what is in that directory. Engines covers which engine a rule goes to and why; this page covers what you find once it is there, and for runtime rules, everything the CLI expects of a check.ts.

Everything below was measured against the CLI rather than transcribed from elsewhere: against v0.11.2 when the page was written. The fixture layout follows the v0.13.4 recipes that introduced it.

The three layouts

.taskless/rules/sg/no-eval/
    no-eval.yml                             the rule: pattern, message, severity, files
    tests/fail/eval-of-payload/handler.ts   code it must fire on
    tests/pass/json-parse/handler.ts        code it must leave alone

.taskless/rules/vale/no-simply/
    no-simply.yml                           the style: what the rule looks for
    .vale.ini                               the scope: which files it applies to
    tests/fail/simply-in-setup/README.md    prose it must flag
    tests/pass/simple-noun/README.md        prose it must leave alone

.taskless/rules/runtime/env-read/
    captures/env-read.yml                   an ast-grep rule that narrows which files the check sees
    check.ts                                the code that decides, given those matches
    tests/fail/undeclared/                  a case: as many files as it needs
    tests/pass/declared/                    likewise

Two things are the same in all three. The directory is named for the rule, and its fixtures are case directories under tests/fail/ and tests/pass/, judged the same way whatever the engine. Fixtures below covers how.

An sg rule

One YAML file, named for the rule, holding an ast-grep rule with a Taskless-shaped envelope: id (matching the filename), language in ast-grep’s spelling, severity, message, and the rule object, plus optional note, fix, files, and ignores. The rule carries its own file scope, so the file is the whole rule.

Each fixture case holds a source file in the rule’s language, at a path its files: admits. The sg engine page covers authoring it.

A vale rule

Two files, and both are required. The style YAML says what the rule looks for (extends, message, level, and the check’s own fields). The .vale.ini says which files it applies to and switches the style on. A rule with only one of them does not error; it never fires, which is why validate exists. Each fixture case holds a document at a path one of the .vale.ini matchers admits.

The Vale engine page covers the layout of each and the silent failure in detail.

A runtime rule

Capture rules, a check, and directory fixtures. The rest of this page is about this one, because it is the only engine whose rule is a program, and a program has a contract.

Runtime rules

A runtime rule exists because its evidence spans more than one file, or lies outside the files entirely. The example throughout is env-read: every process.env.X read in TypeScript must have X declared in the repository-root .env. The read is in one file and the declaration is in another, so no single-file engine can compare them.

Two parts run in sequence. The captures are ast-grep rules that find the syntactic anchor (here, every process.env.$VAR) and hand the matches to the check. The check reads whatever else it needs from disk and returns findings. If the captures match nothing, the check is never called.

captures/*.yml

Each file in captures/ is an ordinary ast-grep rule with one addition: a metadata.taskless block that marks it as a runtime capture and names it.

captures/env-read.yml

id: env-read
language: TypeScript
rule:
  pattern: process.env.$VAR
metadata:
  taskless:
    version: 1
    kind: runtime
    name: env-read
    check: check.ts
    match: anchor

Field	Meaning
version	The metadata schema version. This build implements 1; a value it does not implement is refused, not treated as 1.
kind	runtime. This is what marks the file as a capture rather than an sg rule. A capture without it, or with any other value, is ignored with a reason.
name	The stable name the check branches on. It arrives on every match as match.rule. A capture with no string name is refused, because the check would have nothing to branch on.
check	The filename of the check this capture feeds. It is recorded, not resolved: the CLI always runs the check.ts in the rule directory and never follows this value as a path, so a capture cannot point execution or signing at a file outside the rule.
match	The scan mode. anchor (the default when omitted) is a syntactic narrow: one ast-grep scan over the rule, each match carrying its location, text, and captured metavariables. broad is a whole-language enumerator (rule: { kind: program }), reporting matching paths only, with line and column set to 1 and empty captures. Any other value is refused rather than defaulted, because the two modes scan different things and substituting one for the other would report a clean pass for a rule that never looked.

A rule may hold several captures, and they may mix modes. All the anchor captures run in one scan and all the broad captures in another, so adding a capture does not add a scan. The id is an ordinary ast-grep id; delivered rules carry a hashed one, which is why the check uses name instead.

check.ts

A module whose default export is an async function. taskless check calls it with the repository root and the normalized matches, and uses what it returns. Exit codes and stdout are not read; the return value is the result.

export default async function check(
  root: string,
  matches: Match[]
): Promise<Finding[]>;

The two shapes, as the CLI hands them over and takes them back:

/** One normalized ast-grep match. */
interface Match {
  /** The capture rule's `metadata.taskless.name`. Branch on this. */
  rule: string;
  /** The capture's hashed ast-grep id. Opaque; never branch on it. */
  ruleId: string;
  /** Path relative to `root`; `path.join(root, file)` to read it. */
  file: string;
  /** 1-indexed. */
  line: number;
  /** 1-indexed. */
  column: number;
  /** The matched source text. */
  text: string;
  /** Captured metavariables by name; empty for a `broad` match. */
  captures: Record<string, string>;
}

/** One result the check returns. */
interface Finding {
  /** Relative to `root`. */
  file: string;
  line?: number;
  column?: number;
  message: string;
  /** Omitted means `warning`. Only `error` affects the exit code. No `hint`: a finding's severities are a subset of an `sg` rule's. */
  severity?: 'error' | 'warning' | 'info';
}

Here is the env-read check in full. It is short because the capture already did the finding; the check only has to compare.

import { readFileSync } from 'node:fs';
import { join } from 'node:path';

/** Keys declared in the repository-root `.env`, or none when it is absent. */
function declaredKeys(root: string): Set<string> {
  let raw: string;
  try {
    raw = readFileSync(join(root, '.env'), 'utf8');
  } catch {
    return new Set();
  }

  const keys = new Set<string>();
  for (const line of raw.split('\n')) {
    const text = line.trim();
    if (text === '' || text.startsWith('#') || text.startsWith('//')) continue;
    const equals = text.indexOf('=');
    if (equals <= 0) continue;
    keys.add(text.slice(0, equals).trim());
  }
  return keys;
}

export default async function check(
  root: string,
  matches: Match[]
): Promise<Finding[]> {
  const declared = declaredKeys(root);
  const findings: Finding[] = [];

  for (const match of matches) {
    // Branch on the capture's name, never its id.
    if (match.rule !== 'env-read') continue;

    const name = match.captures.VAR;
    if (name === undefined || declared.has(name)) continue;

    findings.push({
      file: match.file,
      line: match.line,
      column: match.column,
      message: `process.env.${name} is read here, but ${name} is not declared in .env`,
      severity: 'error',
    });
  }

  return findings;
}

A check that throws does not abort the run. The throw becomes one error-severity finding for that rule, and every other rule still reports.

The contract is structural

Notice what the check does not do. It imports nothing from @taskless/*; the Match and Finding types above are declared in the file or omitted, and the CLI checks the shape of what comes back, not where the types came from. A delivered rule therefore never depends on a package being installed, and a rule you wrote by hand is indistinguishable on disk from one the CLI wrote.

It branches on match.rule, the capture’s name. The ruleId is a baked-in hash that differs between deliveries of the same rule, and a check that compares against it stops working the first time the rule is regenerated.

It also stands alone. The only module allowed in a runtime rule directory, outside tests/, is check.ts. A helpers.ts beside it, or a lib/ under it, causes the CLI to refuse the whole rule rather than the stray file, because the signature covers check.ts and an import is one hop from anything. Read what you need from disk under root; do not import it.

A runtime fixture case

A runtime rule’s case directory is the root the test runner hands the check, with every match.file relative to it.

tests/
    fail/undeclared/
        .env                 API_URL=https://api.example.com
        src/config.ts        reads API_URL and API_KEY
    pass/declared/
        .env                 API_URL=https://api.example.com
        src/config.ts        reads API_URL only

A runtime rule is why a case is a directory. Its evidence spans files, so a layout allowing one file per case could not express the rules this engine is for. A source file and a .env side by side in one case is the ordinary shape, and a case that needs three source files holds three.

Run them with verify, which validates the rule first:

./.taskless/bin/cli.mjs verify .taskless/rules/runtime/env-read --dangerously-run-scripts

The flag is not optional here. The next section says why.

What the runtime engine demands

Three things, and each one shows up the same way when you miss it: a rule that reports nothing, for a reason the output names. Read the output.

Every fixture case must fire a capture

Both buckets have to be populated, and every case, in pass/ as well as fail/, has to match at least one capture. This is the runtime form of the rule every engine’s fixtures follow: a case the rule never read fails. A case the captures do not match never reaches the check, and the test runner reports that as a defect in the case rather than a pass.

This catches the tempting wrong fixture. A pass/ case with no process.env reads at all passes trivially and proves nothing about the check. The pass/declared/ case above reads API_URL, so the capture fires, the check runs, and the empty result is a decision rather than an absence.

A check gets 10 seconds

Each check.ts invocation is bounded by a wall-clock timeout, 10 seconds by default. A check that exceeds it is terminated and recorded as one error-severity finding for that rule, and the run continues. Raise it per run when a rule legitimately needs longer:

./.taskless/bin/cli.mjs check --timeout 30

The bound is per invocation, not per match, so a check that does work proportional to its matches should read each file once rather than once per match.

An unverified check does not run

check signs each check.ts, reconciles the signatures with the Taskless service, and runs only what came back verified. Logged out, --anonymous, with no GitHub remote, or with the service unreachable, every runtime rule is skipped and reported as skipped, and the sg and vale rules run as they always do. Under --json the skipped rules appear in an additive skipped: [{ rule, reason }] array; they never change the exit code.

A rule you wrote a minute ago has no reconciled signature and never will until it has been through that service, so on your own machine the way to run it is to say so:

./.taskless/bin/cli.mjs check --dangerously-run-scripts

That runs every runtime rule trusting the local signatures, with no network call, behind a warning. verify takes the same flag and takes only that flag: it never consults the service, so it is the one place the flag is the whole gate rather than an escape from it. Without it, verify prints ○ for the rule, says the fixtures did not run, names the flag, and exits 0, which is deliberately neither a pass nor a failure.

The flag is for iterating on a rule you wrote and have read. It is not a way to ship a rule to a team, and suggesting it to get past a login turns a deliberate gate into an unreviewed code-execution path on someone else’s machine. Runtime rules covers why the gate exists, and Security covers the verification model in full.

Fixtures

Every engine’s fixtures follow the same rules, and verify enforces them:

• A case is a directory, named for what it shows (tests/fail/npx-latest/, not tests/fail/case-1/). A file directly in pass/ or fail/, anything in tests/ besides those two, or a case with no file in it is refused.
• Each case runs like a small repository. The case directory is the project root, and the rule’s own scope applies, so a fixture has to sit at a path the rule’s files: or .vale.ini matchers admit. The project’s .gitignore applies inside a case too, so keep fixtures out of directory names such as dist/ or build/.
• Firing is the whole test. A fail/ case passes when the rule reports at least one finding in it, and a pass/ case when it reports none. The line, column, message, and count are not asserted. Both buckets need at least one case.
• Every case must be read. A case the rule read no file in fails in either bucket, because it shows nothing. verify says the case “was never read”: either the scope excluded every file in it, or the engine walked none of them.
• “The rule’s scope excluded it” means the case fires once the scope is removed, so the pattern is fine and the fixture’s path is not. Move the fixture, or widen the scope if the rule was meant to reach it.

Fixtures used to live in .tests/, where ast-grep’s were a valid:/invalid: test YAML handed to sg test. The first install after upgrading to CLI 0.13.4 converts them: each ast-grep snippet becomes its own case, named fail-01, pass-01, and so on, and each Vale document a case named for it. Rename the converted cases for what they show. Anything the conversion could not place stays in .tests/, named in the install’s output, and verify fails the rule until it is moved.

Validating the shape

Whichever engine, two commands close the loop:

./.taskless/bin/cli.mjs validate .taskless/rules/runtime/env-read
./.taskless/bin/cli.mjs verify   .taskless/rules/runtime/env-read --dangerously-run-scripts

validate checks that a rule has the components its engine requires: a capture in captures/ and a check.ts for a runtime rule (without a capture, the check would never be invoked), a style and a .vale.ini for Vale, a rule ast-grep loads for sg. It also names an unimplemented match mode, a stray module, or a missing name before anything runs. verify runs the fixtures, after validating first.