Anatomy of a Rule

A rule is a directory. Which engine it belongs to decides what the directory holds, and for three of the four engines the whole rule is in there: nothing outside it is read, and deleting it deletes the rule. (The linter engine is the exception. It writes into ESLint’s, Ruff’s, RuboCop’s, or Stylelint’s own configuration and has no directory here.)

This page is the reference for what is in that directory. Engines covers which engine a rule goes to and why; this page covers what you find once it is there, and for runtime rules, everything the CLI expects of a check.ts.

Everything below was measured against CLI v0.11.2 rather than transcribed from elsewhere.

The three layouts

.taskless/rules/sg/no-eval/
    no-eval.yml                        the rule: pattern, message, severity, files
    .tests/no-eval-YYYYMMDD-test.yml   valid and invalid snippets, handed to ast-grep verbatim

.taskless/rules/vale/no-simply/
    no-simply.yml                      the style: what the rule looks for
    .vale.ini                          the scope: which files it applies to
    .tests/fail/bad.md                 prose it must flag
    .tests/pass/ok.md                  prose it must leave alone

.taskless/rules/runtime/env-read/
    captures/env-read.yml              an ast-grep rule that narrows which files the check sees
    check.ts                           the code that decides, given those matches
    .tests/fail/undeclared/            a directory: as many files as the case needs
    .tests/pass/declared/              likewise

Two things are the same in all three. The directory is named for the rule, and the tests live in a .tests/ directory whose leading dot is required: ast-grep parses every .yml it walks as a rule, so a plain tests/ directory turns your test files into rules and fails the scan for the whole project. Engines has the full account.

An sg rule

One YAML file, named for the rule, holding an ast-grep rule with a Taskless-shaped envelope: id (matching the filename), language in ast-grep’s spelling, severity, message, and the rule object, plus optional note, fix, files, and ignores. The rule carries its own file scope, so the file is the whole rule.

Its test file is an ast-grep test YAML with valid and invalid lists, and it uses those words because the file is handed to sg test as-is. The sg engine page covers authoring it.

A vale rule

Two files, and both are required. The style YAML says what the rule looks for (extends, message, level, and the check’s own fields). The .vale.ini says which files it applies to and switches the style on. A rule with only one of them does not error; it never fires, which is why verify exists. Fixtures are files under .tests/fail/ and .tests/pass/.

The Vale engine page covers the layout of each and the silent failure in detail.

A runtime rule

Capture rules, a check, and directory fixtures. The rest of this page is about this one, because it is the only engine whose rule is a program, and a program has a contract.

Runtime rules

A runtime rule exists because its evidence spans more than one file, or lies outside the files entirely. The example throughout is env-read, the rule taskless demo installs: every process.env.X read in TypeScript must have X declared in the repository-root .env. The read is in one file and the declaration is in another, so no single-file engine can compare them.

Two parts run in sequence. The captures are ast-grep rules that find the syntactic anchor (here, every process.env.$VAR) and hand the matches to the check. The check reads whatever else it needs from disk and returns findings. If the captures match nothing, the check is never called.

captures/*.yml

Each file in captures/ is an ordinary ast-grep rule with one addition: a metadata.taskless block that marks it as a runtime capture and names it.

captures/env-read.yml

id: env-read
language: TypeScript
rule:
  pattern: process.env.$VAR
metadata:
  taskless:
    version: 1
    kind: runtime
    name: env-read
    check: check.ts
    match: anchor

Field	Meaning
version	The metadata schema version. This build implements 1; a value it does not implement is refused, not treated as 1.
kind	runtime. This is what marks the file as a capture rather than an sg rule. A capture without it, or with any other value, is ignored with a reason.
name	The stable name the check branches on. It arrives on every match as match.rule. A capture with no string name is refused, because the check would have nothing to branch on.
check	The filename of the check this capture feeds. It is recorded, not resolved: the CLI always runs the check.ts in the rule directory and never follows this value as a path, so a capture cannot point execution or signing at a file outside the rule.
match	The scan mode. anchor (the default when omitted) is a syntactic narrow: one ast-grep scan over the rule, each match carrying its location, text, and captured metavariables. broad is a whole-language enumerator (rule: { kind: program }), reporting matching paths only, with line and column set to 1 and empty captures. Any other value is refused rather than defaulted, because the two modes scan different things and substituting one for the other would report a clean pass for a rule that never looked.

A rule may hold several captures, and they may mix modes. All the anchor captures run in one scan and all the broad captures in another, so adding a capture does not add a scan. The id is an ordinary ast-grep id; delivered rules carry a hashed one, which is why the check uses name instead.

check.ts

A module whose default export is an async function. taskless check calls it with the repository root and the normalized matches, and uses what it returns. Exit codes and stdout are not read; the return value is the result.

export default async function check(
  root: string,
  matches: Match[]
): Promise<Finding[]>;

The two shapes, as the CLI hands them over and takes them back:

/** One normalized ast-grep match. */
interface Match {
  /** The capture rule's `metadata.taskless.name`. Branch on this. */
  rule: string;
  /** The capture's hashed ast-grep id. Opaque; never branch on it. */
  ruleId: string;
  /** Path relative to `root`; `path.join(root, file)` to read it. */
  file: string;
  /** 1-indexed. */
  line: number;
  /** 1-indexed. */
  column: number;
  /** The matched source text. */
  text: string;
  /** Captured metavariables by name; empty for a `broad` match. */
  captures: Record<string, string>;
}

/** One result the check returns. */
interface Finding {
  /** Relative to `root`. */
  file: string;
  line?: number;
  column?: number;
  message: string;
  /** Omitted means `warning`. Only `error` affects the exit code. No `hint`: a finding's severities are a subset of an `sg` rule's. */
  severity?: 'error' | 'warning' | 'info';
}

Here is the env-read check in full. It is short because the capture already did the finding; the check only has to compare.

import { readFileSync } from 'node:fs';
import { join } from 'node:path';

/** Keys declared in the repository-root `.env`, or none when it is absent. */
function declaredKeys(root: string): Set<string> {
  let raw: string;
  try {
    raw = readFileSync(join(root, '.env'), 'utf8');
  } catch {
    return new Set();
  }

  const keys = new Set<string>();
  for (const line of raw.split('\n')) {
    const text = line.trim();
    if (text === '' || text.startsWith('#') || text.startsWith('//')) continue;
    const equals = text.indexOf('=');
    if (equals <= 0) continue;
    keys.add(text.slice(0, equals).trim());
  }
  return keys;
}

export default async function check(
  root: string,
  matches: Match[]
): Promise<Finding[]> {
  const declared = declaredKeys(root);
  const findings: Finding[] = [];

  for (const match of matches) {
    // Branch on the capture's name, never its id.
    if (match.rule !== 'env-read') continue;

    const name = match.captures.VAR;
    if (name === undefined || declared.has(name)) continue;

    findings.push({
      file: match.file,
      line: match.line,
      column: match.column,
      message: `process.env.${name} is read here, but ${name} is not declared in .env`,
      severity: 'error',
    });
  }

  return findings;
}

A check that throws does not abort the run. The throw becomes one error-severity finding for that rule, and every other rule still reports.

The contract is structural

Notice what the check does not do. It imports nothing from @taskless/*; the Match and Finding types above are declared in the file or omitted, and the CLI checks the shape of what comes back, not where the types came from. A delivered rule therefore never depends on a package being installed, and a rule you wrote by hand is indistinguishable on disk from one the CLI wrote.

It branches on match.rule, the capture’s name. The ruleId is a baked-in hash that differs between deliveries of the same rule, and a check that compares against it stops working the first time the rule is regenerated.

It also stands alone. The only module allowed in a runtime rule directory, outside .tests/, is check.ts. A helpers.ts beside it, or a lib/ under it, causes the CLI to refuse the whole rule rather than the stray file, because the signature covers check.ts and an import is one hop from anything. Read what you need from disk under root; do not import it.

.tests/pass/<case>/ and .tests/fail/<case>/

A fixture case is a directory, and that directory is the root the test runner hands the check, with every match.file relative to it.

.tests/
    fail/undeclared/
        .env                 API_URL=https://api.example.com
        src/config.ts        reads API_URL and API_KEY
    pass/declared/
        .env                 API_URL=https://api.example.com
        src/config.ts        reads API_URL only

That is the layout, not a detail of it. A runtime rule exists because its evidence spans files, so a layout allowing one file per case could not express the rules this engine is for. A source file and a .env side by side in one case is the ordinary shape, and a case that needs three source files holds three.

The buckets are pass and fail, matching Vale’s. (ast-grep’s valid and invalid are keys inside a test YAML that is handed to sg test verbatim. Nothing hands a runtime fixture to ast-grep, so borrowing the word here would make it mean two things.)

Run them with test, which verifies the rule first:

npx @taskless/cli test .taskless/rules/runtime/env-read --dangerously-run-scripts

The flag is not optional here. The next section says why.

What the runtime engine demands

Three things, and each one shows up the same way when you miss it: a rule that reports nothing, for a reason the output names. Read the output.

Every fixture case must fire a capture

Both buckets have to be populated, and every case, in pass/ as well as fail/, has to match at least one capture. A case the captures do not match never reaches the check, and the test runner reports that as a defect in the case rather than a pass.

This catches the tempting wrong fixture. A pass/ case with no process.env reads at all passes trivially and proves nothing about the check. The pass/declared/ case above reads API_URL, so the capture fires, the check runs, and the empty result is a decision rather than an absence.

A check gets 10 seconds

Each check.ts invocation is bounded by a wall-clock timeout, 10 seconds by default. A check that exceeds it is terminated and recorded as one error-severity finding for that rule, and the run continues. Raise it per run when a rule legitimately needs longer:

npx @taskless/cli check --timeout 30

The bound is per invocation, not per match, so a check that does work proportional to its matches should read each file once rather than once per match.

An unverified check does not run

check signs each check.ts, reconciles the signatures with the Taskless service, and runs only what came back verified. Logged out, --anonymous, with no GitHub remote, or with the service unreachable, every runtime rule is skipped and reported as skipped, and the sg and vale rules run as they always do. Under --json the skipped rules appear in an additive skipped: [{ rule, reason }] array; they never change the exit code.

A rule you wrote a minute ago has no reconciled signature and never will until it has been through that service, so on your own machine the way to run it is to say so:

npx @taskless/cli check --dangerously-run-scripts

That runs every runtime rule trusting the local signatures, with no network call, behind a warning. test takes the same flag and takes only that flag: it never consults the service, so it is the one place the flag is the whole gate rather than an escape from it. Without it, test prints ○ for the rule, says the fixtures did not run, names the flag, and exits 0, which is deliberately neither a pass nor a failure.

The flag is for iterating on a rule you wrote and have read. It is not a way to ship a rule to a team, and suggesting it to get past a login turns a deliberate gate into an unreviewed code-execution path on someone else’s machine. Runtime rules covers why the gate exists, and Security covers the verification model in full.

Verifying the shape

Whichever engine, two commands close the loop:

npx @taskless/cli verify .taskless/rules/runtime/env-read
npx @taskless/cli test   .taskless/rules/runtime/env-read --dangerously-run-scripts

verify checks that a rule has the components its engine requires: a capture in captures/ and a check.ts for a runtime rule (without a capture, the check would never be invoked), a style and a .vale.ini for Vale, a parseable rule for sg. It also names an unimplemented match mode, a stray module, or a missing name before anything runs. test runs the fixtures, after verifying first.

To see a working runtime rule in your own repository, install the demonstration:

npx @taskless/cli demo

It writes env-read with the capture, check, and fixtures shown on this page.