Skip to content

Pipelines (Reference)

This page is a curated entry point into the generated internal API reference for TopMark's pipeline system.

Note

This page contains a curated subset of the internal API reference. The authoritative API reference is generated by tools/docs/gen_api_pages.py and is available under docs/api/internals/topmark.

Note

The canonical vocabulary used throughout the documentation is defined in Terminology and Canonical Vocabulary.

Pipelines operate on an immutable FrozenConfig plus runtime options assembled via the TOML → FrozenConfig → runtime flow.

Pipelines also operate on selected processing paths. Filesystem-identity evaluation occurs before ordinary pipeline execution begins and includes both processing-path normalization and processing-target eligibility checks.

Filesystem-identity normalization collapses equivalent path spellings, such as symlinks, into selected processing paths. Pipeline steps therefore consume processing paths rather than preserving original CLI, configuration, glob, or symlink spellings.

Processing-target eligibility checks run at the pipeline-engine boundary. For example, hard-linked selected processing paths are blocked before ordinary per-file pipeline execution while unrelated selected paths continue through the requested pipeline.

See Architecture for the conceptual overview.

Source-local TOML options such as [config].root and strict are resolved before pipeline execution. They influence configuration discovery and staged config-loading validation behavior, but do not become layered configuration fields.

Path and filesystem-identity responsibilities are split between input selection, pipeline processing, and presentation layers:

  • BuilderStep generates header metadata path fields (file_relpath, file_abspath, relpath, and abspath) for the selected processing target using POSIX / serialization on all platforms. If a file was reached through a symlink, generated metadata describes the resolved target that TopMark reads and writes.
  • The pipeline engine applies invocation-wide processing-target eligibility checks before ordinary step execution. If multiple selected processing paths are hard links to the same filesystem object, each affected path receives a terminal policy-blocked result; no source, target, winner, or loser path is selected.
  • PatcherStep generates unified diffs for human review; diff file labels use human-facing display paths rather than machine-readable path serialization.
  • TEXT and Markdown frontends share display-path helpers so STDIN-backed processing consistently displays the logical --stdin-filename when available.

Machine-readable path fields are generated from the selected processing path. Serialization and presentation occur after filesystem-identity evaluation and do not preserve invocation spellings. Hard-linked selected processing paths remain separate result payloads and are reported as unsupported, policy-blocked processing targets rather than being serialized as a single preferred path.

Pipeline steps may also produce and consume phase-scoped view payloads. The Step protocol exposes consumes_views, and BaseStep stores those declarations as instance metadata. Runtime view pruning uses this typed metadata to release consumed view payloads between steps without relying on step-name string matching. The authoritative slot names are exposed by ViewSlot, and the view bundle is exposed by Views.

Note

[config].strict is a TOML-source-local strictness preference controlling staged configuration-loading validation for the current TOML source.

Effective strictness is evaluated across:

  • TOML-source diagnostics;
  • merged-config diagnostics;
  • runtime applicability diagnostics.

When strict validation fails, TopMark exits with CONFIG_ERROR. The diagnostics that triggered the failure remain visible in human-readable and machine-readable output formats.

strict is resolved during TOML loading and does not become a layered configuration field.

Line-ending handling (contract)

Pipeline steps operate under a frozen 1.0 newline contract:

  • Only LF (\n), CRLF (\r\n), and CR (\r) are recognized as physical line-ending styles.
  • SnifferStep detects and counts only these styles to determine newline histograms and mixed-line-ending diagnostics.
  • ReaderStep confirms and preserves the dominant newline style for downstream steps (rendering, planning, patching, writing).

Non-standard Unicode separators such as NEL (U+0085), Line Separator (U+2028), and Paragraph Separator (U+2029) are treated as ordinary text content. They are not considered line endings and do not contribute to newline detection or mixed-newline diagnostics.

Some file-type-specific checks (notably XML) may conservatively skip mutation when such characters appear near insertion boundaries, as idempotence cannot be guaranteed. This is a localized safety mechanism and not a general extension of newline support.

Canonical reference (generated)

Use the links below (or the documentation search box) to navigate directly to the relevant internal API module.

Pipeline definitions

Pipeline steps

Axes and statuses

Internals index

See also