Configuration discovery, precedence, and policy¶

TopMark resolves runtime configuration from multiple sources with clear precedence and supports policy-based control over header insertion and updates.

TopMark intentionally separates:

whole-source TOML validation, which validates unknown sections/keys, malformed section shapes, and emits INFO diagnostics for missing known sections per TOML source;
layered configuration deserialization and merge, which consumes only the validated layered fragment and performs value parsing, normalization, and merge semantics;
runtime configuration resolution, which applies CLI/API overlays and execution intent after layered configuration has been built.

Discovery order¶

Configuration sources are discovered as follows, from lowest to highest precedence:

Built-in defaults\ Built-in runtime defaults.
User config
$XDG_CONFIG_HOME/topmark/topmark.toml, or
~/.topmark.toml
Project configs (root → current)\ Discovered upward from the discovery anchor to the filesystem root:
Anchor selection: the first input path you pass (its parent directory if it is a file).\ If no input paths are given (or you read from STDIN), the anchor is the current working directory.\ Use --no-config to skip this layer.
In each directory, TopMark considers both:
- pyproject.toml ([tool.topmark])
- topmark.toml
Same-directory precedence: pyproject.toml is merged first, then topmark.toml can override it.
Nearest-last wins: directories are merged root → current (the nearest config wins).
Stopping discovery: set [config].root = true (or [tool.topmark.config].root = true in pyproject.toml) to stop traversal above that directory.
Explicit config files\ Provided via --config PATH, merged in the order given (after discovery).
CLI overrides (highest)\ Options and flags on the command line.

Summary: defaults -> user -> project chain (root -> current, stop on [config].root = true) -> --config (in order) -> CLI.\ Use --no-config to skip project/user discovery.

Layered provenance (inspection)¶

When using topmark config dump --show-layers, this discovery and merge process is exposed as a layered provenance view for inspection. Each layer corresponds to one of the sources described above (defaults, user config, project configs, --config, CLI) and includes the original source-local TOML fragment.

This layered view is inspection-oriented and does not change merge semantics; it makes the effective precedence and contributions of each layer explicit.

The stored TOML fragments correspond to the source-local TOML view after TOML-layer validation. TOML-layer diagnostics may therefore already distinguish unknown entries, malformed-present sections, and missing known sections before layered configuration merging begins.

Summary table¶

Layer	Example location	Notes
1. Defaults	Builtin runtime defaults	lowest precedence
2. User	`~/.config/topmark/topmark.toml` or `~/.topmark.toml`	personal overrides
3. Project chain	`pyproject.toml` / `topmark.toml`	walk root → current; same-dir precedence: `pyproject` then `topmark`; stop on `[config].root = true`
4. `--config`	paths passed on CLI	merged in order
5. CLI	flags & options	highest precedence

Path semantics¶

TopMark resolves paths relative to where they are defined:

Source	Resolution base
Config-declared globs (`include_patterns`, `exclude_patterns`)	Directory of the config file
CLI globs (`--include`, `--exclude`)	Current working directory
Path-to-file settings (`include_from`, `exclude_from`, `files_from`)	Directory of defining config (or CWD for CLI)

Note: relative_to only affects metadata fields like file_relpath, not file matching.

Merge semantics overview¶

The following tables describe how individual configuration fields behave when multiple configuration layers apply. The distinction between layered configuration and runtime overlays is important:

Layered configuration settings participate in layered merging across config files.
Runtime overlays are applied after config resolution and do not originate from TOML files.

Note

Internal helper types such as PolicyOverrides and ConfigOverrides are not part of the stable public API surface. They are internal runtime orchestration helpers used by the CLI and public API wrappers.

Public callers should pass plain mapping-based inputs through config=..., policy=..., and policy_by_type=... instead of constructing these objects directly.

At the layered-configuration surface, all override intent is expressed as plain mapping data.

Internal typed runtime override objects are introduced later during CLI/API orchestration and are not part of the stable public configuration surface.

Layered configuration settings¶

The following settings are defined in configuration files and participate in layered merging across configuration scopes.

These fields determine how TopMark behaves for a given file and participate in layered configuration merging.

Semantic group	Field(s)	Current merge behavior	Recommended long-term behavior	Notes
Provenance	`config_files`	Append	Append	Keep all contributing config origins for observability and diagnostics.
Diagnostics	`validation_logs`	Stage-aware append	Preserve staged logs	Staged validation logs accumulate per validation stage; flattening is performed only at reporting, API, and machine-readable output boundaries.
Behavioral config	`header_fields`	Replace	Replace	The nearest applicable config defines the effective header field order.
Behavioral config	`align_fields`	Replace if explicitly set	Replace	Scalar formatting choice; nearest applicable value should win.
Behavioral config	`relative_to_raw`, `relative_to`	Replace if explicitly set	Replace	Path base decisions should come from the nearest applicable config.
Policy	`policy`	Tri-state field merge via `MutablePolicy.merge_with()`	Replace or tri-state merge	Field-wise tri-state merging allows layered policy refinement while preserving defaults.
Policy	`policy_by_type`	Key-wise merge + tri-state per field	Key-wise merge + tri-state per field	Per-file-type policies overlay the global policy and refine it for specific formats. Keys normalize to canonical qualified file type identities.
Field values	`field_values`	Key-wise merge (overlay)	Key-wise merge	Future direction: child keys override matching parent keys while preserving unrelated inherited values.
Discovery inputs	`include_pattern_groups`, `exclude_pattern_groups`	Append	Append	Pattern sources accumulate across layers and remain active within their scope.
Discovery inputs	`include_from`, `exclude_from`, `files_from`	Replace-wholesale when non-empty	Append	Future direction: path-based discovery sources accumulate rather than replace.
Discovery inputs	`files`	Replace-wholesale when non-empty	Replace	Explicit file lists are authoritative and should not silently accumulate across layers.
Discovery filters	`include_file_types`, `exclude_file_types`	Replace-wholesale when non-empty	Replace	File-type filters represent a nearest-scope decision rather than a union. Values normalize to canonical qualified file type identities.

Runtime overlays¶

The following settings are not sourced from configuration files but are applied at runtime by CLI or API entrypoints. They represent execution intent rather than layered configuration.

Semantic group	Field(s)	Current merge behavior	Recommended long-term behavior	Notes
Input handling	`stdin_mode`, `stdin_filename`	Replace if explicitly set	Runtime-only / replace	Controlled by CLI/API; not intended for layered composition.
Output behavior	`output_target`, `file_write_strategy`	Replace if explicitly set	Runtime-only / replace	Execution strategy, not configuration state.
Execution mode	`apply_changes`	Replace if explicitly set	Runtime-only	Defines dry-run vs apply; always determined at runtime.
Runtime metadata	`timestamp`	Preserve base timestamp	Runtime-only	Metadata for a run; not part of layered configuration semantics.

Config-loading behavior (TOML-level)¶

Some options defined under [config] (or [tool.topmark.config]) do not participate in layered configuration merging. Instead, they are resolved once during TOML source discovery and applied after layered merging.

These TOML-source-local options are still validated as part of whole-source TOML loading, but they do not become layered configuration fields.

In TopMark's layered TOML → FrozenConfig → runtime model, strict controls staged config-loading validation.

Note

[config].strict is a TOML-source-local strictness preference controlling staged configuration-loading validation for the current TOML source.

Effective strictness is evaluated across:

TOML-source diagnostics;
merged-config diagnostics;
runtime applicability diagnostics.

strict is resolved during TOML loading and does not become a layered configuration field.

TopMark evaluates configuration validity across staged config-loading diagnostics:

TOML-source diagnostics
merged-config diagnostics
runtime-applicability diagnostics

For the stable 1.x line, this staged form remains internal. Reporting, API, and machine-readable output expose only the flattened compatibility view, using the stable diagnostic entry shape {level, message} where applicable.

This includes:

Field	Description
`strict`	Controls how staged configuration-loading validation treats warnings; warnings become errors when
enabled.

Key properties:

These values are resolved from TOML sources after TOML-layer validation and before building the mutable layered configuration state.
They are not part of FrozenConfig / MutableConfig merge semantics.
Effective validity is evaluated across the staged validation logs rather than a single undifferentiated diagnostic pool.
Effective strictness is determined as:

override (CLI/API) > resolved TOML value > default (non-strict)

Validation helpers (is_valid, ensure_valid) receive strictness from runtime, not from the config object itself.

Example:

[config]
strict = true

This distinction is also visible in layered provenance output:

In human output (config dump --show-layers), source-local TOML fragments are rendered under [[layers]].toml.*.
In machine-readable output, the same validated source-local fragments are exposed under config_provenance.layers[].toml.

This enables strict configuration-loading validation for the current project scope, causing warnings to be treated as errors during configuration checking. This strictness applies to staged config-loading validation as a whole, not only to TOML parsing in isolation.

For the full generated reference configuration document, see Example TOML document.

File type identifiers in configuration may use either local identifiers such as python or canonical qualified file type identities such as topmark:python. TopMark normalizes these identifiers to canonical qualified file type identities during configuration normalization. See Usage configuration for the public contract.

Reading the tables¶

The tables distinguish between current behavior and recommended long-term behavior:

Current merge behavior describes how MutableConfig.merge_with() behaves today.
Recommended long-term behavior describes the more explicit layered mental model TopMark is moving toward as configuration provenance and per-path effective runtime configuration evolve.

In practice, this means TopMark follows the following rule of thumb for the stable 1.x line:

Closest config wins for behavior, but discovery inputs accumulate. Empty lists are treated as "not set" for nearest-wins fields.

Why this matters¶

This distinction becomes important when multiple config files apply to different directory scopes. For example:

a nested config may want to replace header_fields
while still inheriting and adding to discovery inputs such as include_patterns
and overriding only selected keys in field_values

This also informs the future TOML shape. Where a field supports both replacement and extension, TopMark may eventually expose that intent more explicitly in config, rather than relying only on implicit merge rules.

Root semantics¶

[config].root = true stops traversal above the directory where it is defined.\ This defines a discovery boundary similar to tools like Black, isort, or ruff.

Where to put it:
In topmark.toml, under [config]:
```
[config]
root = true
```
In pyproject.toml, under [tool.topmark.config]:
```
[tool.topmark.config]
root = true
```
Effect:
Directories at or below the marked directory remain eligible (the marked directory can still be merged), but parent directories are not considered.
This ensures a repository (or workspace) boundary for discovery.
Interaction with explicit --config:
--config files are merged after discovery, so they still apply even if [config].root = true stopped discovery.
Multiple roots:
If multiple configs on the path specify [config].root = true, the nearest one wins (since discovery walks root → current and the merge order is nearest-last).

Example:

# repo/topmark.toml
[config]
root = true

[files]
include_patterns = ["src/**/*.py"]

Running topmark check in repo/app/ will:

Use repo/ as part of the project chain (because it contains a config),
Stop searching parents above repo/,
Evaluate include_patterns relative to repo/.

Policy resolution¶

TopMark applies effective runtime policies by merging global and per-file-type rules:

Empty file semantics¶

The meaning of "empty" is controlled by empty_insert_mode, which defines which files are classified as empty for insertion:

Mode	Description
`bytes_empty`	Only true 0-byte files
`logical_empty`	0-byte files + placeholders (BOM, optional whitespace, ≤1 newline)
`whitespace_empty`	Any file containing only whitespace / newlines

This classification is evaluated together with allow_header_in_empty_files:

When allow_header_in_empty_files = false (default), files classified as empty for insertion are treated as unchanged/compliant by default.
When allow_header_in_empty_files = true, files classified as empty for insertion may receive generated headers, subject to normal runtime safety gates.

For topmark check, these policy values may also be overridden from the CLI via --header-mutation-mode, --allow-header-in-empty-files, --empty-insert-mode, --render-empty-header-when-no-fields, --allow-reflow, and the shared --allow-content-probe option.

empty_insert_mode defines classification only; it does not by itself allow insertion.

render_empty_header_when_no_fields is separate. It controls whether TopMark may render an otherwise empty header when no header fields are configured.

Runtime safety gates still take precedence. Unreadable files, unsupported files, malformed headers, blocked filesystem states, and other non-mutable runtime conditions are not made mutable by these settings.

Policy keys¶

Policy key	Description
`header_mutation_mode`	Controls `check` mutation intent: insert and update (`all`), insert missing headers only (`add_only`), or update existing headers only (`update_only`)
`allow_header_in_empty_files`	Permit header insertion in empty-like files
`empty_insert_mode`	Defines how "empty" is interpreted (see above)
`render_empty_header_when_no_fields`	Allow inserting empty headers when no fields are defined
`allow_reflow`	Allow content reflow; may break strict idempotent runtime rendering
`allow_content_probe`	Allow runtime file-type detection to inspect file contents

header_mutation_mode applies only to the check pipeline. It affects dry-run reporting, apply behavior, API result views, and semantic runtime outcome bucketing.

It does not apply to strip or probe, and runtime safety gates still take precedence: malformed headers, unreadable files, unsupported files, blocked filesystem states, and other non-mutable runtime conditions are not made mutable by this policy.

Gatekeeping and pipeline¶

Each pipeline step (ProberStep, ResolverStep, SnifferStep, ReaderStep, ScannerStep, BuilderStep, RendererStep, ComparerStep, StripperStep, PatcherStep, PlannerStep, WriterStep) is protected by a may_proceed(ctx) gating helper.

These runtime gates consider:

File system status (including empty vs empty-like classification)
Content status (readability, encoding, newline policy)
Comparison result and update intent
Runtime policy permissions (for example allow_insert_into_empty_like)
Feasibility (can_change)

This ensures TopMark only mutates files when explicitly permitted by both runtime policy and pipeline feasibility rules.

Observability¶

Run with -v or --verbose to increase TEXT output detail and observe:

Discovery anchor and workspace base
Configuration chain (root → current)
Normalization of pattern-file paths
Active policy and per-file-type overrides
Canonical file type identities used by filters and policy lookup

Notes:

Verbosity (-v) affects only human TEXT output.
Markdown output is document-oriented and does not use verbosity for progressive disclosure.
Machine-readable formats are unaffected by verbosity controls.

Examples¶

Allow headers in empty Python files¶

[tool.topmark.policy_by_type."python"]
allow_header_in_empty_files = true

Now, even an empty __init__.py or placeholder.py can receive a header.

Runtime configuration model¶