API Helpers¶

consist.api provides top-level helper functions that forward to the active tracker. Use these when you want concise scripts and notebooks without passing a tracker object to every call.

When to use helpers vs class methods¶

Prefer	When
`consist.*` helpers	Notebook/script workflows where `use_tracker(...)` is already set
`Tracker.*` methods	Library/application code where explicit object wiring is preferred

High-traffic helpers¶

Run execution: consist.run, consist.scenario, consist.trace, consist.start_run
Input wiring: consist.ref, consist.refs
Context and output paths: consist.use_tracker, consist.output_dir, consist.output_path
Artifact logging/loading: consist.log_artifact, consist.log_dataframe, consist.load, consist.load_df
Transit feeds: consist.canonicalize_gtfs
Recovery/staging/archive: consist.hydrate_run_outputs, consist.hydrate_run_outputs_to_destinations, consist.materialize_run_outputs, consist.materialize_artifact, consist.stage_artifact, consist.stage_inputs, consist.set_artifact_recovery_roots, consist.register_artifact_recovery_copy, consist.register_run_output_recovery_copies, consist.archive_artifact, consist.archive_run_outputs, consist.archive_current_run_outputs, consist.archive_run_output_files
Querying: consist.find_run, consist.find_runs, consist.find_latest_run, consist.find_matching_run, consist.find_matching_runs, consist.run_query, consist.run_set, consist.get_run_result, consist.config_run_query, consist.config_run_rows, consist.ibis_connection, consist.ibis_view, consist.ibis_grouped_view

consist.find_runs(...), consist.find_run(...), and consist.find_latest_run(...) forward standard run filters, including stage= and phase= for workflow-level queries. consist.find_matching_run(...) and consist.find_matching_runs(...) add a restart-friendly semantic matcher with optional cache_epoch= and caller-owned run_scope= prefix filtering.

consist.archive_run_outputs(...) and consist.archive_current_run_outputs(...) return ArchivedOutputs, a dict-like mapping of archived paths whose .paths attribute exposes the same read-only path mapping and whose .outputs attribute exposes the refreshed artifacts for downstream inputs=... reuse.

consist.archive_run_output_files(...) is the conservative, file-only archive helper for a completed run. It returns an ArchivedRunOutputFilesReport, a read-only mapping of per-key results. It never overwrites an existing archive target: with preserve_existing=True, a matching target can be verified and registered; otherwise that key remains a reported outcome. Inspect .complete and each result before treating the archive as ready. .complete reports this call's selected files and metadata registration; it does not establish durable workflow state or prove that another application consumed the archive.

Minimal runnable helper workflow¶

from pathlib import Path
import consist
from consist import Tracker

tracker = Tracker(run_dir="./runs", db_path="./provenance.duckdb")

def step() -> Path:
    out = consist.output_path("step", ext="txt")
    out.write_text("done\n")
    return out

with consist.use_tracker(tracker):
    result = consist.run(fn=step, outputs=["step"])
    latest = consist.find_run(run_id=result.run.id)

print(result.outputs["step"].path)
print(latest.id if latest else None)

For class-level equivalents, see Tracker and Workflow Contexts.

`RelationConnectionLeakWarning` ¶

Bases: RuntimeWarning

Warning emitted when relation connections appear to accumulate.

`view(model, name=None)` ¶

Create a SQLModel class backed by a Consist hybrid view.

This is a convenience wrapper around the view factory that lets you define a SQLModel schema for a concept (e.g., a canonicalized config table, an ingested dataset, or a computed artifact view) and then query it as a normal SQLModel table. The returned class is a dynamic subclass with table=True that points at a database view, so you can use it in sqlmodel.Session queries without creating a physical table.

If you need explicit control over view naming or want to create multiple named views for the same concept, use Tracker.create_view(...) or Tracker.view(...) directly.

Parameters:

Name	Type	Description	Default
`model`	`Type[T]`	Base SQLModel describing the schema (columns and types).	required
`name`	`Optional[str]`	Optional override for the generated view name (defaults to the model's table name).	`None`

Returns:

Type	Description
`Type[T]`	SQLModel subclass with `table=True` pointing at the hybrid view.

Examples:

from sqlmodel import Session, select
from consist import view
from consist.models.activitysim import ActivitySimConstantsCache

# Create a dynamic view class for querying constants
ConstantsView = view(ActivitySimConstantsCache)

with Session(tracker.engine) as session:
    rows = session.exec(
        select(ConstantsView)
        .where(ConstantsView.key == "sample_rate")
    ).all()

`use_tracker(tracker)` ¶

Set a fallback (default) tracker for Consist API entrypoints.

This configures which tracker is used by consist.run(), consist.start_run(), etc. when called outside an active run context (i.e., when the tracker stack is empty). Once inside a run, the tracker becomes "active" via push_tracker() and is accessed by logging functions like consist.log_artifact().

`run(fn=None, name=None, *, tracker=None, adapter=None, identity_inputs=None, cache_options=None, output_policy=None, execution_options=None, **kwargs)` ¶

Execute a computational task as a tracked and cached Consist run.

This high-level entrypoint encapsulates the execution of a callable within a managed provenance context. It automates the lifecycle of the run, including signature computation for cache identity, artifact capture, and result persistence.

Parameters:

Name	Type	Description	Default
`fn`	`Optional[Callable]`	The computational function to execute.	`None`
`name`	`Optional[str]`	A semantic identifier for the run. If omitted, the function's `__name__` is utilized as the default.	`None`
`tracker`	`Optional[Tracker]`	The Tracker instance responsible for provenance and caching. If None, the active global tracker is resolved.	`None`
`adapter`	`Optional[ConfigAdapter]`	Config adapter used to resolve adapter-driven identity for the run.	`None`
`identity_inputs`	`IdentityInputs`	Additional hash-only identity inputs that contribute to cache keys.	`None`
`cache_options`	`Optional[CacheOptions]`	Grouped cache controls for run execution.	`None`
`output_policy`	`Optional[OutputPolicyOptions]`	Grouped output mismatch/missing policy controls.	`None`
`execution_options`	`Optional[ExecutionOptions]`	Grouped runtime execution controls. For path-bound workflows, this is also where you request local input staging via `ExecutionOptions(input_binding="paths", input_materialization="requested", input_paths={...})`.	`None`
`**kwargs`	`Any`	Arguments forwarded to `Tracker.run`, including `inputs`, `config`, `tags`, and other core run metadata. Legacy direct policy kwargs are rejected (for example `cache_mode`, `output_missing`, `inject_context`). Use options objects instead: `cache_options=CacheOptions(...)`, `output_policy=OutputPolicyOptions(...)`, and `execution_options=ExecutionOptions(...)`.	`{}`

Returns:

Type	Description
`RunResult`	A container holding the function's return value and the immutable `Run` record.

Raises:

Type	Description
`TypeError`	Raised when legacy run-policy kwargs are provided directly instead of via options objects.

`ref(run_result, key=None)` ¶

Select a specific output artifact from a RunResult.

Parameters:

Name	Type	Description	Default
`run_result`	`RunResult`	Result object returned by `consist.run`/`tracker.run`.	required
`key`	`Optional[str]`	Output key to select. If omitted, `run_result` must have exactly one output.	`None`

Returns:

Type	Description
`Artifact`	Selected artifact reference, suitable for `inputs=...`.

`refs(*selectors, **aliases)` ¶

refs(run_result: RunResult) -> Dict[str, Artifact]

refs(
    run_result: RunResult, /, *keys: str
) -> Dict[str, Artifact]

refs(
    run_result: RunResult, alias_map: Mapping[str, str]
) -> Dict[str, Artifact]

refs(
    *selectors: tuple[Any, ...],
    **aliases: Union[RunResult, tuple[RunResult, str]],
) -> Dict[str, Artifact]

Build an inputs={...} mapping from one or more RunResult objects.

Supported forms

Single result shorthand: consist.refs(prep, "households", "persons")
Single result alias mapping: consist.refs(prep, {"hh": "households", "pp": "persons"})
Multi-result positional groups: consist.refs((prep, "households"), (skim, "skims"))
Keyword aliases: consist.refs(hh=(prep, "households"), skims=(skim, "skims"))

Notes

Bare string selectors (for example consist.refs("households")) are not supported. Provide a RunResult context via positional or keyword forms.

Returns:

Type	Description
`Dict[str, Artifact]`	Mapping suitable for `inputs=...`.

`trace(name, *, tracker=None, adapter=None, identity_inputs=None, **kwargs)` ¶

Create a nested tracing context within an active run.

Use trace to break down a large run into smaller, logical steps. Each traced step is recorded as a sub-run with its own inputs and outputs.

Parameters:

Name	Type	Description	Default
`name`	`str`	Semantic name for this step/trace.	required
`tracker`	`Optional[Tracker]`	The tracker instance to use.	`None`
`adapter`	`Optional[ConfigAdapter]`	Config adapter used to resolve adapter-driven identity for the trace.	`None`
`identity_inputs`	`IdentityInputs`	Additional hash-only identity inputs that contribute to cache keys.	`None`
`**kwargs`	`Any`	Additional metadata or parameters for this trace.	`{}`

Yields:

Type	Description
`Tracker`	The active tracker instance.

`start_run(run_id, model, tracker=None, **kwargs)` ¶

Initiate and manage a Consist run.

This context manager marks the beginning of a discrete unit of work (a "run"). All artifacts logged within this context will be associated with this run.

Parameters:

Name	Type	Description	Default
`run_id`	`str`	A unique identifier for this run.	required
`model`	`str`	The name of the model or system being run.	required
`tracker`	`Optional[Tracker]`	The tracker instance to use. Defaults to the active global tracker.	`None`
`**kwargs`	`Any`	Additional parameters for the run, such as `tags`, `config`, or `inputs`.	`{}`

Yields:

Type	Description
`Tracker`	The active tracker instance.

Example

with consist.start_run("run_123", "my_model"):
    consist.log_artifact("data.csv", "input_data")

`define_step(*, model=None, name_template=None, outputs=None, schema_outputs=None, output_paths=None, output_sets=None, inputs=None, input_keys=None, optional_input_keys=None, config=None, adapter=None, identity_inputs=None, facet=None, facet_index=None, cache_mode=None, cache_hydration=None, cache_version=None, validate_cached_outputs=None, input_binding=None, load_inputs=None, tags=None, facet_from=None, facet_schema_version=None, description=None, **extra)` ¶

Attach metadata to a function without changing execution behavior.

This decorator lets you attach defaults such as outputs or tags to a function. Tracker.run and ScenarioContext.run read this metadata. Callable values are resolved at runtime with a StepContext.

Use output_sets for defaults that describe one logical output written as many files. Each OutputSet requires root and include. Optional fields such as expected_members and expected_count add completeness checks but are not required.

`require_runtime_kwargs(*names)` ¶

Enforce the presence of required runtime_kwargs when a step is executed.

Use this decorator to catch missing runtime-only inputs early, before executing a run. It validates that each required name is present in the runtime_kwargs passed to Tracker.run or ScenarioContext.run.

`scenario(name, tracker=None, *, enabled=True, step_tags=None, step_facet=None, **kwargs)` ¶

Context manager for grouping multiple execution steps into a semantic scenario.

A scenario creates a parent "header" run that aggregates metadata, lineage, and artifacts from all nested steps. This enables multi-stage simulations to be tracked as a single unit while maintaining fine-grained provenance for each internal component.

Parameters:

Name	Type	Description	Default
`name`	`str`	Unique identifier or name for the scenario header.	required
`tracker`	`Optional[Tracker]`	The Tracker instance to use for persistence. Defaults to the active global tracker.	`None`
`enabled`	`bool`	If False, returns a no-op context that executes functions without recording provenance, preserving the Scenario API ergonomics for debugging.	`True`
`step_tags`	`Optional[list[str]]`	Default tags applied to child `scenario.run(...)` and `scenario.trace(...)` calls. These defaults do not apply to the scenario header run itself.	`None`
`step_facet`	`Optional[dict[str, Any]]`	Default facet mapping applied to child `scenario.run(...)` and `scenario.trace(...)` calls. These defaults do not apply to the scenario header run itself, and per-step `facet=...` values take precedence on key collisions.	`None`
`**kwargs`	`Any`	Additional metadata (e.g., tags, config) forwarded to the header run.	`{}`

Yields:

Type	Description
`ScenarioContext`	An active scenario context with an internal Coupler for artifact chaining.

`single_step_scenario(name, step_name=None, tracker=None, **kwargs)` ¶

Convenience wrapper that exposes a single step scenario.

Parameters:

Name	Type	Description	Default
`name`	`str`	Name of the scenario header.	required
`step_name`	`Optional[str]`	Name for the single step; defaults to `name`.	`None`
`tracker`	`Optional[Tracker]`	Tracker to execute the scenario; defaults to the active tracker.	`None`
`**kwargs`	`Any`	Arguments forwarded to `Tracker.scenario`.	`{}`

Yields:

Type	Description
`ScenarioContext`	Scenario context manager for the single step.

`current_tracker()` ¶

Retrieves the active Tracker instance from the global context.

If no run is active, this function falls back to the default tracker (if set via consist.use_tracker or consist.set_current_tracker).

Returns:

Type	Description
`Tracker`	The `Tracker` instance currently active in the execution context.

Raises:

Type	Description
`RuntimeError`	If no `Tracker` is active in the current context and no default tracker has been configured.

`current_run()` ¶

Return the active Run record if one is in progress, otherwise None.

A Run record contains metadata about the current execution, such as its unique ID, model name, and start time.

`current_consist()` ¶

Return the active ConsistRecord if one is in progress, otherwise None.

The ConsistRecord is the internal state object that tracks the active run's inputs, outputs, and metadata during execution.

`output_dir(namespace=None)` ¶

Resolve the managed output directory for the active run context.

Parameters:

Name	Type	Description	Default
`namespace`	`Optional[str]`	Optional relative subdirectory under the active run's managed output directory.	`None`

Returns:

Type	Description
`Path`	Absolute path to the managed output directory.

Raises:

Type	Description
`RuntimeError`	If called outside an active run context.

`output_path(key, ext='parquet')` ¶

Resolve a deterministic managed output path for the active run context.

Parameters:

Name	Type	Description	Default
`key`	`str`	Artifact key used as the output filename stem.	required
`ext`	`str`	File extension to append to the output filename.	`"parquet"`

Returns:

Type	Description
`Path`	Absolute managed output path for the current run.

Raises:

Type	Description
`RuntimeError`	If called outside an active run context.

`cached_artifacts(direction='output')` ¶

Returns hydrated cached artifacts for the active run, if any.

Parameters:

Name	Type	Description	Default
`direction`	`str`	"output" or "input".	`"output"`

Returns:

Type	Description
`Dict[str, Artifact]`	Mapping from artifact key to Artifact, or empty dict if no cache hit.

`cached_output(key=None)` ¶

Fetch a hydrated cached output artifact for the active run.

Parameters:

Name	Type	Description	Default
`key`	`Optional[str]`	Specific artifact key to look up; defaults to the first available artifact.	`None`

Returns:

Type	Description
`Optional[Artifact]`	Cached artifact instance or `None` if no cache hit exists.

`get_run_result(run_id, *, keys=None, validate='lazy', tracker=None)` ¶

Retrieve a historical run as a RunResult with selected outputs.

Parameters:

Name	Type	Description	Default
`run_id`	`str`	Identifier of the historical run.	required
`keys`	`Optional[Iterable[str]]`	Optional output keys to include. If omitted, includes all outputs.	`None`
`validate`	`(lazy, strict, none)`	Output validation policy forwarded to `Tracker.get_run_result`.	`"lazy"`
`tracker`	`Optional[Tracker]`	Tracker instance to query. If omitted, resolves the active/default tracker.	`None`

Returns:

Type	Description
`RunResult`	Historical run metadata plus selected outputs.

`get_artifact(run_id, key=None, key_contains=None, direction='output')` ¶

Retrieve a single artifact from a historical run.

Parameters:

Name	Type	Description	Default
`run_id`	`str`	Identifier of the run that produced the artifact.	required
`key`	`Optional[str]`	Exact artifact key to match.	`None`
`key_contains`	`Optional[str]`	Substring filter for artifact keys.	`None`
`direction`	`str`	Either "input" or "output".	`"output"`

Returns:

Type	Description
`Optional[Artifact]`	Matching artifact or `None` if not found.

`register_artifact_facet_parser(prefix, parser_fn, *, tracker=None)` ¶

Register a key-prefix parser for deriving artifact facets at log time.

The parser is invoked when logging an artifact without an explicit facet=.

`log_artifact(path, key=None, direction='output', schema=None, strict_schema=False, driver=None, content_hash=None, force_hash_override=False, validate_content_hash=False, reuse_if_unchanged=False, reuse_scope='same_uri', parent_artifact_id=None, facet=None, facet_schema_version=None, facet_index=False, *, enabled=True, **meta)` ¶

Logs an artifact (file or data reference) to the currently active run.

This function is a convenient proxy to consist.core.tracker.Tracker.log_artifact. It automatically links the artifact to the current run context, handles path virtualization, and performs lineage discovery.

Parameters:

Name	Type	Description	Default
`path`	`ArtifactRef`	A file path (str/Path) or an existing `Artifact` reference to be logged.	required
`key`	`Optional[str]`	A semantic, human-readable name for the artifact (e.g., "households"). Required if `path` is a path-like (str/Path).	`None`
`direction`	`str`	Specifies whether the artifact is an "input" or "output" for the current run. Defaults to "output".	`"output"`
`schema`	`Optional[Type[SQLModel]]`	An optional SQLModel class that names the artifact's logical schema. Its name will be stored in artifact metadata. This tags metadata; it does not validate file contents by default.	`None`
`strict_schema`	`bool`	Whether a schema tag should also mark the artifact as strict for validation workflows. Schema tagging itself is non-strict by default.	`False`
`driver`	`Optional[str]`	Explicitly specify the driver (e.g., 'h5_table'). If None, the driver is inferred from the file extension.	`None`
`content_hash`	`Optional[str]`	Precomputed artifact fingerprint to use instead of hashing the path on disk. When provided, Consist stores it on the canonical public fingerprint field, `artifact.hash`.	`None`
`force_hash_override`	`bool`	If True, overwrite an existing artifact hash when it differs from `content_hash`. By default, mismatched overrides are ignored with a warning.	`False`
`validate_content_hash`	`bool`	If True, verify `content_hash` against the on-disk data and raise on mismatch.	`False`
`reuse_if_unchanged`	`bool`	Deprecated for outputs. Consist always creates a fresh output artifact row; identical bytes are deduplicated via `artifact.content_id`. Setting this on outputs emits a warning and does not reuse prior rows. Input-side behavior is unaffected.	`False`
`reuse_scope`	`(same_uri, any_uri)`	Deprecated for outputs. `any_uri` is ignored for outputs; content identity (`content_id`) governs deduplication. Input-side behavior is unaffected.	`"same_uri"`
`parent_artifact_id`	`UUID \| None`	Canonical parent artifact relation for child/member artifacts.	`None`
`facet`	`Optional[Any]`	Optional artifact-level facet payload (dict or Pydantic model).	`None`
`facet_schema_version`	`Optional[Union[str, int]]`	Optional schema version for artifact facet compatibility.	`None`
`facet_index`	`bool`	If True, flatten scalar facet values for fast artifact querying.	`False`
`enabled`	`bool`	If False, returns a noop artifact object without requiring an active run.	`True`
`**meta`	`Any`	Additional key-value pairs to store in the artifact's flexible `meta` field. `recovery_roots` may be supplied here and will be normalized to an ordered list of absolute filesystem roots.	`{}`

Returns:

Type	Description
`Artifact`	The created or updated `Artifact` object.

Raises:

Type	Description
`RuntimeError`	If called outside an active run context.
`ValueError`	If `key` is not provided when `path` is a path-like (str/Path).

`log_artifacts(outputs, *, direction='output', driver=None, metadata_by_key=None, facets_by_key=None, facet_schema_versions_by_key=None, facet_index=False, reuse_if_unchanged=False, reuse_scope='same_uri', enabled=True, **shared_meta)` ¶

Log multiple artifacts in a single call for efficiency.

This function is a convenient proxy to consist.core.tracker.Tracker.log_artifacts. It logs a batch of related artifacts with optional per-key metadata customization.

Parameters:

Name	Type	Description	Default
`outputs`	`Mapping[str, ArtifactRef]`	Mapping of key -> path/Artifact to log. Keys must be strings and explicitly chosen by the caller (not inferred from filenames).	required
`direction`	`str`	"input" or "output" for the current run context.	`"output"`
`driver`	`Optional[str]`	Explicitly specify driver for all artifacts. If None, inferred from file extension.	`None`
`metadata_by_key`	`Optional[Mapping[str, Dict[str, Any]]]`	Per-key metadata overrides applied on top of shared metadata.	`None`
`facets_by_key`	`Optional[Mapping[str, Any]]`	Per-key artifact facet payloads.	`None`
`facet_schema_versions_by_key`	`Optional[Mapping[str, Union[str, int]]]`	Optional per-key schema versions for artifact facets.	`None`
`facet_index`	`bool`	Whether to index scalar artifact facet fields in `artifact_kv`.	`False`
`reuse_if_unchanged`	`bool`	Deprecated for outputs. Batch output logging still creates a fresh artifact row per call; identical bytes are deduplicated via `artifact.content_id`. Setting this on outputs emits a warning and does not reuse prior rows. Input-side behavior is unaffected.	`False`
`reuse_scope`	`(same_uri, any_uri)`	Deprecated for outputs. `any_uri` is ignored for outputs; deduplication is governed by `content_id` instead. Input-side behavior is unaffected.	`"same_uri"`
`enabled`	`bool`	If False, returns noop artifact objects without requiring an active run.	`True`
`**shared_meta`	`Any`	Metadata key-value pairs applied to ALL logged artifacts.	`{}`

Returns:

Type	Description
`Mapping[str, ArtifactLike]`	Mapping of key -> logged Artifact-like objects.

Raises:

Type	Description
`RuntimeError`	If called outside an active run context.
`ValueError`	If metadata_by_key contains keys not in outputs, or if any value is None.
`TypeError`	If mapping keys are not strings.

Examples:

Log multiple outputs with shared metadata:

artifacts = consist.log_artifacts(
    {
        "persons": "results/persons.parquet",
        "households": "results/households.parquet",
        "jobs": "results/jobs.parquet"
    },
    year=2030,
    scenario="base"
)
# All three artifacts get year=2030 and scenario="base"

Mix shared and per-key metadata:

artifacts = consist.log_artifacts(
    {
        "persons": "output/persons.parquet",
        "households": "output/households.parquet"
    },
    metadata_by_key={
        "households": {"role": "primary_unit", "weight": 1.0}
    },
    dataset_version="v2",
    simulation_id="run_001"
)
# persons gets: dataset_version="v2", simulation_id="run_001"
# households gets: dataset_version="v2", simulation_id="run_001",
#                  role="primary_unit", weight=1.0

`log_input(path, key=None, *, schema=None, driver=None, content_hash=None, force_hash_override=False, validate_content_hash=False, facet=None, facet_schema_version=None, facet_index=False, enabled=True, **meta)` ¶

Log an input artifact to the active run.

An input artifact represents a data source that the current run reads from. Logging it creates a lineage link, allowing Consist to track which versions of data produced which results.

Parameters:

Name	Type	Description	Default
`path`	`ArtifactRef`	File path (str/Path) or an existing `Artifact` object.	required
`key`	`Optional[str]`	Semantic name for the artifact (e.g. "raw_households"). Required if `path` is a file path.	`None`
`schema`	`Optional[Type[SQLModel]]`	Optional SQLModel class defining the data structure.	`None`
`driver`	`Optional[str]`	Explicit format driver (e.g. "parquet"). Inferred from extension if None.	`None`
`content_hash`	`Optional[str]`	Precomputed artifact fingerprint to avoid re-hashing large files. When provided, Consist stores it on the canonical public fingerprint field, `artifact.hash`.	`None`
`force_hash_override`	`bool`	Overwrite existing hash in the database if different from `content_hash`.	`False`
`validate_content_hash`	`bool`	Re-hash the file on disk to ensure it matches the provided `content_hash`.	`False`
`facet`	`Optional[Any]`	Optional artifact-level facet payload.	`None`
`facet_schema_version`	`Optional[Union[str, int]]`	Optional facet schema version.	`None`
`facet_index`	`bool`	Whether to index scalar facet fields for querying.	`False`
`enabled`	`bool`	If False, returns a dummy artifact object for disconnected execution.	`True`
`**meta`	`Any`	Additional metadata fields to store with the artifact.	`{}`

Returns:

Type	Description
`ArtifactLike`	The logged artifact reference.

`log_output(path, key=None, *, artifact_kind='file', schema=None, driver=None, content_hash=None, force_hash_override=False, validate_content_hash=False, reuse_if_unchanged=False, reuse_scope='same_uri', facet=None, facet_schema_version=None, facet_index=False, enabled=True, **meta)` ¶

Log an output artifact produced by the current run.

Parameters:

Name	Type	Description	Default
`path`	`ArtifactRef`	File path (str/Path) or an existing `Artifact` object.	required
`key`	`Optional[str]`	Semantic name for the artifact (e.g. "processed_results"). Required if `path` is a file path.	`None`
`artifact_kind`	`(file, directory)`	Declare an immutable directory artifact with a complete persisted tree manifest. The artifact retains its content driver (for example, `"zarr"`) for loading and type narrowing.	`"file"`
`schema`	`Optional[Type[SQLModel]]`	Optional SQLModel class defining the data structure.	`None`
`driver`	`Optional[str]`	Explicit format driver (e.g. "parquet"). Inferred from extension if None.	`None`
`content_hash`	`Optional[str]`	Precomputed artifact fingerprint to avoid re-hashing large files. When provided, Consist stores it on the canonical public fingerprint field, `artifact.hash`.	`None`
`force_hash_override`	`bool`	Overwrite existing hash in the database if different from `content_hash`.	`False`
`validate_content_hash`	`bool`	Re-hash the file on disk to ensure it matches the provided `content_hash`.	`False`
`reuse_if_unchanged`	`bool`	Deprecated for outputs. A fresh output artifact row is always created; identical bytes share `content_id`. Setting this emits a warning and does not reuse prior rows.	`False`
`reuse_scope`	`(same_uri, any_uri)`	Deprecated for outputs. `any_uri` is ignored; deduplication is by `content_id`.	`"same_uri"`
`facet`	`Optional[Any]`	Optional artifact-level facet payload.	`None`
`facet_schema_version`	`Optional[Union[str, int]]`	Optional facet schema version.	`None`
`facet_index`	`bool`	Whether to index scalar facet fields for querying.	`False`
`enabled`	`bool`	If False, returns a dummy artifact object.	`True`
`**meta`	`Any`	Additional metadata fields to store.	`{}`

Returns:

Type	Description
`ArtifactLike`	The logged artifact reference.

`log_dataframe(df, key, schema=None, direction='output', tracker=None, path=None, driver=None, meta=None, parent_artifact_id=None, **to_file_kwargs)` ¶

Serialize a DataFrame, log it as an artifact, and trigger optional ingestion.

Parameters:

Name	Type	Description	Default
`df`	`DataFrame`	Data to persist.	required
`key`	`str`	Logical artifact key.	required
`schema`	`Optional[Type[SQLModel]]`	Schema used for ingestion, if provided.	`None`
`direction`	`str`	Artifact direction relative to the run.	`"output"`
`tracker`	`Optional[Tracker]`	Tracker instance to use; defaults to the active tracker. If None and no active run context exists, raises RuntimeError.	`None`
`path`	`Optional[Union[str, Path]]`	Output path; defaults to `<run_dir>/outputs/<run_subdir>/<key>.<driver>` for the active run as determined by the tracker's run subdir configuration.	`None`
`driver`	`Optional[str]`	File format driver (e.g., "parquet" or "csv").	`None`
`meta`	`Optional[Dict[str, Any]]`	Additional metadata for the artifact.	`None`
`parent_artifact_id`	`UUID \| None`	Canonical parent artifact relation for child/member artifacts.	`None`
`**to_file_kwargs`	`Any`	Keyword arguments forwarded to `pd.DataFrame.to_parquet` or `to_csv`.	`{}`

Returns:

Type	Description
`Artifact`	The artifact logged for the written dataset.

Raises:

Type	Description
`ValueError`	If the requested driver is unsupported.

`log_meta(**kwargs)` ¶

Updates the active run's metadata with the provided key-value pairs.

This function is a convenient proxy to consist.core.tracker.Tracker.log_meta. It allows users to log additional information about the current run, such as performance metrics, experimental parameters, or tags, directly to the run's metadata. This information is then persisted to both the JSON log and the DuckDB database.

Parameters:

Name	Type	Description	Default
`**kwargs`	`Any`	Arbitrary key-value pairs to merge into the `meta` dictionary of the current run. Existing keys will be updated, and new keys will be added.	`{}`

Raises:

Type	Description
`RuntimeError`	If called when no `Tracker` is active in the current context.

`ingest(artifact, data=None, schema=None, run=None)` ¶

Ingests data associated with an Artifact into the active run's database.

This function is a convenient proxy to consist.core.tracker.Tracker.ingest. It materializes data into the DuckDB, leveraging dlt for efficient loading and injecting provenance information. This is part of the "Hot Data Strategy".

Parameters:

Name	Type	Description	Default
`artifact`	`Artifact`	The artifact object representing the data being ingested.	required
`data`	`Optional[Union[Iterable[Dict[str, Any]], Any]]`	The data to ingest. Can be an iterable of dictionaries (rows), a file path (str or Path), a Pandas DataFrame, or any other data type that `dlt` can handle. If `None`, Consist attempts to read the data directly from the artifact's resolved file path.	`None`
`schema`	`Optional[Type[SQLModel]]`	An optional SQLModel class that defines the expected schema for the ingested data. If provided, `dlt` will use this for strict validation and schema inference.	`None`
`run`	`Optional[Run]`	The `Run` context for ingestion. If provided, the data will be tagged with this run's ID (Offline Mode). If `None`, it defaults to the currently active run (Online Mode).	`None`

Returns:

Type	Description
`Any`	The result information from the underlying `dlt` ingestion process.

Raises:

Type	Description
`RuntimeError`	If no database is configured for the active `Tracker` or if `ingest` is called outside of an active run context.
`Exception`	Any exception raised by the underlying `dlt` ingestion process.

`canonicalize_gtfs(feed_paths, *, service_date=None, feed_keys=None, run=None, run_id=None, key=GTFS_BUNDLE_SOURCE_KEY, ingest=True, profile_schema=False, log_sources=True, tracker=None)` ¶

Canonicalize GTFS feeds with tracker-managed provenance and ingestion.

This is a convenience proxy to :meth:Tracker.canonicalize_gtfs.

`register_views(*models)` ¶

Register hybrid view models for the active tracker.

Parameters:

Name	Type	Description	Default
`*models`	`Type[SQLModel]`	SQLModel classes describing the schema of hybrid views.	`()`

Returns:

Type	Description
`Dict[str, Type[SQLModel]]`	Mapping from model class name to the generated view model.

`find_run(tracker=None, **filters)` ¶

Convenience proxy for Tracker.find_run.

Parameters:

Name	Type	Description	Default
`tracker`	`Optional[Tracker]`	Tracker instance to query; defaults to the active tracker.	`None`
`**filters`	`Any`	Filter values forwarded to `Tracker.find_run`.	`{}`

Returns:

Type	Description
`Optional[Run]`	Matching run or `None` when no match exists.

`find_runs(tracker=None, **filters)` ¶

Convenience proxy for Tracker.find_runs.

Parameters:

Name	Type	Description	Default
`tracker`	`Optional[Tracker]`	Tracker instance to query; defaults to the active tracker.	`None`
`**filters`	`Any`	Filter values forwarded to `Tracker.find_runs`. This includes workflow filters such as `stage` and `phase` and facet predicates via `facet={...}`.	`{}`

Returns:

Type	Description
`Union[list[Run], Dict[Hashable, Run]]`	Results returned by `Tracker.find_runs`.

`find_latest_run(tracker=None, **filters)` ¶

Convenience proxy for Tracker.find_latest_run.

Parameters:

Name	Type	Description	Default
`tracker`	`Optional[Tracker]`	Tracker instance to query; defaults to the active tracker.	`None`
`**filters`	`Any`	Filter values forwarded to `Tracker.find_latest_run`. This includes workflow filters such as `stage` and `phase` and facet predicates via `facet={...}`.	`{}`

Returns:

Type	Description
`Run`	The latest matching run.

`find_matching_run(tracker=None, **filters)` ¶

Convenience proxy for Tracker.find_matching_run.

Parameters:

Name	Type	Description	Default
`tracker`	`Optional[Tracker]`	Tracker instance to query; defaults to the active tracker.	`None`
`**filters`	`Any`	Semantic filters forwarded to `Tracker.find_matching_run`.	`{}`

Returns:

Type	Description
`Optional[Run]`	Latest matching run, or `None` when no run matches.

`find_matching_runs(tracker=None, **filters)` ¶

Convenience proxy for Tracker.find_matching_runs.

Parameters:

Name	Type	Description	Default
`tracker`	`Optional[Tracker]`	Tracker instance to query; defaults to the active tracker.	`None`
`**filters`	`Any`	Semantic filters forwarded to `Tracker.find_matching_runs`.	`{}`

Returns:

Type	Description
`list[Run]`	Matching runs in deterministic recency order.

`run_set(tracker=None, label=None, **filters)` ¶

Convenience proxy for Tracker.run_set.

Parameters:

Name	Type	Description	Default
`tracker`	`Optional[Tracker]`	Tracker instance to query; defaults to the active tracker.	`None`
`label`	`Optional[str]`	Optional label attached to the returned RunSet.	`None`
`**filters`	`Any`	Filter values forwarded to `Tracker.run_set`.	`{}`

Returns:

Type	Description
`RunSet`	A tracker-backed RunSet for grouping/alignment workflows.

Notes

This convenience helper resolves a tracker from context and forwards to Tracker.run_set(...).

`db_session(tracker=None)` ¶

Provide a SQLModel Session connected to the tracker's database.

Parameters:

Name	Type	Description	Default
`tracker`	`Optional[Tracker]`	Tracker instance supplying the engine; defaults to active tracker.	`None`

Yields:

Type	Description
`Session`	SQLModel session bound to the tracker engine.

`run_query(query, tracker=None)` ¶

Execute a SQLModel/SQLAlchemy query via the tracker engine.

Parameters:

Name	Type	Description	Default
`query`	`Executable`	Query object (`select`, etc.).	required
`tracker`	`Optional[Tracker]`	Tracker instance supplying the engine; defaults to the active tracker.	`None`

Returns:

Type	Description
`list`	Results of the executed query.

`config_run_query(table, *, link_table, table_name=None, columns=None, where=None, join_on=None)` ¶

Build a query joining a config cache table to its run-link table.

This is a convenience helper for config adapter tables that are deduplicated by content hash and linked to runs via a separate link table.

Parameters:

Name	Type	Description	Default
`table`	`Type[SQLModel]`	Config cache table to query (e.g., ActivitySimCoefficientsCache).	required
`link_table`	`Type[SQLModel]`	Link table storing run_id/content_hash (e.g., ActivitySimConfigIngestRunLink).	required
`table_name`	`Optional[str]`	Optional filter for link_table.table_name. Defaults to table.tablename if link_table has a table_name column.	`None`
`columns`	`Optional[Iterable[Any]]`	Columns to select. Defaults to (link_table.run_id, table).	`None`
`where`	`Optional[Union[Iterable[Any], Any]]`	Optional filter clauses to apply to the query.	`None`
`join_on`	`Optional[Iterable[str]]`	Column names to join on. Defaults to content_hash and file_name when available.	`None`

Returns:

Type	Description
`Executable`	SQL query joining the config table to runs.

`config_run_rows(table, *, link_table, table_name=None, columns=None, where=None, join_on=None, tracker=None)` ¶

Execute a config-to-run join query and return the results as a list of rows.

This is a high-level wrapper around config_run_query and run_query. It is frequently used by integration adapters (like ActivitySim or BEAM) to retrieve configuration parameters that were active during a specific run.

Parameters:

Name	Type	Description	Default
`table`	`Type[SQLModel]`	The configuration cache table to query.	required
`link_table`	`Type[SQLModel]`	The table that links configuration entries to specific run IDs.	required
`table_name`	`Optional[str]`	Optional filter for the link table's `table_name` column.	`None`
`columns`	`Optional[Iterable]`	Specific columns to select. Defaults to the run ID and the config record.	`None`
`where`	`Optional[Clause]`	Additional SQLAlchemy filter clauses.	`None`
`join_on`	`Optional[Iterable[str]]`	Column names to join on (e.g. `["content_hash"]`).	`None`
`tracker`	`Optional[Tracker]`	The tracker to use for database access.	`None`

Returns:

Type	Description
`list`	The results of the query.

`pivot_facets(*, namespace, keys, value_column='value_num', value_columns=None, label_prefix='', label_map=None, run_id_label='run_id', table=RunConfigKV)` ¶

Build a pivoted facet subquery keyed by run_id.

This is a convenience helper for turning flattened config facets (run_config_kv) into a wide table suitable for joins.

Parameters:

Name	Type	Description	Default
`namespace`	`Optional[str]`	Facet namespace to filter by (typically the model name). If `None`, the namespace filter is skipped.	required
`keys`	`Iterable[str]`	Facet keys to pivot into columns.	required
`value_column`	`str`	Default value column to read from. Must be one of: `value_num`, `value_str`, `value_bool`, `value_json`.	`"value_num"`
`value_columns`	`Optional[Mapping[str, str]]`	Optional per-key override of `value_column`.	`None`
`label_prefix`	`str`	Optional prefix for pivoted column labels.	`""`
`label_map`	`Optional[Mapping[str, str]]`	Optional per-key label overrides.	`None`
`run_id_label`	`str`	Label to use for the run id column in the returned subquery.	`"run_id"`
`table`	`Type[RunConfigKV]`	Table/model providing the facet KV rows.	`RunConfigKV`

Returns:

Type	Description
`Any`	A SQLAlchemy subquery with columns: `run_id` and one column per key.

`capture_outputs(directory, pattern='*', recursive=False)` ¶

Context manager to automatically capture and log new or modified files in a directory within the current active run context.

This function is a convenient proxy to consist.core.tracker.Tracker.capture_outputs. It watches a specified directory for any file changes (creations or modifications) that occur within its with block. These changes are then automatically logged as output artifacts of the current Consist run.

Parameters:

Name	Type	Description	Default
`directory`	`Union[str, Path]`	The path to the directory to monitor for new or modified files.	required
`pattern`	`str`	A glob pattern (e.g., ".csv", "data_.parquet") to filter which files are captured within the specified directory. Defaults to all files.	`"*"`
`recursive`	`bool`	If True, the capture will recursively scan subdirectories within `directory` for changes.	`False`

Yields:

Type	Description
`OutputCapture`	An `OutputCapture` object containing a list of `Artifact` objects that were captured and logged after the `with` block finishes.

Raises:

Type	Description
`RuntimeError`	If `capture_outputs` is used outside of an active `start_run` context.

`load(artifact, tracker=None, *, db_fallback='inputs-only', **kwargs)` ¶

load(
    artifact: ZarrArtifact,
    tracker: Optional[Tracker] = None,
    *,
    db_fallback: str = "inputs-only",
    **kwargs: Any,
) -> XarrayDatasetType

load(
    artifact: NetCdfArtifact,
    tracker: Optional[Tracker] = None,
    *,
    db_fallback: str = "inputs-only",
    **kwargs: Any,
) -> XarrayDatasetType

load(
    artifact: OpenMatrixArtifact,
    tracker: Optional[Tracker] = None,
    *,
    db_fallback: str = "inputs-only",
    **kwargs: Any,
) -> XarrayDatasetType

load(
    artifact: SpatialArtifact,
    tracker: Optional[Tracker] = None,
    *,
    db_fallback: str = "inputs-only",
    **kwargs: Any,
) -> GeoDataFrameType

load(
    artifact: HdfStoreArtifact,
    tracker: Optional[Tracker] = None,
    *,
    db_fallback: str = "inputs-only",
    **kwargs: Any,
) -> pd.HDFStore

load(
    artifact: DataFrameArtifact,
    tracker: Optional[Tracker] = None,
    *,
    db_fallback: str = "inputs-only",
    **kwargs: Any,
) -> duckdb.DuckDBPyRelation

load(
    artifact: JsonArtifact,
    tracker: Optional[Tracker] = None,
    *,
    db_fallback: str = "inputs-only",
    **kwargs: Any,
) -> duckdb.DuckDBPyRelation

load(
    artifact: TabularArtifact,
    tracker: Optional[Tracker] = None,
    *,
    db_fallback: str = "inputs-only",
    **kwargs: Any,
) -> duckdb.DuckDBPyRelation

load(
    artifact: Artifact,
    tracker: Optional[Tracker] = None,
    *,
    db_fallback: str = "inputs-only",
    **kwargs: Any,
) -> LoadResult

load(
    artifact: ArtifactLike,
    tracker: Optional[Tracker] = None,
    *,
    db_fallback: str = "inputs-only",
    **kwargs: Any,
) -> LoadResult

Smart loader that retrieves data for an artifact from the best available source.

This function attempts to load the data associated with an Artifact object. It prioritizes loading from disk (raw format) if the file exists. If the file is missing but the artifact is marked as ingested, it can optionally recover the data from the Consist DuckDB database ("Ghost Mode"). By default, DB recovery is only allowed when the artifact is an input to an active, non-cached run.

Parameters:

Name	Type	Description	Default
`artifact`	`Artifact`	The Consist `Artifact` object whose data is to be loaded.	required
`tracker`	`Optional[Tracker]`	The `Tracker` instance to use for path resolution and database access. If `None`, the function attempts to use the active global tracker context. Explicitly passing a `tracker` is recommended for clarity or when no global context is available.	`None`
`db_fallback`	`str`	Controls when the loader is allowed to fall back to DuckDB ("Ghost Mode") when the file is missing but the artifact is marked as ingested. "inputs-only": allow DB fallback only if the artifact is declared as an input to the current active run AND the current run is not a cache hit. "always": allow DB fallback whenever `artifact.meta["is_ingested"]` is true and a tracker with a DB connection is available. "never": disable DB fallback entirely.	`"inputs-only"`
`**kwargs`	`Any`	Additional keyword arguments to pass to the underlying data loader function (e.g., `pd.read_parquet`, `pd.read_csv`, `xr.open_zarr`, `pd.read_sql`).	`{}`

Returns:

Type	Description
`LoadResult`	The loaded data, typically a DuckDB Relation for tabular data, an xarray Dataset for array formats, or another data object depending on the artifact's `driver` and the data format.

Raises:

Type	Description
`RuntimeError`	If no `Tracker` instance can be resolved (neither provided nor active in context) and the artifact's absolute path is not directly resolvable. Also if the artifact is marked as ingested but no tracker with a DB connection is available.
`FileNotFoundError`	If the artifact's data cannot be found on disk or recovered from the database.

`load_df(artifact, tracker=None, *, db_fallback='inputs-only', close=True, **kwargs)` ¶

Load a tabular artifact and return a pandas DataFrame.

Parameters:

Name	Type	Description	Default
`artifact`	`ArtifactLike`	Artifact to load.	required
`tracker`	`Optional[Tracker]`	Tracker to use for resolving paths or DB fallback.	`None`
`db_fallback`	`str`	Controls when DB recovery is allowed for ingested artifacts.	`"inputs-only"`
`close`	`bool`	Whether to close the underlying DuckDB connection after materialization when the load returns a Relation.	`True`
`**kwargs`	`Any`	Additional loader options.	`{}`

`load_relation(artifact, tracker=None, *, db_fallback='inputs-only', **kwargs)` ¶

Context manager that yields a DuckDB Relation and ensures the underlying connection is closed on exit.

`ibis_connection(tracker=None)` ¶

Return an Ibis DuckDB backend connected to the active Consist database.

`ibis_view(tracker=None, *, model, key=None)` ¶

Ensure a Consist view exists and return it as a native Ibis table.

`ibis_grouped_view(tracker=None, *, view_name, artifact_id, namespace=None, params=None, drivers=None, attach_facets=None, include_system_columns=True, mode='hybrid', if_exists='replace', missing_files='warn', run_id=None, parent_run_id=None, model=None, status=None, year=None, iteration=None, schema_compatible=False)` ¶

Create a grouped Consist view and expose it as a native Ibis table.

`hydrate_run_outputs(run_id, *, target_root, source_root=None, keys=None, preserve_existing=True, on_missing='warn', db_fallback='if_ingested', tracker=None)` ¶

Hydrate the linked outputs of a historical run into a new root.

This is the higher-level historical recovery helper. It performs the same physical copy/export work as materialize_run_outputs(...) but returns a keyed result with detached artifacts, destination paths, statuses, and resolvability information for each requested output.

Parameters:

Name	Type	Description	Default
`run_id`	`str`	Identifier of the run whose outputs should be restored.	required
`target_root`	`str \| Path`	Destination root for hydrated outputs.	required
`source_root`	`str \| Path \| None`	Optional alternate source root for filesystem-backed recovery.	`None`
`keys`	`Sequence[str] \| None`	Optional subset of output keys to restore.	`None`
`preserve_existing`	`bool`	If True, existing destinations are preserved and reported as reusable.	`True`
`on_missing`	`(warn, 'raise')`	Missing-source handling policy forwarded to the tracker.	`"warn"`
`db_fallback`	`(never, if_ingested)`	DB export fallback policy forwarded to the tracker.	`"never"`
`tracker`	`Optional[Tracker]`	Tracker instance to use. If omitted, resolves the active/default tracker.	`None`

Returns:

Type	Description
`HydratedRunOutputsResult`	Keyed hydration outcome for the selected run outputs. Each detached returned artifact preserves the canonical fingerprint surface on `artifact.hash`.

Examples:

>>> hydrated = consist.hydrate_run_outputs(
...     "prior_run_id",
...     keys=["persons"],
...     target_root="restored",
... )
>>> hydrated["persons"].status
'materialized_from_filesystem'
>>> hydrated["persons"].artifact.as_path()
PosixPath('.../restored/.../persons.parquet')

`hydrate_run_outputs_to_destinations(run_id, *, destinations_by_key, source_root=None, preserve_existing=True, on_missing='warn', db_fallback='if_ingested', tracker=None)` ¶

Hydrate selected historical outputs to caller-chosen destinations.

destinations_by_key is the complete requested output set: each key is restored exactly to its corresponding path. Destinations must be under the tracker's run directory or configured mount roots unless external paths are enabled on the tracker. The returned keyed result can describe a mixture of successful, preserved, and unavailable outputs.

Parameters:

Name	Type	Description	Default
`run_id`	`str`	Identifier of the historical run whose outputs should be restored.	required
`destinations_by_key`	`Mapping[str, str \| Path]`	Exact destination path for each requested output key.	required
`source_root`	`str \| Path \| None`	Optional alternate source root for filesystem-backed recovery.	`None`
`preserve_existing`	`bool`	If True, existing destinations are preserved and reported as reusable.	`True`
`on_missing`	`(warn, 'raise')`	Missing-source handling policy forwarded to the tracker.	`"warn"`
`db_fallback`	`(never, if_ingested)`	DB export fallback policy forwarded to the tracker.	`"never"`
`tracker`	`Optional[Tracker]`	Tracker instance to use. If omitted, resolves the active/default tracker.	`None`

Returns:

Type	Description
`HydratedRunOutputsResult`	Keyed hydration outcome for the requested outputs.

`materialize_run_outputs(run_id, *, target_root, source_root=None, keys=None, preserve_existing=True, on_missing='warn', db_fallback='if_ingested', tracker=None)` ¶

Materialize the linked outputs of a historical run into a new root.

This is the top-level convenience wrapper around Tracker.materialize_run_outputs(...). It restores historical bytes and returns the legacy aggregate MaterializationResult summary.

Prefer hydrate_run_outputs(...) when you want a result keyed by output name, detached hydrated artifacts, or directly usable destination paths.

Parameters:

Name	Type	Description	Default
`run_id`	`str`	Identifier of the run whose outputs should be restored.	required
`target_root`	`str \| Path`	Destination root for rematerialized outputs.	required
`source_root`	`str \| Path \| None`	Optional alternate source root for filesystem-backed recovery.	`None`
`keys`	`Sequence[str] \| None`	Optional subset of output keys to restore.	`None`
`preserve_existing`	`bool`	If True, existing destinations are preserved and reported as skipped.	`True`
`on_missing`	`(warn, 'raise')`	Missing-source handling policy forwarded to the tracker.	`"warn"`
`db_fallback`	`(never, if_ingested)`	DB export fallback policy forwarded to the tracker.	`"never"`
`tracker`	`Optional[Tracker]`	Tracker instance to use. If omitted, resolves the active/default tracker.	`None`

Returns:

Type	Description
`MaterializationResult`	Structured materialization outcome for the selected run outputs.

Examples:

>>> restored = consist.materialize_run_outputs(
...     "prior_run_id",
...     keys=["persons"],
...     target_root="restored",
... )
>>> restored.materialized
{'persons': '.../restored/.../persons.parquet'}

`materialize_artifact(artifact, *, target_root, source_root=None, preserve_existing=True, on_missing='warn', validate_content_hash='if-present', allow_external_paths=None, tracker=None)` ¶

Materialize one tracked artifact under a target root.

Use this helper when you already have an artifact object and need its bytes restored into a fresh workspace root. For known historical run outputs, prefer hydrate_run_outputs(...).

`stage_artifact(artifact, *, destination, overwrite=False, mode='copy', validate_content_hash='if-present', allow_external_paths=None, tracker=None)` ¶

Stage a single artifact to a requested filesystem destination.

This is the low-level convenience wrapper for canonical input staging. It copies the artifact bytes into the requested destination, preserving existing identical content when possible and returning a detached staged artifact view for immediate reuse.

Prefer ExecutionOptions(input_materialization="requested", input_paths={...}) on run(...) or ScenarioContext.run(...) when the goal is to prepare declared inputs for one execution. Use this helper when you need the same staging behavior outside a run lifecycle.

`stage_inputs(inputs_by_key, *, destinations_by_key, overwrite=False, mode='copy', validate_content_hash='if-present', allow_external_paths=None, tracker=None)` ¶

Stage multiple keyed artifacts to explicitly requested filesystem paths.

Keys must match the input artifact mapping keys. The returned result is keyed in the same order as destinations_by_key.

Prefer run-level requested input materialization for standard workflow code. Use this helper when you already have resolved artifacts and need to stage them before or outside normal run execution.

`set_artifact_recovery_roots(artifact, roots, *, append=False, tracker=None)` ¶

Persist ordered advisory recovery roots for an artifact.

These roots are probed after per-call overrides and historical run/mount locations when rematerializing historical outputs or hydrating cache-hit outputs. Use this when a workflow promotes bytes into a stable archive root but you want to keep container_uri as the canonical logical location.

Parameters:

Name	Type	Description	Default
`artifact`	`Artifact`	Artifact whose recovery metadata should be updated.	required
`roots`	`str \| Path \| Sequence[str \| Path]`	One or more filesystem roots. The artifact's URI-relative layout is preserved beneath each root.	required
`append`	`bool`	If True, append unique roots after existing ones. If False, replace the existing list. Passing an empty sequence clears `recovery_roots`.	`False`
`tracker`	`Tracker \| None`	Tracker used for persistence. If omitted, resolves the active/default tracker.	`None`

Returns:

Type	Description
`Artifact`	The supplied artifact with refreshed in-memory recovery metadata.

`register_artifact_recovery_copy(artifact, recovery_root, *, verify=True, content_hash=None, append=True, tracker=None)` ¶

Verify and register an externally copied artifact recovery location.

Parameters:

Name	Type	Description	Default
`artifact`	`Artifact`	Artifact represented by an existing recovery copy.	required
`recovery_root`	`str \| Path`	Root containing the artifact at its URI-relative path.	required
`verify`	`bool`	Whether to verify the copy's full-file SHA-256 digest.	`True`
`content_hash`	`str \| None`	Expected SHA-256 digest, taking precedence over `artifact.hash`.	`None`
`append`	`bool`	Whether to append rather than replace recovery-root metadata.	`True`
`tracker`	`Tracker \| None`	Tracker used for validation and persistence.	`None`

Returns:

Type	Description
`ArtifactRecoveryCopyRegistration`	Per-artifact verification and metadata-persistence outcome.

Notes

This function never copies bytes. Directory artifacts and HDF5 child tables are deliberately blocked until they have an independent recovery contract.

`register_run_output_recovery_copies(run_id, recovery_root, *, keys=None, verify=True, append=True, content_hashes=None, tracker=None)` ¶

Verify and register existing recovery copies for run outputs.

Parameters:

Name	Type	Description	Default
`run_id`	`str`	Completed run whose outputs are being registered.	required
`recovery_root`	`str \| Path`	Root containing URI-relative recovery copies.	required
`keys`	`sequence of str \| None`	Outputs to register; `None` selects all outputs.	`None`
`verify`	`bool`	Whether selected copies require a matching full-file hash.	`True`
`append`	`bool`	Whether successful registrations append recovery-root metadata.	`True`
`content_hashes`	`mapping of str to str \| None`	Optional per-key SHA-256 proofs for artifacts without full hashes.	`None`
`tracker`	`Tracker \| None`	Tracker used for validation and persistence.	`None`

Returns:

Type	Description
`RunOutputRecoveryCopiesRegistration`	Mapping-style result with one registration outcome per selected key.

Raises:

Type	Description
`KeyError`	If requested output or content-hash keys are unknown.

`archive_artifact(artifact, archive_root, *, mode='copy', append=True, tracker=None)` ¶

Archive an artifact into a stable recovery root and register it.

Parameters:

Name	Type	Description	Default
`artifact`	`Artifact`	Artifact whose bytes will be copied or moved.	required
`archive_root`	`str \| Path`	Root below which its URI-relative layout is recreated.	required
`mode`	`(copy, move)`	Filesystem operation used when the destination is absent.	`"copy"`
`append`	`bool`	Whether to append `archive_root` to recovery-root metadata.	`True`
`tracker`	`Tracker \| None`	Tracker that owns the artifact and metadata database.	`None`

Returns:

Type	Description
`Path`	Archive destination containing the artifact bytes.

`archive_run_outputs(run_id, archive_root, *, keys=None, mode='copy', append=True, tracker=None)` ¶

Archive selected historical run outputs into a recovery root.

Parameters:

Name	Type	Description	Default
`run_id`	`str`	Completed run whose outputs should be archived.	required
`archive_root`	`str \| Path`	Root below which URI-relative output paths are recreated.	required
`keys`	`sequence of str \| None`	Output keys to archive; `None` selects all outputs.	`None`
`mode`	`(copy, move)`	Filesystem operation used for each output.	`"copy"`
`append`	`bool`	Whether to append the root to recovery-root metadata.	`True`
`tracker`	`Tracker \| None`	Tracker that owns the historical run.	`None`

Returns:

Type	Description
`ArchivedOutputs`	Read-only key-to-path mapping plus refreshed output artifacts.

Notes

Manifest-backed OutputSets are archived as one validated logical recovery unit. Use :func:archive_run_output_files for no-overwrite, per-key report semantics for regular files.

`archive_current_run_outputs(archive_root, *, keys=None, mode='copy', append=True, tracker=None)` ¶

Archive outputs for the current active run using legacy semantics.

Parameters:

Name	Type	Description	Default
`archive_root`	`str \| Path`	Root below which URI-relative output paths are recreated.	required
`keys`	`sequence of str \| None`	Output keys to archive; `None` selects all active-run outputs.	`None`
`mode`	`(copy, move)`	Filesystem operation used for each output.	`"copy"`
`append`	`bool`	Whether to append the root to recovery-root metadata.	`True`
`tracker`	`Tracker \| None`	Tracker that owns the active run context.	`None`

Returns:

Type	Description
`ArchivedOutputs`	Read-only key-to-path mapping plus refreshed output artifacts.

Raises:

Type	Description
`RuntimeError`	If no run is active in the supplied tracker context.

`archive_run_output_files(run_id, recovery_root, *, keys=None, preserve_existing=True, verify=True, append=True, tracker=None)` ¶

Archive regular run-output files with per-key copy and metadata status.

Parameters:

Name	Type	Description	Default
`run_id`	`str`	Completed run whose regular output files should be archived.	required
`recovery_root`	`str \| Path`	Root below which canonical URI-relative paths are created.	required
`keys`	`sequence of str \| None`	Output keys to archive; `None` selects all outputs.	`None`
`preserve_existing`	`bool`	Whether a matching existing target may be retained.	`True`
`verify`	`bool`	Whether copied or retained files require full-hash verification.	`True`
`append`	`bool`	Whether successful registrations append recovery-root metadata.	`True`
`tracker`	`Tracker \| None`	Tracker that owns the historical run.	`None`

Returns:

Type	Description
`ArchivedRunOutputFilesReport`	Mapping-style per-key copy, verification, and metadata results.

Raises:

Type	Description
`KeyError`	If a requested output key is unknown, before target mutation.

Notes

This file-only helper never overwrites a target. complete is an invocation result, not a durable archive-workflow state claim.

`to_df(relation, *, close=True)` ¶

Convert a DuckDB Relation to a pandas DataFrame.

Parameters:

Name	Type	Description	Default
`relation`	`DuckDBPyRelation`	Relation to materialize into a DataFrame.	required
`close`	`bool`	Whether to close the underlying DuckDB connection after materialization. Use `close=False` if you plan to continue using the relation.	`True`

`active_relation_count()` ¶

Return the number of active DuckDB relations tracked by Consist.

`set_current_tracker(tracker)` ¶

Set the default (fallback) tracker used by Consist entrypoints.

Entrypoints like consist.run(), consist.start_run(), and consist.scenario() use this tracker if they are called outside of an active run context and no explicit tracker= argument is provided.

Parameters:

Name	Type	Description	Default
`tracker`	`Optional[Tracker]`	The tracker instance to set as the default, or `None` to clear.	required

Returns:

Type	Description
`Optional[Tracker]`	The previously configured default tracker, if any.

`noop_scenario(name, **kwargs)` ¶

Creates a scenario context that executes without provenance tracking.

This is useful for debugging or running simulations where you want the ergonomics of the Consist scenario API (like the Coupler and RunResult) but do not want to record any metadata or artifacts to the database.

Parameters:

Name	Type	Description	Default
`name`	`str`	Name of the scenario (for display/logging purposes).	required
`**kwargs`	`Any`	Additional arguments forwarded to the noop context.	`{}`

`is_dataframe_artifact(artifact)` ¶

Type guard: narrow artifact to tabular types (parquet, csv, gtfs, h5_table).

Use this to enable type-safe loading and IDE autocomplete:

if is_dataframe_artifact(artifact):
    rel = load(artifact)  # Type checker knows return is Relation
    df.head()  # IDE autocomplete works!

Parameters:

Name	Type	Description	Default
`artifact`	`ArtifactLike`	Artifact to check.	required

Returns:

Type	Description
`bool`	True if artifact driver is parquet, csv, gtfs, or h5_table.

`is_tabular_artifact(artifact)` ¶

Type guard: narrow artifact to any tabular format (parquet, csv, h5_table, json).

Note: This is broader than is_dataframe_artifact(), so load() returns a Relation for tabular artifacts. Use load_df() for a pandas escape hatch.

Parameters:

Name	Type	Description	Default
`artifact`	`ArtifactLike`	Artifact to check.	required

Returns:

Type	Description
`bool`	True if artifact driver produces tabular data.

`is_json_artifact(artifact)` ¶

Type guard: narrow artifact to JSON format.

Parameters:

Name	Type	Description	Default
`artifact`	`ArtifactLike`	Artifact to check.	required

Returns:

Type	Description
`bool`	True if artifact driver is json.

`is_zarr_artifact(artifact)` ¶

Type guard: narrow artifact to Zarr format.

Use this when you know an artifact should be Zarr and want type-safe loading:

if is_zarr_artifact(artifact):
    ds = load(artifact)  # Type checker knows return is xarray.Dataset
    ds.dims  # IDE autocomplete works!

Parameters:

Name	Type	Description	Default
`artifact`	`ArtifactLike`	Artifact to check.	required

Returns:

Type	Description
`bool`	True if artifact driver is zarr.

`is_hdf_artifact(artifact)` ¶

Type guard: narrow artifact to HDF5 format (h5 or hdf5).

Parameters:

Name	Type	Description	Default
`artifact`	`ArtifactLike`	Artifact to check.	required

Returns:

Type	Description
`bool`	True if artifact driver is h5 or hdf5.

`is_spatial_artifact(artifact)` ¶

Type guard: narrow artifact to spatial formats (GeoDataFrame outputs).

Parameters:

Name	Type	Description	Default
`artifact`	`ArtifactLike`	Artifact to check.	required

Returns:

Type	Description
`bool`	True if artifact driver is geojson, geoparquet, shapefile, or geopackage.

API Helpers¶

When to use helpers vs class methods¶

High-traffic helpers¶

Minimal runnable helper workflow¶

RelationConnectionLeakWarning ¶

view(model, name=None) ¶

use_tracker(tracker) ¶

run(fn=None, name=None, *, tracker=None, adapter=None, identity_inputs=None, cache_options=None, output_policy=None, execution_options=None, **kwargs) ¶

ref(run_result, key=None) ¶

refs(*selectors, **aliases) ¶

trace(name, *, tracker=None, adapter=None, identity_inputs=None, **kwargs) ¶

start_run(run_id, model, tracker=None, **kwargs) ¶

require_runtime_kwargs(*names) ¶

scenario(name, tracker=None, *, enabled=True, step_tags=None, step_facet=None, **kwargs) ¶

single_step_scenario(name, step_name=None, tracker=None, **kwargs) ¶

current_tracker() ¶

current_run() ¶

current_consist() ¶

output_dir(namespace=None) ¶

output_path(key, ext='parquet') ¶

cached_artifacts(direction='output') ¶

cached_output(key=None) ¶

get_run_result(run_id, *, keys=None, validate='lazy', tracker=None) ¶

get_artifact(run_id, key=None, key_contains=None, direction='output') ¶

register_artifact_facet_parser(prefix, parser_fn, *, tracker=None) ¶

log_artifacts(outputs, *, direction='output', driver=None, metadata_by_key=None, facets_by_key=None, facet_schema_versions_by_key=None, facet_index=False, reuse_if_unchanged=False, reuse_scope='same_uri', enabled=True, **shared_meta) ¶

log_input(path, key=None, *, schema=None, driver=None, content_hash=None, force_hash_override=False, validate_content_hash=False, facet=None, facet_schema_version=None, facet_index=False, enabled=True, **meta) ¶

log_output(path, key=None, *, artifact_kind='file', schema=None, driver=None, content_hash=None, force_hash_override=False, validate_content_hash=False, reuse_if_unchanged=False, reuse_scope='same_uri', facet=None, facet_schema_version=None, facet_index=False, enabled=True, **meta) ¶

log_dataframe(df, key, schema=None, direction='output', tracker=None, path=None, driver=None, meta=None, parent_artifact_id=None, **to_file_kwargs) ¶

log_meta(**kwargs) ¶

ingest(artifact, data=None, schema=None, run=None) ¶

canonicalize_gtfs(feed_paths, *, service_date=None, feed_keys=None, run=None, run_id=None, key=GTFS_BUNDLE_SOURCE_KEY, ingest=True, profile_schema=False, log_sources=True, tracker=None) ¶

register_views(*models) ¶

find_run(tracker=None, **filters) ¶

find_runs(tracker=None, **filters) ¶

find_latest_run(tracker=None, **filters) ¶

find_matching_run(tracker=None, **filters) ¶

find_matching_runs(tracker=None, **filters) ¶

run_set(tracker=None, label=None, **filters) ¶

db_session(tracker=None) ¶

run_query(query, tracker=None) ¶

config_run_query(table, *, link_table, table_name=None, columns=None, where=None, join_on=None) ¶

config_run_rows(table, *, link_table, table_name=None, columns=None, where=None, join_on=None, tracker=None) ¶

pivot_facets(*, namespace, keys, value_column='value_num', value_columns=None, label_prefix='', label_map=None, run_id_label='run_id', table=RunConfigKV) ¶

capture_outputs(directory, pattern='*', recursive=False) ¶

load(artifact, tracker=None, *, db_fallback='inputs-only', **kwargs) ¶

load_df(artifact, tracker=None, *, db_fallback='inputs-only', close=True, **kwargs) ¶

load_relation(artifact, tracker=None, *, db_fallback='inputs-only', **kwargs) ¶

ibis_connection(tracker=None) ¶

ibis_view(tracker=None, *, model, key=None) ¶

hydrate_run_outputs(run_id, *, target_root, source_root=None, keys=None, preserve_existing=True, on_missing='warn', db_fallback='if_ingested', tracker=None) ¶

hydrate_run_outputs_to_destinations(run_id, *, destinations_by_key, source_root=None, preserve_existing=True, on_missing='warn', db_fallback='if_ingested', tracker=None) ¶

materialize_run_outputs(run_id, *, target_root, source_root=None, keys=None, preserve_existing=True, on_missing='warn', db_fallback='if_ingested', tracker=None) ¶

materialize_artifact(artifact, *, target_root, source_root=None, preserve_existing=True, on_missing='warn', validate_content_hash='if-present', allow_external_paths=None, tracker=None) ¶

stage_artifact(artifact, *, destination, overwrite=False, mode='copy', validate_content_hash='if-present', allow_external_paths=None, tracker=None) ¶

stage_inputs(inputs_by_key, *, destinations_by_key, overwrite=False, mode='copy', validate_content_hash='if-present', allow_external_paths=None, tracker=None) ¶

set_artifact_recovery_roots(artifact, roots, *, append=False, tracker=None) ¶

register_artifact_recovery_copy(artifact, recovery_root, *, verify=True, content_hash=None, append=True, tracker=None) ¶

register_run_output_recovery_copies(run_id, recovery_root, *, keys=None, verify=True, append=True, content_hashes=None, tracker=None) ¶

archive_artifact(artifact, archive_root, *, mode='copy', append=True, tracker=None) ¶

archive_run_outputs(run_id, archive_root, *, keys=None, mode='copy', append=True, tracker=None) ¶

archive_current_run_outputs(archive_root, *, keys=None, mode='copy', append=True, tracker=None) ¶

archive_run_output_files(run_id, recovery_root, *, keys=None, preserve_existing=True, verify=True, append=True, tracker=None) ¶

to_df(relation, *, close=True) ¶

active_relation_count() ¶

set_current_tracker(tracker) ¶

noop_scenario(name, **kwargs) ¶

is_dataframe_artifact(artifact) ¶

is_tabular_artifact(artifact) ¶

is_json_artifact(artifact) ¶

is_zarr_artifact(artifact) ¶

is_hdf_artifact(artifact) ¶

is_spatial_artifact(artifact) ¶

`RelationConnectionLeakWarning` ¶

`view(model, name=None)` ¶

`use_tracker(tracker)` ¶

`run(fn=None, name=None, *, tracker=None, adapter=None, identity_inputs=None, cache_options=None, output_policy=None, execution_options=None, **kwargs)` ¶

`ref(run_result, key=None)` ¶

`refs(*selectors, **aliases)` ¶

`trace(name, *, tracker=None, adapter=None, identity_inputs=None, **kwargs)` ¶

`start_run(run_id, model, tracker=None, **kwargs)` ¶

`require_runtime_kwargs(*names)` ¶

`scenario(name, tracker=None, *, enabled=True, step_tags=None, step_facet=None, **kwargs)` ¶

`single_step_scenario(name, step_name=None, tracker=None, **kwargs)` ¶

`current_tracker()` ¶

`current_run()` ¶

`current_consist()` ¶

`output_dir(namespace=None)` ¶

`output_path(key, ext='parquet')` ¶

`cached_artifacts(direction='output')` ¶

`cached_output(key=None)` ¶

`get_run_result(run_id, *, keys=None, validate='lazy', tracker=None)` ¶

`get_artifact(run_id, key=None, key_contains=None, direction='output')` ¶

`register_artifact_facet_parser(prefix, parser_fn, *, tracker=None)` ¶

`log_artifacts(outputs, *, direction='output', driver=None, metadata_by_key=None, facets_by_key=None, facet_schema_versions_by_key=None, facet_index=False, reuse_if_unchanged=False, reuse_scope='same_uri', enabled=True, **shared_meta)` ¶

`log_input(path, key=None, *, schema=None, driver=None, content_hash=None, force_hash_override=False, validate_content_hash=False, facet=None, facet_schema_version=None, facet_index=False, enabled=True, **meta)` ¶

`log_output(path, key=None, *, artifact_kind='file', schema=None, driver=None, content_hash=None, force_hash_override=False, validate_content_hash=False, reuse_if_unchanged=False, reuse_scope='same_uri', facet=None, facet_schema_version=None, facet_index=False, enabled=True, **meta)` ¶

`log_dataframe(df, key, schema=None, direction='output', tracker=None, path=None, driver=None, meta=None, parent_artifact_id=None, **to_file_kwargs)` ¶

`log_meta(**kwargs)` ¶

`ingest(artifact, data=None, schema=None, run=None)` ¶

`canonicalize_gtfs(feed_paths, *, service_date=None, feed_keys=None, run=None, run_id=None, key=GTFS_BUNDLE_SOURCE_KEY, ingest=True, profile_schema=False, log_sources=True, tracker=None)` ¶

`register_views(*models)` ¶

`find_run(tracker=None, **filters)` ¶

`find_runs(tracker=None, **filters)` ¶

`find_latest_run(tracker=None, **filters)` ¶

`find_matching_run(tracker=None, **filters)` ¶

`find_matching_runs(tracker=None, **filters)` ¶

`run_set(tracker=None, label=None, **filters)` ¶

`db_session(tracker=None)` ¶

`run_query(query, tracker=None)` ¶

`config_run_query(table, *, link_table, table_name=None, columns=None, where=None, join_on=None)` ¶

`config_run_rows(table, *, link_table, table_name=None, columns=None, where=None, join_on=None, tracker=None)` ¶

`pivot_facets(*, namespace, keys, value_column='value_num', value_columns=None, label_prefix='', label_map=None, run_id_label='run_id', table=RunConfigKV)` ¶

`capture_outputs(directory, pattern='*', recursive=False)` ¶

`load(artifact, tracker=None, *, db_fallback='inputs-only', **kwargs)` ¶

`load_df(artifact, tracker=None, *, db_fallback='inputs-only', close=True, **kwargs)` ¶

`load_relation(artifact, tracker=None, *, db_fallback='inputs-only', **kwargs)` ¶

`ibis_connection(tracker=None)` ¶

`ibis_view(tracker=None, *, model, key=None)` ¶

`hydrate_run_outputs(run_id, *, target_root, source_root=None, keys=None, preserve_existing=True, on_missing='warn', db_fallback='if_ingested', tracker=None)` ¶

`hydrate_run_outputs_to_destinations(run_id, *, destinations_by_key, source_root=None, preserve_existing=True, on_missing='warn', db_fallback='if_ingested', tracker=None)` ¶

`materialize_run_outputs(run_id, *, target_root, source_root=None, keys=None, preserve_existing=True, on_missing='warn', db_fallback='if_ingested', tracker=None)` ¶

`materialize_artifact(artifact, *, target_root, source_root=None, preserve_existing=True, on_missing='warn', validate_content_hash='if-present', allow_external_paths=None, tracker=None)` ¶

`stage_artifact(artifact, *, destination, overwrite=False, mode='copy', validate_content_hash='if-present', allow_external_paths=None, tracker=None)` ¶

`stage_inputs(inputs_by_key, *, destinations_by_key, overwrite=False, mode='copy', validate_content_hash='if-present', allow_external_paths=None, tracker=None)` ¶

`set_artifact_recovery_roots(artifact, roots, *, append=False, tracker=None)` ¶

`register_artifact_recovery_copy(artifact, recovery_root, *, verify=True, content_hash=None, append=True, tracker=None)` ¶

`register_run_output_recovery_copies(run_id, recovery_root, *, keys=None, verify=True, append=True, content_hashes=None, tracker=None)` ¶

`archive_artifact(artifact, archive_root, *, mode='copy', append=True, tracker=None)` ¶

`archive_run_outputs(run_id, archive_root, *, keys=None, mode='copy', append=True, tracker=None)` ¶

`archive_current_run_outputs(archive_root, *, keys=None, mode='copy', append=True, tracker=None)` ¶

`archive_run_output_files(run_id, recovery_root, *, keys=None, preserve_existing=True, verify=True, append=True, tracker=None)` ¶

`to_df(relation, *, close=True)` ¶

`active_relation_count()` ¶

`set_current_tracker(tracker)` ¶

`noop_scenario(name, **kwargs)` ¶

`is_dataframe_artifact(artifact)` ¶

`is_tabular_artifact(artifact)` ¶

`is_json_artifact(artifact)` ¶

`is_zarr_artifact(artifact)` ¶

`is_hdf_artifact(artifact)` ¶

`is_spatial_artifact(artifact)` ¶