Skip to content

Artifact

Artifact represents a tracked input or output file with stable provenance metadata (run_id, hash, container_uri, driver, and custom metadata).

Container-backed child artifacts, such as HDF5 tables discovered from a parent file, are still ordinary Artifact rows. Consist records their canonical parent-child relation on artifact.parent_artifact_id. Legacy metadata-based links in artifact.meta["parent_id"] remain for compatibility and debugging, but new code should treat parent_artifact_id as the source of truth.

When to use Artifact directly

  • You want to pass outputs from one step to another (inputs={...} mappings).
  • You need a portable path via artifact.path rather than hard-coded file paths.
  • You need metadata checks (artifact.get_meta(...), artifact.is_tabular, artifact.is_matrix) before loading.
  • You need to inspect or traverse container child-artifact relations after logging, hydration, or query retrieval.

Container child artifacts

Use Tracker.get_child_artifacts(...) and Tracker.get_parent_artifact(...) to navigate first-class parent-child artifact relations. This is the supported query surface for HDF5 container artifacts and other container-child patterns.

The parent-child relation describes lineage and container membership; it does not by itself make a child artifact independently recoverable. For HDF5 containers, recovery authority comes from the parent container policy recorded by log_h5_container(...). The default restart-safe pattern is parent-file recovery with descriptive child h5_table artifacts.

Minimal runnable example

from pathlib import Path
import consist
from consist import Tracker

tracker = Tracker(run_dir="./runs", db_path="./provenance.duckdb")

def write_output() -> Path:
    out = consist.output_path("report", ext="txt")
    out.write_text("hello\n")
    return out

with consist.use_tracker(tracker):
    result = consist.run(fn=write_output, outputs=["report"])

artifact = result.outputs["report"]
print(artifact.key)
print(artifact.path)
print(artifact.path.read_text().strip())

See API Helpers for helper functions that return and consume artifacts (consist.run, consist.ref, consist.refs, consist.load*). For query helpers that traverse child artifacts, see Tracker.

Archived Outputs

tracker.archive_run_outputs(...) and tracker.archive_current_run_outputs(...) return ArchivedOutputs, a dict-like mapping of archived paths. Use .paths when you want the explicit read-only archived-path mapping, and use .outputs when you want the refreshed artifacts with the new recovery root already attached for downstream inputs=... reuse.

Bases: SQLModel

Represents a physical data object in the Consist database.

This table stores canonical metadata for any file/dataset Consist tracks. It is linked to runs via run_artifact_link to record whether an artifact was an input or output. The run_id field records the producing run (if any) and is often None for external inputs.

Artifacts are the core building blocks of provenance and caching. Each artifact has a unique identity, a virtualized location, and rich metadata, supporting both "hot" (ingested) and "cold" (file-based) data strategies.

The public fingerprint surface for downstream consumers is Artifact.hash. Consist preserves that field across logging, querying, replay, staging, and hydration so external provenance/bookkeeping code can read one stable fingerprint without probing wrapper-specific fields. Fingerprint semantics follow the configured hashing strategy: full is generally content-based, while fast may use metadata-based hashing for files or directories. Artifact.content_id is a database-local deduplication key and is not the public fingerprint contract.

Attributes: id (uuid.UUID): A unique identifier for the artifact. key (str): A semantic, human-readable name for the artifact (e.g., "households", "parcels"). container_uri (str): A portable, virtualized Uniform Resource Identifier (URI) for the artifact's location (e.g., "inputs://land_use.csv"). table_path (Optional[str]): Optional path inside a container (e.g., "/tables/households"). array_path (Optional[str]): Optional path inside a container for array artifacts. driver (str): The name of the format handler used to read or write the artifact (e.g., "parquet", "csv", "zarr"). hash (Optional[str]): Canonical portable artifact fingerprint. This is usually a SHA256 digest and remains the stable public identity field across logging, replay, staging, and hydration. parent_artifact_id (Optional[uuid.UUID]): Canonical parent artifact relation for container child artifacts. Null for standalone artifacts. run_id (Optional[str]): The ID of the run that generated this artifact. Null for inputs. meta (dict[str, object]): A flexible JSON field for storing arbitrary metadata, such as schema signatures, or data dimensions. created_at (datetime): The timestamp when the artifact was first logged.

abs_path property writable

Runtime-only helper to access the absolute path of this artifact.

This property provides the resolved absolute file system path for the artifact. It is not persisted to the database but is crucial for local file operations and for chaining Consist runs within the same script or environment.

Returns:

Type Description
Optional[str]

The absolute file system path of the artifact, or None if it has not yet been resolved or set.

path property

Resolve this artifact to a filesystem Path.

Uses the tracker when available to handle mount-aware URIs. If that canonical resolution does not exist but a runtime abs_path is set (for example after historical hydration or archive move), this falls back to abs_path before returning the raw URI.

is_matrix property

Indicates if the artifact represents a multi-dimensional array or matrix-like data.

This property helps in dispatching to appropriate data loaders or processing functions that handle array-based data structures, such as those typically found in scientific computing.

Returns:

Type Description
bool

True if the artifact's driver is associated with matrix-like data formats (e.g., Zarr, HDF5, NetCDF, OpenMatrix), False otherwise.

is_tabular property

Indicates if the artifact represents tabular data (rows and columns).

This property assists in identifying artifacts that can be loaded and processed using tools designed for structured, record-based data, such as Pandas DataFrames.

Returns:

Type Description
bool

True if the artifact's driver is associated with tabular data formats (e.g., Parquet, CSV, SQL), False otherwise.

created_at_iso property

Return created_at as an ISO 8601 formatted string.

Useful for serialization to external systems that expect string timestamps.

Returns:

Type Description
Optional[str]

The created_at timestamp as ISO 8601 string, or None if not set.

get_meta(key, default=None)

Safely retrieves a value from the 'meta' dictionary.

Args: key (str): The key to look up in the metadata. default (Any, optional): The default value to return if the key is not found. Defaults to None.

Returns: Any: The value associated with the key, or the default value if the key is not present.