1 rollouts.storage.packing

rollouts.storage.packing

Lossless physical merge/split of complete rollout chains.

The factual tables remain authoritative: this transformation only remaps store local keys and dictionaries, then rebuilds the existing derived QH projection. Every output repeats the small common source-reference table so the reader’s strict cross-store split-manifest identity remains consistent after splitting. Original roots, acquisition receipts and scientific labels are unchanged.

1.1 Classes

Name Description
RootSourceBinding Exact old/new actor row identities for a coordinated root transformation.

1.1.1 RootSourceBinding

rollouts.storage.packing.RootSourceBinding(
    source_record,
    source_version,
    destination_record,
    destination_manifest_hash,
    destination_version,
)

Exact old/new actor row identities for a coordinated root transformation.

1.1.1.1 Attributes

Name Description
source_record Original actor row, including immutable key, split and physical coordinates.
source_version Original actor schema version required by the persisted rollout reference.
destination_record Rewritten actor row; only local index and physical coordinates may differ.
destination_manifest_hash Fresh content-bound actor manifest identity used by the new QH join.
destination_version Destination actor capability version, unchanged by physical repacking.

1.2 Functions

Name Description
repack_rollout_stores Merge and split whole chains into fresh, atomically published stores.

1.2.1 repack_rollout_stores

rollouts.storage.packing.repack_rollout_stores(
    sources,
    destination,
    *,
    chains_per_store,
    source_bindings=None,
)

Merge and split whole chains into fresh, atomically published stores.

Sources must share one unchanged original root dependency, replay semantics and retained schema. Distinct chains and targets remain distinct even when their human-readable names agree. Exact source-reference aliases are shared; conflicting aliases and unknown retained fields reject before publication. Numeric values, dtypes, masks, labels, selected-depth/crop rows and external acquisition IDs are conserved. Physical row IDs and successor references are mapped in each authenticated transformation receipt. Existing destinations reject; a failed/interrupted write leaves no published partial destination.

The destination is a new directory containing store-NNNNNN.zarr members. All payloads for one output are currently materialized in memory; choose a bounded chain count. source_bindings admits a coordinated transformation: every original source field is checked against the old actor record before new indices/shards/manifest hashes are installed. Output stores are grouped by new root, physical split and campaign assignment, so filtered QH datasets reproduce their exact ordered source hash without mixing the two split roles.

Back to top