1 rollouts.storage.packing
rollouts.storage.packing
Lossless physical merge/split of complete rollout chains.
The factual tables remain authoritative: this transformation only remaps store local keys and dictionaries, then rebuilds the existing derived QH projection. Every output repeats the small common source-reference table so the reader’s strict cross-store split-manifest identity remains consistent after splitting. Original roots, acquisition receipts and scientific labels are unchanged.
1.1 Classes
| Name | Description |
|---|---|
| RootSourceBinding | Exact old/new actor row identities for a coordinated root transformation. |
1.1.1 RootSourceBinding
rollouts.storage.packing.RootSourceBinding(
source_record,
source_version,
destination_record,
destination_manifest_hash,
destination_version,
)Exact old/new actor row identities for a coordinated root transformation.
1.1.1.1 Attributes
| Name | Description |
|---|---|
| source_record | Original actor row, including immutable key, split and physical coordinates. |
| source_version | Original actor schema version required by the persisted rollout reference. |
| destination_record | Rewritten actor row; only local index and physical coordinates may differ. |
| destination_manifest_hash | Fresh content-bound actor manifest identity used by the new QH join. |
| destination_version | Destination actor capability version, unchanged by physical repacking. |
1.2 Functions
| Name | Description |
|---|---|
| repack_rollout_stores | Merge and split whole chains into fresh, atomically published stores. |
1.2.1 repack_rollout_stores
rollouts.storage.packing.repack_rollout_stores(
sources,
destination,
*,
chains_per_store,
source_bindings=None,
)Merge and split whole chains into fresh, atomically published stores.
Sources must share one unchanged original root dependency, replay semantics and retained schema. Distinct chains and targets remain distinct even when their human-readable names agree. Exact source-reference aliases are shared; conflicting aliases and unknown retained fields reject before publication. Numeric values, dtypes, masks, labels, selected-depth/crop rows and external acquisition IDs are conserved. Physical row IDs and successor references are mapped in each authenticated transformation receipt. Existing destinations reject; a failed/interrupted write leaves no published partial destination.
The destination is a new directory containing store-NNNNNN.zarr members. All payloads for one output are currently materialized in memory; choose a bounded chain count. source_bindings admits a coordinated transformation: every original source field is checked against the old actor record before new indices/shards/manifest hashes are installed. Output stores are grouped by new root, physical split and campaign assignment, so filtered QH datasets reproduce their exact ordered source hash without mixing the two split roles.