1 data_handling._offline_format
data_handling._offline_format
Typed manifest and index records for the VIN offline dataset format.
The new offline dataset format is an immutable indexed-shard layout optimized for multi-worker random access. This module defines the normalized metadata records shared by the writer and runtime dataset reader:
- the top-level dataset manifest,
- per-shard block descriptors, and
- sample-index records used for global random access and split membership.
1.1 Classes
| Name | Description |
|---|---|
| VinOfflineBlockSpec | Descriptor for one stored block inside a shard. |
| VinOfflineShardSpec | Descriptor for one immutable dataset shard. |
| VinOfflineMaterializedBlocks | Materialized block flags for a VIN offline dataset. |
| VinOfflineManifest | Top-level manifest for one immutable VIN offline dataset. |
| VinOfflineIndexRecord | Global sample-index entry for VIN offline random access. |