1 data_handling.vin_store.diagnostics

data_handling.vin_store.diagnostics

Diagnostics for immutable VIN offline stores.

The helpers in this module inspect the current VinOfflineDataset storage format directly for store, sample, RRI, geometry, backbone, memory, batch, and raw-dataset coverage diagnostics.

1.1 Classes

Name Description
NumericSummary Finite-value summary for one numeric diagnostic series.
VinOfflineBackboneDiagnostic Streaming numeric summary for one stored backbone field.
VinOfflineBlockDiagnostic Render-ready manifest summary for one stored offline block.
VinOfflineCoverageSceneDiagnostic Per-scene raw-dataset coverage against one immutable offline store.
VinOfflineCoverageStats Raw-dataset coverage summary for one immutable VIN offline store.
VinOfflineDatasetStats Store-level diagnostics for an immutable VIN offline dataset.
VinOfflineDatasetStatsRequest Complete generation-bound request for one root diagnostics acquisition.
VinOfflineMemoryDiagnostic Estimated per-sample runtime memory for one offline-store component.
VinOfflineSampleDiagnostic Per-row sanity summary for one sampled VIN offline record.

1.1.1 NumericSummary

data_handling.vin_store.diagnostics.NumericSummary(
    count,
    minimum,
    mean,
    maximum,
)

Finite-value summary for one numeric diagnostic series.

1.1.1.1 Attributes

Name Description
count Number of observed finite values.
minimum Minimum finite value, or None when no finite values exist.
mean Mean finite value, or None when no finite values exist.
maximum Maximum finite value, or None when no finite values exist.

1.1.2 VinOfflineBackboneDiagnostic

data_handling.vin_store.diagnostics.VinOfflineBackboneDiagnostic(
    field,
    shape,
    numel,
    sampled_rows,
    count,
    mean,
    std,
    abs_mean,
    nz_frac,
)

Streaming numeric summary for one stored backbone field.

1.1.2.1 Attributes

Name Description
field Backbone field name without the "backbone." prefix.
shape Per-row tensor shape.
numel Number of elements per sampled row.
sampled_rows Number of rows contributing to the summary.
count Number of finite elements contributing to summary moments.
mean Mean finite value.
std Population standard deviation for finite values.
abs_mean Mean absolute finite value.
nz_frac Fraction of finite values whose absolute value is larger than 1e-6.

1.1.3 VinOfflineBlockDiagnostic

data_handling.vin_store.diagnostics.VinOfflineBlockDiagnostic(
    shard_id,
    name,
    kind,
    dtype,
    shape,
    optional,
    estimated_bytes,
)

Render-ready manifest summary for one stored offline block.

1.1.3.1 Attributes

Name Description
shard_id Shard that declares the block.
name Logical block name.
kind Storage kind such as zarr_array or msgpack_indexed_records.
dtype Stored NumPy dtype for numeric blocks.
shape Stored array shape or record count.
optional Whether the block is optional.
estimated_bytes Estimated numeric bytes, or None for non-numeric blocks.

1.1.4 VinOfflineCoverageSceneDiagnostic

data_handling.vin_store.diagnostics.VinOfflineCoverageSceneDiagnostic(
    scene_id,
    dataset_snippets,
    store_snippets,
    covered_snippets,
    missing_in_store,
    outside_dataset,
    coverage,
)

Per-scene raw-dataset coverage against one immutable offline store.

1.1.4.1 Attributes

Name Description
scene_id ASE scene identifier.
dataset_snippets Number of raw-dataset snippets found in scanned tar headers.
store_snippets Number of immutable-store snippets for the scene.
covered_snippets Number of snippets present in both the raw dataset and the store.
missing_in_store Raw-dataset snippets missing from the immutable store.
outside_dataset Store snippets not found in the scanned raw-dataset tar headers.
coverage Coverage ratio covered_snippets / dataset_snippets.

1.1.5 VinOfflineCoverageStats

data_handling.vin_store.diagnostics.VinOfflineCoverageStats(
    store_dir,
    tar_shards_scanned,
    dataset_scenes,
    store_scenes,
    dataset_snippets,
    store_snippets,
    covered_snippets,
    missing_in_store,
    outside_dataset,
    coverage,
    per_scene=list(),
    missing_examples=list(),
    outside_examples=list(),
)

Raw-dataset coverage summary for one immutable VIN offline store.

1.1.5.1 Attributes

Name Description
store_dir Absolute store directory inspected for coverage.
tar_shards_scanned Number of raw dataset tar shards scanned.
dataset_scenes Number of scenes found in scanned raw-dataset tar headers.
store_scenes Number of scenes represented by the immutable store sample index.
dataset_snippets Number of raw-dataset snippets found in scanned tar headers.
store_snippets Number of immutable-store snippets.
covered_snippets Number of snippets present in both the raw dataset and immutable store.
missing_in_store Raw-dataset snippets missing from the immutable store.
outside_dataset Immutable-store snippets outside the scanned raw-dataset tar headers.
coverage Overall coverage ratio covered_snippets / dataset_snippets.
per_scene Per-scene coverage rows.
missing_examples Example (scene_id, snippet_id) pairs missing from the store.
outside_examples Example store pairs outside the scanned raw dataset.

1.1.6 VinOfflineDatasetStats

data_handling.vin_store.diagnostics.VinOfflineDatasetStats(
    store_dir,
    version,
    num_samples,
    sampled_samples,
    split_counts,
    num_scenes,
    num_snippets,
    materialized_blocks,
    candidate_count,
    rri,
    vin_points,
    numeric_bytes,
    block_shapes=dict(),
    block_diagnostics=list(),
    sample_summaries=list(),
    candidate_count_values=list(),
    rri_values=list(),
    vin_point_values=list(),
    rri_component_values=dict(),
    rri_component_summaries=dict(),
    candidate_pose_values=dict(),
    candidate_pose_summaries=dict(),
    memory_diagnostics=list(),
    backbone_diagnostics=list(),
    scene_labels_available=True,
    batch_shapes_available=True,
    batch_shapes=dict(),
    rri_component_pairs=dict(),
    rri_pairing_unavailable_sample_keys=list(),
)

Store-level diagnostics for an immutable VIN offline dataset.

1.1.6.1 Attributes

Name Description
store_dir Absolute store directory inspected for diagnostics.
version Offline dataset format version from manifest.json.
num_samples Number of rows listed in sample_index.jsonl.
sampled_samples Number of rows scanned for per-sample statistics.
split_counts Sample counts keyed by split name.
num_scenes Number of unique scene IDs in the sample index.
num_snippets Number of unique (scene_id, snippet_id) pairs.
materialized_blocks Optional block flags from the manifest.
candidate_count Distribution of valid candidate counts per sample.
rri Distribution of finite oracle RRI values across sampled candidates.
vin_points Distribution of VIN point lengths across sampled snippets.
numeric_bytes Approximate bytes occupied by manifest-declared numeric shard blocks.
block_shapes Stored numeric block shapes keyed by logical block name.
block_diagnostics Manifest-declared block diagnostics for Streamlit and CLI tables.
sample_summaries Per-row sanity summaries for sampled records.
candidate_count_values Sampled candidate-count values used for histograms.
rri_values Sampled finite oracle RRI values used for histograms.
vin_point_values Sampled VIN point lengths used for histograms.
rri_component_values Sampled finite RRI component values keyed by component name.
rri_component_summaries Finite-value summaries for RRI component values.
candidate_pose_values Candidate pose diagnostics in the reference-rig frame.
candidate_pose_summaries Finite-value summaries for candidate pose diagnostics.
memory_diagnostics Estimated per-sample runtime memory summaries.
backbone_diagnostics Streaming statistics for sampled backbone numeric fields.
scene_labels_available Whether the root capability includes measured scene supervision, distinct from zero labels.
batch_shapes_available Whether a one-step labelled VIN batch preview is supported by this capability.
batch_shapes Shape preview from one VinOracleBatch read path.
rri_component_pairs Identity-aligned RRI/component observations admitted by a shared finite mask.
rri_pairing_unavailable_sample_keys Scanned labelled sample keys lacking persisted candidate IDs; marginals remain available.

1.1.7 VinOfflineDatasetStatsRequest

data_handling.vin_store.diagnostics.VinOfflineDatasetStatsRequest(
    selection,
    generation,
    max_samples=512,
)

Complete generation-bound request for one root diagnostics acquisition.

1.1.7.1 Attributes

Name Description
selection Canonical root-only store selection.
generation Replacement-sensitive filesystem generation captured before acquisition.
max_samples Maximum rows scanned for tensor statistics; None scans every row.

1.1.7.2 Methods

Name Description
digest Return the deterministic cache and retained-result identity.
1.1.7.2.1 digest
data_handling.vin_store.diagnostics.VinOfflineDatasetStatsRequest.digest()

Return the deterministic cache and retained-result identity.

1.1.8 VinOfflineMemoryDiagnostic

data_handling.vin_store.diagnostics.VinOfflineMemoryDiagnostic(
    component,
    mean_mib,
    median_mib,
    p95_mib,
)

Estimated per-sample runtime memory for one offline-store component.

1.1.8.1 Attributes

Name Description
component Component name such as "backbone" or "oracle_rri".
mean_mib Mean estimated MiB across sampled rows.
median_mib Median estimated MiB across sampled rows.
p95_mib 95th percentile estimated MiB across sampled rows.

1.1.9 VinOfflineSampleDiagnostic

data_handling.vin_store.diagnostics.VinOfflineSampleDiagnostic(
    sample_index,
    sample_key,
    scene_id,
    snippet_id,
    split,
    shard_id,
    row,
    candidate_count,
    rri,
    vin_points,
)

Per-row sanity summary for one sampled VIN offline record.

1.1.9.1 Attributes

Name Description
sample_index Global sample index.
sample_key Stable dataset sample key.
scene_id ASE scene identifier.
snippet_id ASE snippet identifier.
split Dataset split membership.
shard_id Shard that stores the row.
row Shard-local row index.
candidate_count Valid scene candidate count, or None when scene supervision is unavailable.
rri Finite RRI summary for valid candidates.
vin_points VIN point-length summary for the row.

1.2 Functions

Name Description
capture_vin_offline_dataset_stats_request Capture a root-only diagnostics request without reading payload arrays.
collect_vin_offline_dataset_coverage Compare raw ASE tar-header coverage against an immutable VIN store.
collect_vin_offline_dataset_stats Collect coverage, shape, RRI, and memory diagnostics for a VIN store.
inspect_vin_offline_dataset_stats Acquire detached root statistics while rejecting pre-read or mid-read replacement.

1.2.1 capture_vin_offline_dataset_stats_request

data_handling.vin_store.diagnostics.capture_vin_offline_dataset_stats_request(
    store,
    *,
    max_samples=512,
)

Capture a root-only diagnostics request without reading payload arrays.

1.2.2 collect_vin_offline_dataset_coverage

data_handling.vin_store.diagnostics.collect_vin_offline_dataset_coverage(
    store,
    *,
    max_tars=None,
    progress_cb=None,
)

Compare raw ASE tar-header coverage against an immutable VIN store.

1.2.2.1 Parameters

Name Type Description Default
store VinOfflineStoreConfig Immutable VIN offline store to inspect. required
max_tars int | None Optional cap on raw tar shards scanned. None
progress_cb Callable[[int, int], None] | None Optional callback receiving (done, total) tar counts. None

1.2.2.2 Returns

Name Type Description
VinOfflineCoverageStats Coverage summary for Streamlit and tests.

1.2.3 collect_vin_offline_dataset_stats

data_handling.vin_store.diagnostics.collect_vin_offline_dataset_stats(
    store,
    *,
    max_samples=512,
)

Collect coverage, shape, RRI, and memory diagnostics for a VIN store.

1.2.3.1 Parameters

Name Type Description Default
store VinOfflineStoreConfig Immutable VIN offline store to inspect. required
max_samples int | None Optional cap on rows scanned for per-sample tensor stats. None scans the whole store. 512

1.2.3.2 Returns

Name Type Description
VinOfflineDatasetStats Store-level diagnostics suitable for tests, CLI output, or Streamlit.

1.2.4 inspect_vin_offline_dataset_stats

data_handling.vin_store.diagnostics.inspect_vin_offline_dataset_stats(request)

Acquire detached root statistics while rejecting pre-read or mid-read replacement.

Back to top