1 utils.plotting.benchmarks

utils.plotting.benchmarks

Ingest, aggregate, plot, and persist paired Python/Mojo benchmarks.

This module provides typed benchmark records/summaries, CSV ingestion, within-stratum aggregation, Plotly latency/speedup/scaling/throughput figures, and report persistence. It owns comparative summarization and presentation; benchmark execution and raw timing collection remain with the benchmark harnesses.

The records preserve raw trial identity while summaries compare implementations only within the same benchmark and problem-size stratum.

1.1 Classes

Name Description
BenchmarkRecord One measured implementation trial before statistical aggregation.
BenchmarkSummary Aggregated statistics for one benchmark / implementation / size tuple.

1.1.1 BenchmarkRecord

utils.plotting.benchmarks.BenchmarkRecord(
    benchmark,
    implementation,
    latency_ms,
    problem_size='default',
    throughput_items_per_s=None,
    trial=None,
)

One measured implementation trial before statistical aggregation.

Latency is stored in milliseconds; throughput, when available, is measured in processed workload items per second.

1.1.1.1 Attributes

Name Description
benchmark Stable workload name used to pair implementations.
implementation Implementation label, conventionally python or mojo.
latency_ms Wall-clock latency for this trial in milliseconds.
problem_size Human-readable workload-size stratum for like-for-like aggregation.
throughput_items_per_s Optional processed-item rate in items per second.
trial Optional source trial or repetition identifier.

1.1.2 BenchmarkSummary

utils.plotting.benchmarks.BenchmarkSummary(
    benchmark,
    implementation,
    problem_size,
    count,
    latency_q25_ms,
    latency_median_ms,
    latency_q75_ms,
    latency_mean_ms,
    latency_std_ms,
    throughput_mean_items_per_s=None,
)

Aggregated statistics for one benchmark / implementation / size tuple.

1.1.2.1 Attributes

Name Description
benchmark Stable workload name shared with the contributing records.
implementation Implementation summarized by this row.
problem_size Workload-size stratum; implementations are never mixed across strata.
count Number of raw trials included in the summary.
latency_q25_ms 25th percentile of observed trial latency in milliseconds.
latency_median_ms Median wall-clock latency in milliseconds.
latency_q75_ms 75th percentile of observed trial latency in milliseconds.
latency_mean_ms Arithmetic mean wall-clock latency in milliseconds.
latency_std_ms Population standard deviation of latency in milliseconds.
throughput_mean_items_per_s Mean item throughput when every contributing row reports throughput.
key Return the logical grouping key excluding implementation.
display_label Return a compact label for grouped bar charts.

1.2 Functions

Name Description
build_latency_figure Build median latency bars with the observed interquartile range.
build_scaling_figure Build per-benchmark scaling curves across problem sizes.
build_speedup_figure Build a speedup chart using median latency ratios.
build_throughput_figure Build a grouped throughput comparison chart when throughput data is available.
compute_speedups Compute ratios of within-stratum median latencies.
load_benchmark_csv Load benchmark trials from CSV.
summarize_benchmarks Aggregate admitted trials into descriptive latency summaries.
write_benchmark_report Write interactive benchmark figures and optional PNG snapshots.

1.2.1 build_latency_figure

utils.plotting.benchmarks.build_latency_figure(
    summaries,
    *,
    title='Python vs Mojo Latency',
)

Build median latency bars with the observed interquartile range.

1.2.2 build_scaling_figure

utils.plotting.benchmarks.build_scaling_figure(
    summaries,
    *,
    title='Latency Scaling Across Problem Size',
)

Build per-benchmark scaling curves across problem sizes.

1.2.3 build_speedup_figure

utils.plotting.benchmarks.build_speedup_figure(
    summaries,
    *,
    baseline_impl='python',
    target_impl='mojo',
    title='Mojo Speedup Over Python',
)

Build a speedup chart using median latency ratios.

1.2.4 build_throughput_figure

utils.plotting.benchmarks.build_throughput_figure(
    summaries,
    *,
    title='Python vs Mojo Throughput',
)

Build a grouped throughput comparison chart when throughput data is available.

1.2.5 compute_speedups

utils.plotting.benchmarks.compute_speedups(
    summaries,
    *,
    baseline_impl='python',
    target_impl='mojo',
)

Compute ratios of within-stratum median latencies.

Each (benchmark, problem_size, implementation) summary identity must be unique. The summary interface has no repetition identity, so the estimand is the baseline median divided by the target median, not a median of paired trial-level ratios.

1.2.5.1 Raises

Name Type Description
ValueError If more than one summary has the same complete identity.

1.2.6 load_benchmark_csv

utils.plotting.benchmarks.load_benchmark_csv(path)

Load benchmark trials from CSV.

Expected columns: - required: benchmark (or kernel/workload), implementation, latency_ms (or time_ms/duration_ms) - optional: problem_size (or size/n), throughput_items_per_s, trial

1.2.7 summarize_benchmarks

utils.plotting.benchmarks.summarize_benchmarks(records)

Aggregate admitted trials into descriptive latency summaries.

Every latency must be finite and strictly positive. The 25th and 75th percentiles use linear interpolation and describe the observed trial distribution; they are not uncertainty bounds for the median.

1.2.8 write_benchmark_report

utils.plotting.benchmarks.write_benchmark_report(
    records,
    *,
    out_dir,
    title_prefix='Python vs Mojo',
    write_png=False,
)

Write interactive benchmark figures and optional PNG snapshots.

1.2.8.1 Returns

Name Type Description
dict[str, Path] Mapping from figure kind to each generated artifact path.
Back to top