Running the benchmark suite

Run the four-case core suite with uv run poe bench. It measures bounded windows of full-resolution problems and prints warm time, startup time, peak memory and check status. Numerical reference preparation is optional for exploratory timing. The runner is specified by SEP 30. The rules it applies, the measurement windows and what a performance claim reports are in Benchmarks; this page is the runner’s manual.

uv run poe bench --dry-run
uv run poe bench --out .benchmarks/suite/runs/first-look
uv run poe bench --case rdw --threads 1

The default uses the native CPU backend and requires it to be installed in the environment. For a small first check without the native backend, use uv run poe bench --case rdw --backend numpy_ref --threads 1. The 384-square Kerr case is a substantial memory workload: its conservative admission budget is about 36 GiB. The controller records a refusal if available memory is insufficient; select individual cases on smaller machines.

poe bench accepts the same arguments as python -m benchmarks. Omitting a subcommand means run. No profile, baseline environment or reference preparation is required for a quick run. Missing references are labelled not-qualified and do not prevent timing.

Select a measurement

Planning imports metadata only; it does not import NumPy, initialise a backend, translate a simulation or prepare references:

uv run python -m benchmarks list
uv run python -m benchmarks run --dry-run --set core
uv run python -m benchmarks list --all

The table includes conservative memory budgets and reference receipt status. Planning checks reference metadata compatibility only; the worker verifies array hashes and numerical results. Missing cost measurements are shown as unknown.

The default selection is core, quick, native, at the available production width. Declare a machine profile before a standard comparison. The alias must be suitable for a public report; the width must fit the current allocation:

uv run python -m benchmarks profile .benchmarks/suite/profiles/workstation.json \
    --alias workstation --production 8
uv run python -m benchmarks run --set core \
    --machine-profile .benchmarks/suite/profiles/workstation.json

Use --case or --set repeatedly to select a deduplicated union. --all selects all sets, including visible pending definitions. A pending definition produces an explicit exclusion; it never becomes a successful measurement. --threads accepts positive integers and production, for example 1,production. Quick mode defaults to production; standard mode defaults to both widths. Unsupported tracks and widths are reported explicitly.

Core case	Headline measurement	Fixed diagnostic
Four-mode RDW	0.02–0.04 m, following a 0–0.02 m lead-in	Ten 2 mm entrance steps
Six-mode CtC	0.00017–0.00034 m, following its lead-in	Ten 17 micrometre entrance steps
384-square Kerr beam	Fixed RK4IP only, one lead-in step and ten 0.5 micrometre steps	The same row
256-node radial filament	0–0.03 m, four observation planes	Ten frozen RK pilot steps, ending at 0.274524 mm

RDW, CtC and radial headline rows each use default Multistep() and RungeKutta(). The original guides, gas-flow profiles and grids remain intact. The measurement window does not redefine a pressure-gradient guide’s endpoint. The radial diagnostic uses the first ten accepted steps of the NumPy default-RK pilot at rtol=1e-6; its edges are stored with the registry and included in the measurement identity. It does not run a pilot during ordinary collection.

The modal-capillary and mode-averaged sets add nine bounded cases. Both adaptive methods and a ten-step fixed RK diagnostic are available on NumPy, Numba and native CPU. Select Numba with --backend numba_cpu; its first window includes JIT compilation or cache loading in startup, before warm timing. The existing on-disk Numba cache is recorded as uncontrolled, so startup should not be presented as a reproducible cold compilation measurement.

uv run poe bench run --set mode-averaged --backend numba_cpu
uv run poe bench run --case argon_blueshift --track fixed-rk --threads 1

Additional case	Adaptive entrance interval	Fixed diagnostic
One resolved mode RDW	0–39.1377 mm	Ten frozen pilot steps
Reduced two-mode CtC	0–1.31141 mm	Ten 120 micrometre steps
Resolved argon blueshift	0–2.42717 mm	Ten frozen pilot steps
Resolved hydrogen Raman, 6 ps	0–13.2409 mm	Ten frozen pilot steps
Averaged helium RDW	0–39.3310 mm	Ten frozen pilot steps
Averaged argon blueshift	0–2.43336 mm	Ten frozen pilot steps
Averaged hydrogen Raman, 6 ps	0–13.2409 mm	Ten frozen pilot steps
Ideal N=3 GNLSE soliton	0–90.0316 mm	Ten frozen pilot steps
Raman/shock GNLSE	0–0.945275 mm	Ten frozen pilot steps

These windows end at the tenth accepted step of independent NumPy RK pilots at rtol=1e-6, atol=1e-10. The registry retains the full precision of every edge. Each adaptive window saves its endpoint with ordinary statistics. The full guides, including the helium pressure profile over 1.5 m, are preserved. The averaged cases use the effective-area route; the resolved cases keep transverse quadrature even with one mode. The two hydrogen cases retain the explicitly named 6 ps grids. The ideal soliton retains the lesson’s full-period guide but propagates only the short entrance interval. These nine adaptive cases have a reviewed relative field ceiling of 1e-4, independent of the solver tolerance. Preparing or selecting reference receipts remains separate from ordinary timing.

The free-space set also adds these windows. The six adaptive additions use the same ten-step NumPy pilot protocol, with atol=0; both default methods and the frozen fixed-RK diagnostic are available. Kerr remains fixed-RK4IP only.

Additional case	Measured interval	Fixed diagnostic
Radial filament, 512 nodes	0–274.524 micrometres	Ten frozen pilot steps
Cartesian filament, 128-square	0–225.110 micrometres	Ten frozen pilot steps
Cartesian filament, 256-square	0–225.111 micrometres	Ten frozen pilot steps
Line, 128 points	0–10.8716 mm	Ten frozen pilot steps
Planar, two modes by 32 points	0–6.79474 mm	Ten frozen pilot steps
Radial absorber, 64 nodes	0–10.8716 mm	Ten frozen pilot steps
Kerr beam, 192-square	0.5–5.5 micrometres	Ten 0.5 micrometre steps
Kerr beam, 768-square	0.5–3 micrometres	Five 0.5 micrometre steps

Kerr windows follow one lead-in step. The 768-square case excludes one-thread selection unless --diagnostic is explicit. Memory admission still applies: large Cartesian cases can be refused before construction on a smaller machine. Line, planar and radial-absorber cases have reviewed 1e-4 relative field ceilings. The larger filament additions remain unqualified for accuracy.

The advanced set preserves the source guides, coherent wall models and taper corners. Each case supports both adaptive methods and a fixed RK diagnostic:

Additional case	Adaptive entrance endpoint	Fixed diagnostic
Scalar coherent wall	200 micrometres	Ten 20 micrometre steps
Vector TE/TM coherent wall	200 micrometres	Ten 20 micrometre steps
Taper without wall	137.401 micrometres	Ten 20 micrometre steps
Taper with scalar wall	137.889 micrometres	Ten 20 micrometre steps
General step-index HE11 pair	135.895 micrometres	Ten frozen pilot steps
Linear two-channel GRIN	19.2601 micrometres	Ten frozen pilot steps
Nonlinear four-channel GRIN	620.057 micrometres	Ten frozen pilot steps
Moving solid guide	21.8692 micrometres	Ten frozen pilot steps

The linear GRIN case keeps the continuous-wave two-channel launch and its 0.784680 mm self-imaging guide; its temporal grid retains the source’s 1 mm length. Its ten accepted pilot steps include three rejected trials. Nonlinear GRIN retains input-noise seed 41. The moving guide retains its 5 mm taper corner in the unexecuted schedule tail. Weak-absorber and grazing-incidence warnings from the wall tutorials are declared by category and message. General-guide construction can dominate startup, independently of the short measured interval. All eight advanced cases have reviewed 1e-4 relative field ceilings, with reference and policy revisions independent of the workload.

The backend-exercise set covers particular execution paths on small problems:

Exercise	Measured entrance endpoint	Tracks and observations
Vector rotational Raman	0.291590 micrometres	Both adaptive methods; ten fixed pilot steps
Envelope plasma	8 micrometres	Both adaptive methods; four original 2 micrometre fixed steps
Helium/argon pressure gradient	21.8703 mm	Both adaptive methods; ten fixed pilot steps
Dispersive Kerr	15.2189 mm	Both adaptive methods; ten fixed pilot steps
Band/trapezoid plasma integration	135.895 micrometres	Two cases, both adaptive methods; ten fixed pilot steps
Batch	1 mm	Two distinct GNLSE members, ten fixed RK steps
Physical noise	135.895 micrometres	Both adaptive methods; ten fixed pilot steps; seed 13
Dense output	1.45795 mm	Both adaptive methods; eight interpolated observations
Split-step absorber	135.895 micrometres	Both adaptive methods; ten fixed pilot steps
Ordinary statistics	135.895 micrometres	Both adaptive methods; ten fixed pilot steps
Far-field statistics	863.801 micrometres	Both adaptive methods; eight observations with only the exit field retained; ten fixed pilot steps

Vector Raman uses the current optmat rotational hydrogen response with vibration disabled, retaining the circular pump and delayed oblique probe launch. Its low-density linewidth warning is recorded. The envelope-plasma guide ends after four source steps, so that shorter interval is retained. The other new windows use ten accepted default-RK pilot steps. Dense output visits intermediate planes without forcing the integrator to land there. Far-field observations include apertures, spectral regions and periodic absorbed power. Fixed diagnostics time propagation alone and report observation costs separately.

The batch workload uses the actual backend batch implementation, retaining each member’s own statistics, source counts and field check. Reset reuses member buffers and FFT plans. These exercises can be timed before reference review; their numerical evidence is collected separately through qualify. The eleven adaptive exercises have reviewed 1e-4 relative field ceilings. The fixed batch check remains a matched-method diagnostic.

Timing boundaries and accounting

These boundaries implement the measurement windows of the benchmark policy. Each sample runs in a fresh process. Its thread environment is set before scientific imports, with solprop_native loaded before pyFFTW. Budget policies clear inherited Kokkos pool overrides and the FFT-size override. all-pools-N explicitly forces the pools; as-installed preserves the environment. This keeps an automatic-policy measurement separate from a forced-width measurement. The backend and FFT plans are constructed once. After a first complete window, the worker calls set_initial, repeats the bounded lead-in outside warm timing, and measures the declared window again.

Adaptive timing includes propagation, declared observations, ordinary statistics and in-memory output. Fixed timing includes propagation; the final observation and statistics are measured separately. Fresh-process startup is measured by the parent through receipt of the first completed window. Process startup and warm propagation answer different questions and are reported separately.

source_evaluations counts actual Runge–Kutta source dispatches since the most recent set_initial. Rejected trials contribute work; a reused stage contributes no new evaluation. NumPy may therefore report more evaluations than a backend that reuses DOPRI5’s last stage. Adams retains its existing response-call ledger, and low-storage methods retain their existing evaluation counters. These are execution facts, not changes to the simulation or kernel-contract schemas.

The worker reports lifetime peak RSS after measurement and before reference loading. Allocation-ledger bytes are reported separately. Unsupported telemetry carries an unavailable reason; a missing counter is not zero. Admission estimates are conservative provisional budgets until compatible empirical receipts are qualified. An unknown estimate requires --allow-unestimated and an explicit --memory-limit in bytes; such a collection is exploratory. Each attempt also reports controller_cpu_seconds: CPU time consumed by the parent while launching, monitoring and reading the worker. It includes process sampling and receipt handling; it is separate from the child’s warm propagation time.

Collection, recovery and comparison

Quick mode takes one sample per configuration and has no adoption verdict or MAD estimate. Standard mode starts with five samples per configuration, uses balanced shuffled rounds, and alternates adjacent baseline/candidate order. A short separate pilot can select up to ten individually timed repetitions; the same count is frozen for both arms. No repetition is selected merely because it was fastest.

For a first look, collect the same selection before and after a change, then compare the saved bundles. Quick comparison prints descriptive percentage changes even without qualified references or a clean baseline; it does not approve adoption:

uv run poe bench compare .benchmarks/suite/runs/before .benchmarks/suite/runs/after

Use compare --json for the detailed machine-readable result, including resource flags. For a controlled paired comparison, prepare both Python environments first:

uv run python -m benchmarks run --mode standard --set core \
    --machine-profile .benchmarks/suite/profiles/workstation.json \
    --baseline-python /path/to/baseline/bin/python \
    --candidate-python /path/to/candidate/bin/python \
    --out .benchmarks/suite/runs/comparison
uv run python -m benchmarks compare .benchmarks/suite/runs/comparison

The controller admits one measurement worker at a time. A guard miss defers the whole pair once in standard mode, then records contention if it persists. Contended samples remain in all-sample tables. Clean paired summaries omit both members if either is contended. Unknown guard confidence cannot establish a clean comparison. --force skips waiting and explicitly makes the collection exploratory.

Run bundles live below the ignored .benchmarks/suite/ directory unless --out overrides it. They contain a manifest, immutable attempt shards, raw worker logs, and derived samples.csv, summary.csv and accuracy.csv tables. A run lock prevents concurrent writers. Temporary partial files do not count as completed attempts. Use run --resume RESULTS to restore the saved selection and options and retry an interrupted run; completed sample identities are skipped and previous failures remain on disk. Worker files left by a canceled collector remain intact; a retry uses a fresh attempt number. An explicit conflicting option refuses resume. Changed harness, environment, selection or machine identity refuses resume.

Comparison requires the complete affected selection and compatible workload, resolved method, measurement and profile identities. Build identities may differ. Adaptive and fixed tracks, and different widths, have separate aggregates. Standard adoption applies the aggregate rule of the benchmark rules and additionally requires qualified numerical references, sufficient clean paired rounds and a gain exceeding measured adjacent-round repeatability. More than 20% contention, insufficient adjacent clean rounds, or missing qualification leaves the result unresolved. Missing rows are never removed to improve the aggregate. Peak-memory and startup regressions have separate advisory flags.

Use --allow-accuracy-failure when timing should continue despite failed numerical checks. It preserves each failed check and its measured error, completes timing rounds, and labels the entire collection exploratory. Successful execution then exits zero even if an accuracy check failed; crashes, timeouts and memory refusals still fail. This option does not turn a failed reference check into a pass, and an override collection cannot establish adoption:

uv run python -m benchmarks run --case rdw --allow-accuracy-failure \
    --out .benchmarks/suite/runs/rdw-exploratory

Fast A/B comparison while iterating

A standard paired comparison is a final check, not an iteration tool. Each sample is a fresh process that imports, constructs the case, runs a first window and only then its warm repetitions; standard mode adds paired pilots, a counter guard before every pair and five to eleven rounds. Construction dominates: in a standard run of the advanced set on a 36-core workstation it took about 70 % of the collection time, the first and timed windows about 9 % each, and guards, process start and validation the rest. The whole set at two widths took about five hours, though most windows last milliseconds to seconds.

While developing a performance change, time only the cases it should affect with the ab command, and keep the standard comparison for the settled change:

uv run python -m benchmarks ab --case step_index_pair --case taper_wall \
    --track adaptive-rk --threads 1 \
    --baseline-python /path/to/baseline/bin/python \
    --candidate-python /path/to/candidate/bin/python

Selection (--case, --set, --all, --track, --threads, --backend, --machine-profile) works as for run, defaulting like a quick run to the adaptive tracks at the production width. Cases, measurement windows and lead-ins come from the registry unchanged. Each round starts one worker per build and thread width. The worker constructs each selected case once, runs one untimed warm-up window, then times --repeats windows (default 5), each after the usual reset and lead-in. There are --rounds rounds (default 3), and the build that runs first alternates between rounds. A selection of a few cases therefore costs a few constructions per build instead of one per sample: four advanced cases at one track compare in a few minutes.

The command prints, per configuration, each build’s median window time, the median over rounds of the paired candidate/baseline ratio of per-round medians (below 1 means the candidate is faster), the range of those ratios over rounds, whether both builds took the same accepted and rejected steps, and the relative L2 difference between the two builds’ retained fields from the first round. A geometric mean of the ratios follows. Passing the same interpreter for both builds gives an A/A control, whose ratio range shows the noise to expect on that machine.

Results go to .benchmarks/suite/ab/ unless --out names a new directory: plan.json, results.json with every timed window and the runtime identity of each worker, the raw worker logs and the saved fields. The guard takes one one-second reading before each worker and labels a busy machine, but never waits. There is no pilot, reference check, memory sampling, resume or adoption verdict; the batch exercise is not supported. Use a quiet machine, and confirm a promising result with a standard comparison.

Numerical qualification

Reference preparation is explicit and runs outside ordinary timing collection:

uv run python -m benchmarks qualify --case rdw --threads 1 \
    --out .benchmarks/suite/references/rdw-review
uv run python -m benchmarks run --case rdw \
    --references .benchmarks/suite/references/rdw-review

Qualification is a one-time preparation step for each reference revision. Routine timing validates the first successful sample for each arm/configuration after the timer and memory receipt. Later samples check finite output and the validated shape, and link to the original validation rather than repeating its error measurements. A changed reference/policy or a failed sample requires a fresh check. Timing does not run a reference convergence campaign. The qualification path retains tight RK at relative tolerance 1e-10, a 1e-9 refinement witness, a matched-method NumPy result, a candidate result, reset evidence and array hashes. It evaluates the worst-plane relative field error in the requested band and preserves the full-band error. Fixed methods use a same-schedule NumPy reference. Qualification runs also retain fresh-versus-reset fields, work counts, observed controller histories, statistics and noise state. RDW retains its weak-bin and 100–150 nm band metrics; CtC retains its weak-bin metric. Modal energy differences use the ordinary Parseval weights and are diagnostic only.

An empty mode can have an undefined spectral centroid, and a continuous wave can have an undefined pulse width. Reset evidence records these statistic masks and compares the finite entries separately. Non-finite fields, histories or infinities remain errors.

The benchmark’s solver tolerance remains rtol=1e-6. Its separate accuracy envelope bounds the measured field and retained-band errors against the reference:

Adaptive case	Largest pilot error over both integrators	Reviewed ceiling
RDW	2.045 × 10⁻⁵ (UV band)	1 × 10⁻⁴
CtC	2.055 × 10⁻⁶ (weak bins)	1 × 10⁻⁵
Radial 256	1.206 × 10⁻⁶ (worst-plane field)	1 × 10⁻⁴
Nine added modal/GNLSE cases	6.013 × 10⁻⁷ (relative field)	1 × 10⁻⁴

These ceilings reuse target levels from the source campaigns with headroom for ordinary rounding and solver variation; they are not fitted to timing results. The approved defaults match exact measurement identities. A changed workload must be reviewed again. Other measurements without a reviewed positive envelope remain not-qualified. The explicit --accuracy-bound option supplies a reviewed replacement ceiling. Adaptive field and band ceilings must remain at least ten times their respective measured reference refinement floors. An envelope pass alone does not qualify adoption. After reviewing the retained reset fields, controller histories, statistics and work counts, use qualify --recheck ... --reviewed to mark that reference eligible for standard comparison. This checks that all four reference runs succeeded and retained reset evidence, with reset fields inside the same approved field ceiling. History and statistics need their case-specific numerical interpretation; this option confirms that review without imposing identical adaptive trajectories. A changed policy does not inherit a previous review automatically. Ordinary timing needs neither this review nor a reference. Clean-pair, warning and resource checks remain part of each standard comparison.

Revise an accuracy policy or reference

Workload, reference, accuracy-policy and result-schema revisions are independent. An improved reference or a revised accuracy limit does not rename an unchanged benchmark case. Receipts are immutable, carry the source/build evidence, and link to the receipt they supersede with a revision note.

To reassess retained evidence with a reviewed ceiling, without executing another simulation:

uv run python -m benchmarks qualify --case rdw \
    --recheck .benchmarks/suite/references/rdw-v1 --accuracy-bound 2e-4 \
    --revision-note "Reviewed margin after numerical investigation" \
    --out .benchmarks/suite/references/rdw-v2

The ceiling in this example illustrates the command; it is not an additional approved RDW default. Rechecking verifies artifact hashes, reevaluates retained errors and refinement floors, and writes a new receipt. It cannot fix an incorrect reference field. After fixing a reference implementation, prepare fresh evidence:

uv run python -m benchmarks qualify --case rdw \
    --supersedes .benchmarks/suite/references/rdw-v1 \
    --revision-note "Recomputed after correcting the reference implementation" \
    --out .benchmarks/suite/references/rdw-v2

Replacement reference revisions advance automatically; a changed ceiling also advances the policy revision. Explicit --reference-revision and --policy-revision values may advance these further. Both comparison arms must be validated against the same receipt to support adoption. Old bundles remain historical evidence under their original policy; changing a ceiling does not rewrite their accuracy results. Revalidation against a replacement needs retained output fields or a new collection.

Warnings are captured with phase, category, message and occurrence count. A case may declare a narrow expected pattern; undeclared warnings prevent adoption. The Kerr recipe retains its source’s Malitson UV extrapolation warning explicitly.

Native source coverage

Coverage is an explicit developer diagnostic. It reports how much of the compiled native CPU backend the selected benchmark cases execute; it is not run by ordinary timing commands and has no automatic notifications or numerical target.

Prepare a separate Clang CPU build of native/ with SOLPROP_NATIVE_ENABLE_COVERAGE=ON as described in native/README.md. Use LLVM tools from the same compiler release. Keep the ordinary timing environment separate: timing commands refuse an instrumented build. Then, from the repository root:

uv run python -m benchmarks coverage --set backend-exercise \
    --python /path/to/coverage-environment/bin/python \
    --native-root native \
    --llvm-bin /path/to/matching/llvm/bin > coverage.log

The command prints a redirectable line/function/branch summary and the local report path. Defaults are the core set and one thread. All admitted tracks of each case are included; --case, --set, --all, --track, --threads and --diagnostic select a narrower or broader diagnostic. Memory admission and timeouts still apply. Failed, refused and excluded configurations remain visible.

Each configuration runs once, including construction, its declared lead-in and observations. There are no warmups, calibration loops or timing samples. A separate second execution collects Kokkos Tools labels. Capability tags describe intended workloads; observed labels show which named operations executed. Neither implies that every branch of a capability was exercised.

Reports default to ignored .benchmarks/suite/coverage/; --out chooses another local directory. The bundle contains per-case, per-set and union HTML reports, line/function/branch CSV views, labels, raw profiles and a copy of the exact binary. They are local development artifacts, never committed automatically. Delete them when no longer useful. Generated benchmark or coverage results do not belong in the repository; only definitions, tests, policies and concise implementation notes are versioned. export below writes an explicitly requested sanitized snapshot; it does not publish or commit one.

Atomic Clang counters support threaded CPU execution. Process/module-specific profile names avoid collisions; source digests, binary hashes and profile hashes prevent mixing builds. Dependency code is excluded by source path, including dependency templates instantiated in a project translation unit. LLVM supplies the line-count aggregation. Uncovered compiled functions and branch outcomes are reported as misses. Unsupported configurations and compiled-out device capabilities are separate. Uninstantiated templates and preprocessor-elided paths have no LLVM coverage-map denominator; their counts remain unavailable, not zero. See the Clang coverage workflow.

Coverage is informational and has no required percentage. A timeout or memory refusal leaves the selection incomplete and returns a nonzero exit status, while reports for successful configurations remain available. Inspect summary.json and configurations.csv alongside the HTML union to see its actual scope. Instrumented execution can be much slower than ordinary propagation; its elapsed time is not a benchmark result. A retry at another thread width is a separate collection and does not erase the original failure.

Local tables and public export

Collection writes normalized CSV views for runs, cases, configurations, systems, CPU allocations, measured processes and observed libraries alongside the existing sample, summary and accuracy tables. Records link through run, configuration and attempt identifiers. Missing telemetry carries a reason rather than a measured zero. Library records distinguish the initial inventory from pools observed after the measurement, including libraries loaded lazily by a backend.

Linux allocation records include physical cores, sockets, sibling CPUs and NUMA nodes when the operating system exposes them. Scheduler variables describe the allocation but do not prove exclusivity. Missing topology or residency information retains an explicit reason.

Collect thread and binding details separately from a timing campaign:

uv run poe bench diagnose --threads 1,production \
    --out .benchmarks/suite/thread-diagnostics

This runs the small diagnostic case at each width serially, recording per-thread CPU time and, on Linux, allowed CPUs and resident NUMA pages. It does not change affinity. An allowed CPU set is not evidence of where a thread actually executed. These exploratory results cannot establish adoption.

Export a separate public snapshot explicitly:

uv run poe bench export .benchmarks/suite/runs/my-run \
    --out .benchmarks/public/my-run --alias apple-silicon-development

The destination must be new or empty. The export retains timings, failures, contention, work counts, safe build identities and the selected configurations. It omits hostnames, original machine aliases, local paths, process identities, environment values, raw logs and free-form diagnostics. Its manifest states those omissions. The command writes files; it does not commit or upload them.

The runner and explicit native coverage command are ready for development use. Clean machine baselines and complete reference checks for the largest spatial cases remain follow-up work. Planning currently uses conservative memory budgets and reports unknown runtime costs; automatic reuse of measured memory and cost receipts is also a follow-up. These limits do not prevent exploratory timing.

A performance baseline belongs to a particular source and build. Prepare a fresh one after integrating performance changes, using matched workloads and thread budgets when comparing Numba and native CPU. This does not require repeating numerical reference preparation unless the reference’s workload or numerical assumptions changed.

Indicative results of earlier measurements are summarised under indicative results.

See SEP 30, Part II for the implementation decisions, measured evidence and remaining limitations.