solprop.output.schema#

Run metadata, saved-plane records and the one save protocol both handlers implement.

An output file has to answer one question years later: what produced this? The record types here are that answer. RunMetadata carries the grid, the modes, the serialised physics configuration, the package versions, the git provenance of the running checkout, the noise seed and realisation, and the stepper settings; SavedPlane carries one saved position with its field, its statistics and the step history since the previous plane. Output is the protocol solprop.output.memory.MemoryOutput and solprop.output.hdf5.HDF5Output both implement, so processing reads either through the same accessors.

Conventions on the way out. A SavedPlane holds its field in the internal FFT convention, as a backend produces it; a handler converts it with solprop.fourier.spectrum_to_optics() on the way to storage, so that what is stored is the window-centred optics spectrum a spectrometer would agree with, marked spectral_convention = "optics" (docs/dev/conventions.md §2; this module and the handlers are boundary place 2). The noise realisation and noise_phase are internal objects and stay in the FFT convention, marked so by INTERNAL_CONVENTION.

Provenance, not source. The provenance is the resolved configuration, the package versions and the git commit with a dirty flag – never the source text.

The resume key. RunMetadata.configuration_hash() digests the schema version, the grid, the modes, the configuration, the noise, the stepper settings, from schema 8 the identities and policies of the run’s material manifest, and from schema 9 the stepping method when it is not the tableau (SEP 13), and nothing else: not the wall-clock time the run started, not the package versions, and not the git provenance, so that a resume survives an unrelated commit or a dirty working tree. It is a SHA-256 of canonical JSON rather than hash(), so it is stable across processes, and it covers the physics rather than only the statistic names and the state shape, so a resume with different physics of the same shape is refused.

The manual page is docs/guide/results/output.md.

Module Attributes

`SCHEMA_VERSION`	Version of the record layout; incremented whenever a stored field changes meaning.
`INLINE_ARRAY_LIMIT`	The most elements an array input is recorded with inline (SEP 22, Q4).
`SPECTRAL_CONVENTION`	Sign convention of every spectrum a handler stores.
`PEAK_FIELDS_ANNOTATION`	Name of the post-run annotation holding the material manifest's peak-field report.
`INTERPOLATED_RESUMES_ANNOTATION`	Name of the post-run annotation listing where a dense run was resumed, m.
`INTERNAL_CONVENTION`	Sign convention of the arrays that stay internal: the noise field and its phase.
`FIELD_LAYOUT`	Axis order of the stored field, as `docs/dev/conventions.md` §6 fixes it.
`SimulationKind`	Which transform the run used.
`Convention`	Which sign convention an accessor should hand back.
`HISTORY_FIELDS`	Dtype of each `StepHistory` field, in the order they are laid out.
`STEP_STATS_INDEX`	Dtype of each `StepStats` index column, beside the statistics themselves.
`FieldsChoice`	Which saved planes keep their field: none of them, the named ones, or all.

Functions

`field_retention_refusal`(fields, field_planes)	Say what is wrong with a field-retention request, or that nothing is.
`package_versions`()	Return the versions of solprop and of the packages a stored file depends on.
`recorded_input`(value)	Return an input as the run record carries it: inline, or by its digest.
`resolve_metadata_write`(existing, new, *[, force])	Decide whether a handler may write metadata, under one rule for both handlers.
`serialise`(value)	Return any configuration object as a JSON-safe structure, dropping nothing.

Classes

`ArrayDigest`(sha256, dtype, shape)	An array recorded by the digest of its contents rather than inline.
`BackendRecord`(name, propagator_mode[, settings])	What executed a run, as the backend's own `metadata()` reported it.
`CheckpointOutput`(args, *kwargs)	Declare optional outcome and terminal-checkpoint persistence for a handler.
`FieldRetention`([fields, field_planes])	Which saved planes of a run keep their field (SEP 22 section 5.1).
`GitProvenance`(commit, branch, dirty)	Which commit of solprop produced a file, and whether it was modified.
`GridRecord`(*, kind, n_t, n_to, n_omega, ...)	The scalars that identify the grid a run was computed on.
`MarchSnapshot`(z, state, proposal, method, ...)	An Adams march's state at an accepted position, for a resume to continue.
`ModeRecord`(*, kind, effective_area, ...[, label])	One propagated mode, by class, effective area, channel label and configuration.
`NoiseRecord`(*, model, seed, spectrum[, ...])	The shot-noise realisation a run started from, and the seed that drew it.
`Output`(args, *kwargs)	What every output handler does, and all that processing may assume of one.
`PlaneLayout`(field_shape, stat_names, ...[, ...])	What every plane of one run has in common, fixed by the first plane saved.
`PlaneStore`()	The shared half of an output handler: one save protocol for every backing store.
`RunMetadata`(*, grid, modes, simulation_kind, ...)	Everything needed to say what produced a file, and to refuse a wrong resume.
`RunOutcome`(status, z[, message, ...])	Record completion and cumulative attempted work separately from run identity.
`SavedPlane`(*, z, field[, stats, ...])	One saved position: the field there, its statistics and how it was reached.
`StepHistory`(*, z, h, norm, accepted, wall_time)	What the stepper did between two saved planes.
`StepLimits`([dz_min, dz_max, dz_init, ...])	The caps a run put on the controller's step proposal.
`StepStats`(*, z, step[, values])	The statistics a backend evaluated between two saved planes.
`StepperRecord`(*, tableau, controller[, ...])	The solver settings a run used, including the schedule it actually took.
`TerminalCheckpoint`(plane, next_save_index[, ...])	Keep an accepted field between scheduled saves without adding an observation.

class solprop.output.schema.ArrayDigest(sha256, dtype, shape)[source]#

Bases: object

An array recorded by the digest of its contents rather than inline.

What a run record holds for an array input above INLINE_ARRAY_LIMIT elements (SEP 22 section 5.2, Q4): the SHA-256 of its bytes in little-endian order, with its dtype and shape. Two inputs that differ in one sample record different digests, so their runs have different configuration hashes; the array itself is not recoverable from the record.

Attributes:

sha256str: Sixty-four lowercase hexadecimal characters.
dtypestr: The array’s dtype, as NumPy names it ("complex128").
shapetuple of int: The array’s shape.

Examples

>>> import numpy as np
>>> from solprop.output.schema import ArrayDigest
>>> digest = ArrayDigest.of(np.zeros((2, 3)))
>>> digest.dtype, digest.shape, len(digest.sha256)
('float64', (2, 3), 64)

dtype: str#

classmethod of(values)[source]#

Return the digest of an array’s contents.

Parameters:

valuesnumpy.ndarray: The array; read once, in little-endian byte order, without a copy when it is already contiguous and little-endian.

Returns:

ArrayDigest: The digest, the dtype and the shape.

sha256: str#

shape: tuple[int, ...]#

class solprop.output.schema.BackendRecord(name, propagator_mode, settings=<factory>)[source]#

Bases: object

What executed a run, as the backend’s own metadata() reported it.

A saved run has to say what produced it and not only what was computed: the thread configuration alone moved one step by a factor of nine (docs/dev/backends.md section 3.4), and without this record a timing comparison has to read the realised kernel pool out of the backend itself.

It is outside the resume key, like the package versions and the git provenance: a run continued on the other backend agrees with an uninterrupted one to 2.2-2.7e-16 and is the same run. What it records is how, not what.

A resumed file keeps the record it was opened with, creation time and all, so the backend it names is the one that started the run. That is append mode’s rule for the whole record and not a special case for this group.

Attributes:

namestr: The registry name of the backend, "numpy_ref" or "numba_cpu".
propagator_modestr: Which the backend resolved the contract’s "auto" to, or which was asked for: "tabulate" or "recompute".
settingsdict: Everything else the backend reported: the threading regime, the four pool counts, the plan flag, the wisdom digest, the column family. Free-form because it is a backend’s own vocabulary, and JSON-serialisable because it is written to the file.

Examples

>>> from solprop.output.schema import BackendRecord
>>> record = BackendRecord.from_metadata(
...     {"backend": "numba_cpu", "propagator_mode": "recompute",
...      "numba_threads": 4}
... )
>>> record.name, record.propagator_mode, record.settings
('numba_cpu', 'recompute', {'numba_threads': 4})

classmethod from_metadata(metadata)[source]#

Return the record a backend’s metadata() describes.

Parameters:

metadatacollections.abc.Mapping: What solprop.backends.protocol.Backend.metadata() returned. backend and propagator_mode are named fields here; everything else becomes settings.

Returns:

BackendRecord: The record.

Raises:

OutputError: If either named field is missing.

name: str#

propagator_mode: str#

settings: Mapping[str, JsonValue]#

class solprop.output.schema.CheckpointOutput(*args, **kwargs)[source]#

Bases: Protocol

Declare optional outcome and terminal-checkpoint persistence for a handler.

property outcome: RunOutcome | None[source]#

Return recorded status, or None for legacy output.

Returns:

RunOutcome or None: Last persisted execution outcome.

record_outcome(outcome, checkpoint=None)[source]#

Persist execution status and an optional accepted terminal field.

Parameters:

outcomeRunOutcome: Cumulative work and completion status.
checkpointTerminalCheckpoint or None: Accepted field between regular saves. None retains a pending checkpoint during progress and clears it when the run completes.

property terminal_checkpoint: TerminalCheckpoint | None[source]#

Return the latest accepted terminal field outside regular observations.

Returns:

TerminalCheckpoint or None: The checkpoint, if one has been recorded.

solprop.output.schema.Convention#

Which sign convention an accessor should hand back.

alias of Literal[‘optics’, ‘fft’]

solprop.output.schema.FIELD_LAYOUT = '(n_z, n_modes, n_pol, n_omega)'#: Axis order of the stored field, as docs/dev/conventions.md §6 fixes it.

class solprop.output.schema.FieldRetention(fields='all', field_planes=())[source]#

Bases: object

Which saved planes of a run keep their field (SEP 22 section 5.1).

A run is homogeneous: it keeps the field on every plane, on the planes it names, or on none, and its statistics on every plane either way. A plane without its field stores no field and no noise_phase; what it saves is its position, its statistics and its step history.

Attributes:

fields{‘none’, ‘planes’, ‘all’}: Keep no field, the fields of field_planes, or every field (the default).
field_planestuple of int: Saved-plane indices for "planes", the entrance as 0 and negative from the end until resolved(); empty otherwise.

Raises:

OutputError: If the request is not valid (field_retention_refusal()).

Examples

>>> from solprop.output.schema import FieldRetention
>>> kept = FieldRetention("planes", (0, -1)).resolved(5)
>>> kept.field_planes, kept.keeps(4), kept.keeps(2), kept.kept(5)
((0, 4), True, False, 2)

describe()[source]#

Return the rule in words, for a refusal naming it.

Returns:

str: For example "fields='planes' (saved planes 0, 4)".

field_planes: tuple[int, ...] = ()#

fields: Literal['none', 'planes', 'all'] = 'all'#

keeps(index)[source]#

Return whether the saved plane at index keeps its field.

Parameters:

indexint: The plane’s index, the entrance as 0; the rule must be resolved().

Returns:

bool: True for every plane of an "all" run, a named plane of a "planes" run, and no plane of a "none" run.

kept(n_planes)[source]#

Return how many of a run’s planes keep their field.

Parameters:

n_planesint: The planes the run saves; the rule must be resolved().

Returns:

int: The planes whose field is stored.

resolved(n_planes)[source]#

Return the rule with its indices counted from the entrance, sorted, unique.

Parameters:

n_planesint: The planes the run saves, the entrance included.

Returns:

FieldRetention: The same rule with non-negative indices.

Raises:

OutputError: If an index lies outside the run’s planes.

solprop.output.schema.FieldsChoice#

Which saved planes keep their field: none of them, the named ones, or all.

alias of Literal[‘none’, ‘planes’, ‘all’]

class solprop.output.schema.GitProvenance(commit, branch, dirty)[source]#

Bases: object

Which commit of solprop produced a file, and whether it was modified.

Attributes:

commitstr: Full 40-character hexadecimal commit hash of the running checkout.
branchstr: Branch name, or 'HEAD' when the checkout is detached.
dirtybool: Whether the working tree had uncommitted changes to tracked files. A dirty run is reproducible only up to those changes, which is the point of recording it. It is a separate field rather than a suffix on the hash string, so the commit stays parsable.

Examples

>>> from solprop.output.schema import GitProvenance
>>> record = GitProvenance(commit="0" * 40, branch="main", dirty=False)
>>> len(record.commit), record.dirty
(40, False)

branch: str#

commit: str#

classmethod detect()[source]#

Return the provenance of the running solprop checkout, if there is one.

An installed release is not a checkout, git may be absent, and the package may sit inside an unrelated repository’s worktree – a virtual environment inside a user’s own project, whose git rev-parse answers with that project’s commit; every one of those returns None rather than a placeholder string, so that a caller can tell “no provenance” from “this commit”. The test that separates the last case from a development checkout is whether the repository found actually tracks this file.

Returns:

GitProvenance or None: The commit, branch and dirty flag, or None when they cannot be read.

Examples

>>> from solprop.output.schema import GitProvenance
>>> record = GitProvenance.detect()
>>> record is None or len(record.commit) == 40
True

dirty: bool#

class solprop.output.schema.GridRecord(*, kind, n_t, n_to, n_omega, n_omega_fine, dt, dt_fine, ffac, omega0, d_omega, length, reference_wavelength, omega_min, omega_max, omega_max_window, t_window_min, t_window_max)[source]#

Bases: object

The scalars that identify the grid a run was computed on.

Only scalars: the four axis arrays a solprop.grid.TimeGrid holds are reconstructible from these, and keeping them out of the record keeps the metadata JSON, and the configuration hash taken over it, small.

Attributes:

kind{‘field’, ‘envelope’}: Spectral layout, which fixes what the stored field means and which conversion the storage boundary applies.
n_tint: Coarse time samples.
n_toint: Fine time samples.
n_omegaint: Coarse spectral bins, the length of the stored field’s last axis.
n_omega_fineint: Fine spectral bins.
dtfloat: Coarse sample spacing, s.
dt_finefloat: Fine sample spacing, s.
ffacfloat: Oversampling factor, dimensionless.
omega0float: Reference angular frequency, rad/s.
d_omegafloat: Spectral bin spacing, rad/s.
lengthfloat: Propagation length the grid was built for, m.
reference_wavelengthfloat: Wavelength of omega0, m.
omega_minfloat: Lower in-band angular frequency, rad/s.
omega_maxfloat: Upper in-band angular frequency, rad/s.
omega_max_windowfloat: Upper edge of the active band, rad/s.
t_window_minfloat: Lower inner edge of the temporal collar, s: the window is flat between this and t_window_max and tapers to the ends of the axis outside them.
t_window_maxfloat: Upper inner edge of the temporal collar, s.

Examples

>>> from solprop.grid import EnvelopeGrid
>>> from solprop.output.schema import GridRecord
>>> grid = EnvelopeGrid.from_limits(1.0, 835e-9, (400e-9, 2e-6), 2e-12)
>>> record = GridRecord.from_grid(grid)
>>> record.kind, record.n_omega == grid.n_omega
('envelope', True)

d_omega: float#

dt: float#

dt_fine: float#

ffac: float#

classmethod from_grid(grid)[source]#

Return the record of a grid.

Parameters:

gridsolprop.grid.TimeGrid: The grid the run used.

Returns:

GridRecord: Its scalar description.

Examples

>>> from solprop.grid import FieldGrid
>>> from solprop.output.schema import GridRecord
>>> grid = FieldGrid.from_limits(1.0, 800e-9, (400e-9, 2e-6), 1e-12)
>>> GridRecord.from_grid(grid).kind
'field'

kind: Literal['field', 'envelope']#

length: float#

n_omega: int#

n_omega_fine: int#

n_t: int#

n_to: int#

omega0: float#

omega_max: float#

omega_max_window: float#

omega_min: float#

reference_wavelength: float#

t_window_max: float#

t_window_min: float#

solprop.output.schema.HISTORY_FIELDS: Mapping[str, type] = {'accepted': <class 'numpy.bool'>, 'h': <class 'numpy.float64'>, 'norm': <class 'numpy.float64'>, 'wall_time': <class 'numpy.float64'>, 'z': <class 'numpy.float64'>}#: Dtype of each StepHistory field, in the order they are laid out.

solprop.output.schema.INLINE_ARRAY_LIMIT = 1048576#

The most elements an array input is recorded with inline (SEP 22, Q4).

A larger array – a gigabyte spatial launch – is recorded by an ArrayDigest of its bytes, dtype and shape instead, which costs seconds of hashing per input rather than tens of gigabytes of JSON, and still changes with any one sample.

solprop.output.schema.INTERNAL_CONVENTION = 'fft'#: Sign convention of the arrays that stay internal: the noise field and its phase.

solprop.output.schema.INTERPOLATED_RESUMES_ANNOTATION = 'interpolated_resumes'#

Name of the post-run annotation listing where a dense run was resumed, m.

A list of the positions of the interpolated save planes a run observed densely was continued from with resume="interpolated": each continuation started afresh from a field interpolated inside a step, not a state the march passed through (SEP 14). Written by solprop.driver.simulation.Simulation.run(), outside the configuration hash.

type solprop.output.schema.JsonValue = bool | int | float | str | list[JsonValue] | dict[str, JsonValue] | None#

class solprop.output.schema.MarchSnapshot(z, state, proposal, method, order, rejections, last_rejected, positions, identities, slots, rows)[source]#

Bases: object

An Adams march’s state at an accepted position, for a resume to continue.

What a resume needs to carry on as the uninterrupted run would: the accepted state exactly as the backend held it, the history’s samples in their engine rows, which sample each history entry is, the order, the rejection count and the step the controller proposed next. The configuration hash already fixes the method, so a snapshot is never the record of what was computed, only of where a march stood; a resume that cannot use it starts the history afresh.

Equality is by identity (eq=False) because the fields hold arrays.

Attributes:

zfloat: The accepted position, m.
statenumpy.ndarray: The accepted state in the canonical layout and the internal FFT convention, exactly as the backend held it.
proposalfloat: The step the controller proposed next, m.
methodstr: The method’s representation, which the resuming run’s must equal.
orderint: The order chosen for the next step.
rejectionsint: Consecutive rejections so far.
last_rejectedbool: Whether the last trial was rejected.
positionsnumpy.ndarray: Each history entry’s position, m, oldest first.
identitiestuple of str: Each entry’s kind ("accepted" or "predicted").
slotsnumpy.ndarray: Each entry’s engine row.
rowscollections.abc.Mapping: The engine’s rows by name: the samples ("store") and, for a compensated exponential march, the sampled states and rates and each sample’s flow weights.

Raises:

OutputError: If the entries disagree in number, or a position or the state is not finite.

Examples

>>> import numpy as np
>>> from solprop.output.schema import MarchSnapshot
>>> snapshot = MarchSnapshot(
...     z=0.5, state=np.ones((1, 1, 1, 4), complex), proposal=0.1, method="M()",
...     order=2, rejections=0, last_rejected=False,
...     positions=np.array([0.4, 0.5]), identities=("accepted", "predicted"),
...     slots=np.array([0, 1]), rows={"store": np.zeros((4, 1, 1, 1, 4), complex)},
... )
>>> snapshot.nbytes
320

identities: tuple[str, ...]#

last_rejected: bool#

method: str#

property nbytes: int[source]#

Return the bytes the snapshot’s arrays hold.

Returns:

int: The state and every engine row.

order: int#

positions: Float[ndarray, 'n_entries']#

proposal: float#

rejections: int#

rows: Mapping[str, ndarray]#

slots: Int[ndarray, 'n_entries']#

state: Complex[ndarray, 'n_batch n_modes n_pol n_omega']#

z: float#

class solprop.output.schema.ModeRecord(*, kind, effective_area, configuration, label=None)[source]#

Bases: object

One propagated mode, by class, effective area, channel label and configuration.

Attributes:

kindstr: Qualified class name of the mode, 'module.ClassName'.
effective_areafloat: A_eff in m^2 at the reference position, the one number processing needs most often and the one a plot label carries.
configurationdict: The mode object as serialise() records it.
labelcollections.abc.Mapping or None: The physical channel this row of the mode axis is, as solprop.modes.protocol.ChannelLabel.as_mapping() writes it: the family, the azimuthal and radial orders, the parity, the handedness and the display label. Output schema 7; None on a record read back from an older file, which named its channels by class name and position and cannot be given quantum numbers after the fact. Outside RunMetadata.hashed() below schema 7, so that every file written before this field existed keeps the digest it was stamped with.

Examples

>>> from solprop.modes import MarcatiliMode
>>> from solprop.output.schema import ModeRecord
>>> record = ModeRecord.from_mode(MarcatiliMode(125e-6, "He", 1.0))
>>> record.kind
'solprop.modes.capillary.MarcatiliMode'
>>> None if record.label is None else record.label["display_label"]
'HE1,1:even'

configuration: Mapping[str, JsonValue]#

effective_area: float#

classmethod from_mode(mode, z=0.0)[source]#

Return the record of a mode.

Parameters:

modesolprop.modes.protocol.Mode: The mode.
zfloat, optional: Position along the waveguide the effective area is read at, m.

Returns:

ModeRecord: Its description.

Examples

>>> from solprop.modes import MarcatiliMode
>>> from solprop.output.schema import ModeRecord
>>> record = ModeRecord.from_mode(MarcatiliMode(75e-6, "He", 1.0))
>>> f"{record.effective_area:.6e}"
'8.421575e-09'

kind: str#

label: Mapping[str, JsonValue] | None = None#

class solprop.output.schema.NoiseRecord(*, model, seed, spectrum, spectral_convention='fft')[source]#

Bases: object

The shot-noise realisation a run started from, and the seed that drew it.

Equality is by identity (eq=False) because spectrum is an array; compare the seeds, which is what reproducibility means.

The spectrum is stored in the internal FFT convention, unlike the field: it is not a user-facing spectrum but the object the stepper adds at every stage, and conjugating it would make it disagree with the file it has to be read back into. spectral_convention says so explicitly.

Attributes:

model{‘input’, ‘modified’}: Which noise model the realisation belongs to.
seedint or None: Entropy of the generator, or None when it could not be recovered.
spectrumnumpy.ndarray: The realisation, shape (n_modes, n_pol, n_omega), complex, in the square root of watts.
spectral_conventionstr: Always 'fft'.

Examples

>>> import numpy as np
>>> from solprop.grid import EnvelopeGrid
>>> from solprop.noise import input_noise
>>> from solprop.output.schema import NoiseRecord
>>> grid = EnvelopeGrid.from_limits(1.0, 835e-9, (400e-9, 2e-6), 2e-12)
>>> realisation = input_noise(grid, np.random.default_rng(3))
>>> record = NoiseRecord.from_realisation(realisation)
>>> record.seed, record.spectral_convention
(3, 'fft')

classmethod from_realisation(realisation)[source]#

Return the record of a drawn realisation.

Parameters:

realisationsolprop.noise.NoiseRealisation: The realisation the run used.

Returns:

NoiseRecord: Its description.

Examples

>>> import numpy as np
>>> from solprop.grid import EnvelopeGrid
>>> from solprop.noise import modified_noise
>>> from solprop.output.schema import NoiseRecord
>>> grid = EnvelopeGrid.from_limits(1.0, 835e-9, (400e-9, 2e-6), 2e-12)
>>> NoiseRecord.from_realisation(modified_noise(grid, np.random.default_rng(1)))
...
NoiseRecord(model='modified', seed=1, ...)

model: str#

seed: int | None#

spectral_convention: str = 'fft'#

spectrum: Complex[ndarray, 'n_modes n_pol n_omega']#

class solprop.output.schema.Output(*args, **kwargs)[source]#

Bases: Protocol

What every output handler does, and all that processing may assume of one.

One protocol so that solprop.output.memory.MemoryOutput and solprop.output.hdf5.HDF5Output are interchangeable: a run is written to memory for a scan and to disk for a long propagation without the driver knowing which.

Everything a handler hands back through __getitem__ is as stored, so the field is in the optics convention; fields is the accessor that also offers the internal one, for direct comparison with a file stored in the FFT convention.

close()[source]#

Finish the run, releasing any resource held. Calling it twice is harmless.

fields(convention='optics')[source]#

Return every stored field, in the requested sign convention.

Parameters:

convention{‘optics’, ‘fft’}, optional: 'optics' returns the arrays as stored; 'fft' converts them back to the internal convention a backend works in, for direct comparison with a file stored in the FFT convention.

Returns:

numpy.ndarray: Shape (n_z, n_modes, n_pol, n_omega), complex.

property metadata: RunMetadata | None[source]#

Return the record of the run, or None before it is opened.

Returns:

RunMetadata or None: What produced the stored data.

property n_saved: int[source]#

Return how many planes have been stored.

Returns:

int: The number of planes.

open(metadata, *, force=False)[source]#

Begin a run, recording what produced it.

Parameters:

metadataRunMetadata: The record of the run about to start.
forcebool, optional: Whether to overwrite metadata already held for a different run.

save(plane)[source]#

Store one saved plane.

Parameters:

planeSavedPlane: The position, field, statistics and step history to store.

property stat_names: tuple[str, ...][source]#

Return the names of the statistics stored at every plane.

Returns:

tuple of str: The names, sorted; empty before the first plane is saved.

property step_stat_names: tuple[str, ...][source]#

Return the statistics stored between planes, if any.

Returns:

tuple of str: The names, sorted; empty when the run recorded no periodic subset.

solprop.output.schema.PEAK_FIELDS_ANNOTATION = 'material_peak_fields'#

Name of the post-run annotation holding the material manifest’s peak-field report.

A list with one plain mapping per population with an optmat ionisation rate: the largest field it reached against its model’s maximum field, and the energy per event there against the constant the run paid (solprop.nonlinear.optmat_rates.PeakFieldReport). Written by solprop.driver.simulation.Simulation.run() when a run closes, outside the configuration hash (SEP 3, Q13).

class solprop.output.schema.PlaneLayout(field_shape, stat_names, has_noise_phase, step_stat_names=(), retention=FieldRetention(fields='all', field_planes=()))[source]#

Bases: object

What every plane of one run has in common, fixed by the first plane saved.

Attributes:

field_shapetuple of int: (n_modes, n_pol, n_omega) of the stored field, or of the run’s state when it keeps no field; empty when neither is known.
stat_namestuple of str: The statistic names, sorted.
has_noise_phasebool: Whether a noise phase is stored beside each stored field.
step_stat_namestuple of str: The statistics the between-planes channel carries, sorted; empty for a run without a periodic subset, which is the default.
retentionFieldRetention: Which planes keep their field, resolved; every plane by default.

Examples

>>> from solprop.output.schema import PlaneLayout
>>> layout = PlaneLayout(
...     field_shape=(1, 1, 8), stat_names=("energy",), has_noise_phase=False
... )
>>> layout.field_shape, layout.stat_names, layout.retention.fields
((1, 1, 8), ('energy',), 'all')

field_shape: tuple[int, ...]#

has_noise_phase: bool#

retention: FieldRetention = FieldRetention(fields='all', field_planes=())#

stat_names: tuple[str, ...]#

step_stat_names: tuple[str, ...] = ()#

class solprop.output.schema.PlaneStore[source]#

Bases: Displayed, ABC

The shared half of an output handler: one save protocol for every backing store.

solprop.output.memory.MemoryOutput and solprop.output.hdf5.HDF5Output differ only in where the arrays go. What they must not differ in is what goes there: the sign convention of the stored field, the metadata overwrite rule, the key names and the step-history layout. Those live here, once, which is what makes the two handlers store a run bitwise identically, with one overwrite rule and one growth strategy.

The base class also holds the run’s layout – the field’s shape, the statistic names and whether a noise phase is stored – fixed by the first plane saved and checked against every later one, so that a plane that does not fit is refused by name in either handler, rather than being dropped or raising deep inside the writer.

A subclass implements four hooks: _write_metadata, _append, _read and _release, and may implement a fifth, _write_annotation, to persist the post-run annotations (annotate()) a store held only in memory otherwise.

A driver that knows how many planes the run will save says so with reserve(), and each store sizes its first allocation for them; a run that saves more grows past it, so the number is a hint, never a limit.

Examples

>>> from solprop.output import MemoryOutput
>>> from solprop.output.schema import Output, PlaneStore
>>> handler = MemoryOutput()
>>> isinstance(handler, PlaneStore), isinstance(handler, Output)
(True, True)

annotate(name, value)[source]#

Record a post-run annotation, replacing any of the same name.

The run’s record is not touched: an annotation stays outside the configuration hash, so a file annotated after its run is still the same run to solprop.output.hdf5.check_cache() and to a scan. A store may be annotated after it is closed; a file is then reopened briefly to write it.

Parameters:

namestr: A non-empty name without /, such as PEAK_FIELDS_ANNOTATION.
valueobject: Plain JSON values: mappings with string keys, lists, strings, numbers, booleans, None; anything else is refused here, the store’s boundary.

Raises:

OutputError: If the store was never opened, the name is empty or holds /, or the value is not plain JSON.

Examples

>>> from solprop.output import MemoryOutput
>>> from solprop.output.schema import _example_metadata
>>> handler = MemoryOutput()
>>> handler.open(_example_metadata())
>>> handler.annotate("note", {"peak": 1.5})
>>> handler.annotations["note"]
{'peak': 1.5}

property annotations: Mapping[str, JsonValue][source]#

Return the post-run annotations the store holds, by name.

An annotation is something learned only once the run has closed – the material manifest’s peak-field report (PEAK_FIELDS_ANNOTATION) is one – so it is not part of the run’s record: the configuration hash, a resume and a scan’s reuse never see it. A store without any, a file written before annotations existed included, answers with an empty mapping.

Returns:

collections.abc.Mapping: Each annotation’s plain JSON values, by name; a copy.

Examples

>>> from solprop.output import MemoryOutput
>>> dict(MemoryOutput().annotations)
{}

close()[source]#

Finish the run, releasing whatever the store holds open.

Calling it a second time does nothing, so a driver may close in a finally and a caller may close again without checking.

property closed: bool[source]#

Return whether the handler has been closed.

Returns:

bool: True once close() has run.

property field_shape: tuple[int, ...][source]#

Return the shape of one plane’s field, stored or not.

Returns:

tuple of int: (n_modes, n_pol, n_omega); empty before anything has said it.

fields(convention='optics')[source]#

Return every stored field, in the requested sign convention.

Parameters:

convention{‘optics’, ‘fft’}, optional: 'optics' returns the arrays exactly as stored; 'fft' converts them back to the internal convention a backend works in, so that a file stored in the FFT convention can be compared with them bin by bin.

Returns:

numpy.ndarray: Shape (n_z, n_modes, n_pol, n_omega), complex.

Raises:

OutputError: If nothing has been stored yet, or the run did not keep the field on every plane (plane_field() reads a plane that kept one).

held_bytes(n_planes, plane_bytes, *, kept=None, field_bytes=0)[source]#

Return the bytes the store keeps in memory for a run, for its admission.

A store that writes each plane out keeps none beyond the plane being written, which is what this base answer says; one that keeps the run in memory overrides it (solprop.output.memory.MemoryOutput.held_bytes()). The run’s memory admission adds the answer to the backend’s estimate (solprop.driver.memory.admit_memory()).

Parameters:

n_planesint: The planes the run will save, the entrance included.
plane_bytesint: The bytes of one plane as the store holds it: the field, the noise phase and the statistics.
keptint or None, optional: How many of the planes keep their field (FieldRetention); None, every one.
field_bytesint, optional: The part of plane_bytes a plane without its field does not hold: the field and the noise phase.

Returns:

int: Zero: the plane in flight is a transient, not a persistent allocation.

Examples

>>> import tempfile
>>> from pathlib import Path
>>> from solprop.output import HDF5Output
>>> with tempfile.TemporaryDirectory() as folder:
...     HDF5Output(Path(folder) / "run.h5").held_bytes(201, 2**20)
0

property metadata: RunMetadata | None[source]#

Return the record of the run, or None before it is opened.

Returns:

RunMetadata or None: What produced the stored data.

property n_saved: int[source]#

Return how many planes have been stored.

Returns:

int: The number of planes.

open(metadata, *, force=False)[source]#

Begin a run, recording what produced it.

Parameters:

metadataRunMetadata: The record of the run about to start.
forcebool, optional: Whether to overwrite metadata already held for a different run. Without it a second, different record raises; with it the record is replaced and the replacement logged (resolve_metadata_write()).

Raises:

OutputError: If the handler is closed, or a different record is already held and force is false.

property outcome: RunOutcome | None[source]#

Return the execution outcome, absent on legacy files.

Returns:

RunOutcome or None: The last recorded completion state and cumulative ledgers.

plane_field(index, convention='optics')[source]#

Return one saved plane’s field, in the requested sign convention.

The accessor that works whatever the run kept: a plane that kept its field returns it, and one that did not is refused by name.

Parameters:

indexint: The saved plane, the entrance as 0 and negative from the end.
convention{‘optics’, ‘fft’}, optional: As for fields().

Returns:

numpy.ndarray: Shape (n_modes, n_pol, n_omega), complex.

Raises:

OutputError: If the index is out of range, or that plane kept no field, naming the run’s fields.

Examples

>>> import numpy as np
>>> from solprop.output import MemoryOutput
>>> from solprop.output.schema import SavedPlane, _example_metadata
>>> handler = MemoryOutput()
>>> handler.open(_example_metadata())
>>> n_omega = handler.metadata.grid.n_omega
>>> handler.save(SavedPlane(z=0.0, field=np.ones((1, 1, n_omega), complex)))
>>> handler.plane_field(-1).shape
(1, 1, 2048)

record_outcome(outcome, checkpoint=None)[source]#

Persist status and optional terminal field without extending regular saves.

Parameters:

outcomeRunOutcome: Current status and cumulative call ledgers.
checkpointTerminalCheckpoint or None: Accepted terminal field, with no newly requested atomic statistics.

Raises:

OutputError: If the handler is closed, metadata is older than schema 10, or the checkpoint disagrees with the outcome or stored field layout.

reserve(n_planes)[source]#

Expect a run of n_planes saved planes in all, before the first is saved.

A store sizes its plane-indexed arrays for this many planes when the first plane fixes their shapes, rather than growing them plane by plane; a run that saves more grows them past it (by doubling), and one that saves fewer uses only what it saved, so the count is a hint and changes nothing that is stored. The run loop (solprop.driver.run.run_problem()) reserves the entrance plane and every save plane. A later call can only raise the count.

Parameters:

n_planesint: The planes the run will save, the entrance included; non-negative.

Raises:

OutputError: If n_planes is negative.

Examples

>>> from solprop.output import MemoryOutput
>>> handler = MemoryOutput()
>>> handler.reserve(201)
>>> handler.n_saved
0

retain(retention, field_shape)[source]#

Say which planes of the run keep their field, before the first is saved.

The run loop (solprop.driver.run.run_problem()) calls it with the run’s resolved rule. A store then refuses a plane that breaks the rule, sizes its field storage for the planes that keep one, and answers field reads on the others by naming the rule. A store continuing a run (a resumed file) already holds a rule and accepts only the same one.

Parameters:

retentionFieldRetention: The rule, resolved() against the run’s planes.
field_shapetuple of int: (n_modes, n_pol, n_omega) of the run’s state, which a run that keeps no field records in its place.

Raises:

OutputError: If the store already holds planes under another rule.

Examples

>>> from solprop.output import MemoryOutput
>>> from solprop.output.schema import FieldRetention
>>> handler = MemoryOutput()
>>> handler.retain(FieldRetention("none"), (1, 1, 8))
>>> handler.retention.fields, handler.field_shape
('none', (1, 1, 8))

property retention: FieldRetention[source]#

Return which planes of the run keep their field.

Returns:

FieldRetention: The rule the run declared (retain()), every plane by default.

save(plane)[source]#

Store one saved plane, converting its field to the optics convention.

Parameters:

planeSavedPlane: The position, field, statistics and step history to store. Its field is in the internal FFT convention and is written out as solprop.fourier.spectrum_to_optics() of it; a field-free plane stores none.

Raises:

OutputError: If the handler is closed or was never opened, or the plane does not fit the run’s layout: a field where the run keeps none, none where it keeps one (retain()), another shape or other statistics.

property stat_names: tuple[str, ...][source]#

Return the names of the statistics stored at every plane.

Read off the layout the first plane fixed, so that the two handlers answer alike, and a file continued or opened for reading answers from what it holds.

Returns:

tuple of str: The names, sorted; empty before the first plane is saved.

property step_stat_names: tuple[str, ...][source]#

Return the statistics the between-planes channel carries.

Empty for a run that asked for no periodic subset, and for a file written before schema version 3, which is how a reader tells whether the channel is there to prefer over the planes.

Returns:

tuple of str: The names, sorted; empty when the run has no channel.

property terminal_checkpoint: TerminalCheckpoint | None[source]#

Return the terminal checkpoint without treating it as a saved plane.

Returns:

TerminalCheckpoint or None: A field in the internal FFT convention and output amplitude units.

class solprop.output.schema.RunMetadata(*, grid, modes, simulation_kind, configuration, stepper, noise=None, backend=None, versions=<factory>, git=<factory>, created='', schema_version=25, spectral_convention='optics', field_layout='(n_z, n_modes, n_pol, n_omega)', spatial=None, modal=None, materials=<factory>, initial_z=0.0, tdse=None, quadrature=None, reference=None, _stored_launch_fields=None, _stored_step_limit_fields=None)[source]#

Bases: Displayed

Everything needed to say what produced a file, and to refuse a wrong resume.

Equality is by identity (eq=False); two runs are the same run when their configuration_hash() agree, which is what a resume actually asks.

Attributes:

gridGridRecord: The grid the run was computed on.
modestuple: One ModeRecord per propagated mode, in state order.
simulation_kind{‘mode_averaged’, ‘multimode’, ‘radial’, ‘free’, ‘slab’}: Which transform the run used.
configurationdict: The physics objects that built the run – responses, profiles, pulses, boundaries – as serialise() records them. Nothing is silently dropped: a callable is present as its repr with reproducible: False.
stepperStepperRecord: The solver settings and the schedule taken.
noiseNoiseRecord or None: The shot-noise realisation, or None for a noiseless run.
backendBackendRecord or None: What executed the run, written by the driver from the backend’s own metadata(); None for a record built before a backend existed. Outside the hash, like the versions and the provenance beside it: a run continued on the other backend is the same run.
versionsdict: Package versions, from package_versions(). Outside the hash.
gitGitProvenance or None: Provenance of the running checkout, or None. Outside the hash.
createdstr: When the run started, ISO 8601 in UTC. Outside the hash.
schema_versionint: Version of this record layout.
spectral_conventionstr: Sign convention of the stored field, always 'optics'.
field_layoutstr: Axis order of the stored field.
spatialcollections.abc.Mapping or None: Spatial geometry, axis meanings, integration measures and field units. None for modal runs and for legacy version-3 metadata. Included in the version-4 resume key.
modalcollections.abc.Mapping or None: The identity of a general modal guide: the ordered channel labels, the table, metric, response and provider identities, the sampling the reference is rebuilt from, the gauge, the frame, the active band, the noise reference basis and the accessors this basis admits, as solprop.translate.general.modal_identity() writes it. None for every other route, and for any record read back from a file older than output schema 7. Included in the schema-7 resume key, so that a run continued against a different guide, a different table or a reordered mode axis is refused by the field that changed (solprop.output.capability).
materialssolprop.physdata.selection.MaterialManifest: The run’s material manifest of output schema 8: the optmat version, the identity of every optmat model the run resolved, its species, compositions and solids, the policies in force and what was measured about them. Empty for a run of solprop’s own materials, and for any record read back from a file older than schema 8. Its identities and policies are in the schema-8 resume key (hashed()), so a run continued with another model, policy or optmat version is refused by the entry that changed.
initial_zfloat: Absolute launch position in metres, zero when absent in a historical record.
tdsecollections.abc.Mapping or None: Resolved scientific atomic descriptor. Included in the SEP 12 schema-10 lineage and in schema 12; provider execution and device facts belong to the backend record.
quadraturecollections.abc.Mapping or None: The transverse rule of a resolved modal run, as solprop.modes.quadrature.TransverseQuadrature.record() writes it: the family, the counts, the nodes and the physical weights. Output schema 17 and in its resume key (SEP 19). None for the mode-averaged and spatial routes, and for any record older than schema 17, where it means “not recorded”.
referencecollections.abc.Mapping or None: The plane-reference record of output schema 24 (SEP 31): carrier, the grid’s reference frequency in rad/s; origin, the reference of the run’s path origin (phase, delay, calibrated), inherited from a parent plane for a run launched from one; parent, that parent’s provenance or None; frame, the frame integral of solprop.translate.coupling.frame_record(), or None for an operator supplied without frame tables; and basis, the route and the basis identity (solprop.coupling.plane.basis_identity()) at each planned plane z, the launch plane first. solprop.coupling.plane.reference_at() rebuilds any saved plane’s reference from it. Derived numbers, so outside the hash; the inherited origin enters it through configuration["coupling"]. Written to JSON from schema 24 only; None for any record read back from an older file.

Examples

>>> from solprop.grid import EnvelopeGrid
>>> from solprop.output.schema import GridRecord, RunMetadata, StepperRecord
>>> from solprop.stepper import DOPRI5, PIController
>>> grid = EnvelopeGrid.from_limits(1.0, 835e-9, (400e-9, 2e-6), 2e-12)
>>> metadata = RunMetadata(
...     grid=GridRecord.from_grid(grid),
...     modes=(),
...     simulation_kind="mode_averaged",
...     configuration={"gamma": 0.11},
...     stepper=StepperRecord.from_settings(
...         DOPRI5, PIController(rtol=1e-8, atol=0.0)
...     ),
... )
>>> len(metadata.configuration_hash())
64
>>> metadata.spectral_convention
'optics'

backend: BackendRecord | None = None#

configuration: Mapping[str, object]#

configuration_hash()[source]#

Return the resume key: a digest of everything that was computed.

A SHA-256 of the canonical JSON of hashed(), so it is stable across processes – unlike hash(), which is salted per interpreter. It covers the physics and not only the statistic names and the state shape, so a resume with different physics of the same shape is refused.

Returns:

str: Sixty-four lowercase hexadecimal characters.

Examples

>>> from solprop.output.schema import _example_metadata
>>> metadata = _example_metadata()
>>> metadata.configuration_hash() == metadata.configuration_hash()
True

created: str = ''#

differences(other)[source]#

Return the dotted paths at which two records describe different runs.

This is what a refused resume names, so that the reason is the field that changed rather than “the hash does not agree”.

Parameters:

otherRunMetadata: The record to compare against.

Returns:

tuple: Dotted paths into hashed(), in depth-first order; empty when the two describe the same run.

Examples

>>> import dataclasses
>>> from solprop.output.schema import _example_metadata
>>> one = _example_metadata()
>>> two = dataclasses.replace(one, configuration={"gamma": 0.22})
>>> one.differences(two)
('configuration.gamma',)

field_layout: str = '(n_z, n_modes, n_pol, n_omega)'#

classmethod from_json(text)[source]#

Return the record a JSON document describes.

The configuration comes back as the plain structures serialise() made of it, not as the physics objects: a callable was recorded by repr and cannot be rebuilt. That is enough for the hash, which is what a resume compares.

Parameters:

textstr: A document produced by to_json().

Returns:

RunMetadata: The record, hashing identically to the one that produced text.

Raises:

OutputError: If the document is not valid JSON, or a required field is missing.

Examples

>>> from solprop.output.schema import _example_metadata
>>> metadata = _example_metadata()
>>> back = RunMetadata.from_json(metadata.to_json())
>>> back.configuration_hash() == metadata.configuration_hash()
True

git: GitProvenance | None#

grid: GridRecord#

hashed()[source]#

Return the part of the record the resume key is taken over.

Everything that describes what was computed, and nothing that describes when or with which build: the wall-clock time, the package versions and the git provenance are excluded, so that a resume survives an unrelated commit, a dirty working tree or a NumPy upgrade. modal.accessors is excluded for the same reason – it is what this build can compute, not what this run did (_hashed_modal).

Returns:

dict: The serialised schema version, grid, modes, simulation kind, configuration, noise and stepper settings, and from schema 8 the material manifest’s identities and policies.

Examples

>>> from solprop.output.schema import _example_metadata
>>> sorted(_example_metadata().hashed())[:4]
['configuration', 'grid', 'initial_z', 'materials']

initial_z: float = 0.0#

materials: MaterialManifest#

modal: Mapping[str, object] | None = None#

modes: tuple[ModeRecord, ...]#

noise: NoiseRecord | None = None#

property power_convention: str[source]#

Return the amplitude label implied by this record’s schema version.

Returns:

str: poynting for schema 6 onward, vacuum for legacy files.

quadrature: Mapping[str, object] | None = None#

reference: Mapping[str, object] | None = None#

schema_version: int = 25#

simulation_kind: Literal['mode_averaged', 'multimode', 'radial', 'free', 'slab']#

spatial: Mapping[str, object] | None = None#

spectral_convention: str = 'optics'#

stepper: StepperRecord#

tdse: Mapping[str, object] | None = None#

to_json()[source]#

Return the whole record as one JSON document.

Returns:

str: Canonical JSON carrying the hashed part plus the versions, the git provenance, the creation time and – at schema 7 and above – the whole general modal identity, whose published accessor list the hashed part leaves out.

Examples

>>> import json
>>> from solprop.output.schema import _example_metadata
>>> sorted(json.loads(_example_metadata().to_json()))[:3]
['backend', 'configuration', 'created']

property transverse_measure: Literal['area', 'per_unit_width'][source]#

Return what the run’s transverse integrals measure.

A run on modes uniform in y (solprop.modes.slab.SlabMode) records configuration["transverse_measure"] = "per_unit_width": its powers are W/m, its energies J/m and its intensities still W/m². Every other run, and every file written before the key existed, measures an area. The key sits in the free-form configuration and is written only where it is not the default, so no stored field changes meaning and the schema is unchanged.

Returns:

str: "area" or "per_unit_width".

Raises:

OutputError: If the stored value is neither.

Examples

>>> from solprop.output.schema import _example_metadata
>>> _example_metadata().transverse_measure
'area'

versions: Mapping[str, str]#

class solprop.output.schema.RunOutcome(status, z, message='', logical_calls=<factory>, atomic_calls=<factory>, execution=<factory>, previous_executions=())[source]#

Bases: object

Record completion and cumulative attempted work separately from run identity.

Attributes:

status{‘running’, ‘complete’, ‘budget_exhausted’, ‘failed’}: Current execution status. A budget stop is explicitly unfinished.
zfloat: Last accepted or saved position, m.
messagestr: Explanation of a stop or failure; empty on ordinary progress.
logical_callscollections.abc.Mapping: Cumulative logical response ledger, including startup and failed attempts.
atomic_callscollections.abc.Mapping: Cumulative provider ledger; empty when no atomic service is bound.
executioncollections.abc.Mapping: Current backend and provider facts, captured before resources close. Kept separately from the original run metadata so resumes retain their actual device, plan and diagnostic provenance without changing scientific identity.
previous_executionstuple of collections.abc.Mapping: Closing records of earlier executions of the same resumed run, in order.

atomic_calls: Mapping[str, object]#

execution: Mapping[str, object]#

logical_calls: Mapping[str, object]#

message: str = ''#

previous_executions: tuple[Mapping[str, object], ...] = ()#

status: Literal['running', 'complete', 'budget_exhausted', 'failed']#

z: float#

solprop.output.schema.SCHEMA_VERSION = 25#

Version of the record layout; incremented whenever a stored field changes meaning.

Version 9 adds the SEP 13 stepping method. Independent version-10 branches added SEP 12 launch/TDSE identities and terminal checkpoints, and SEP 14 route-dependent step limits. Version 11 on main adds interpolated and natural-step save policies. Version 12 combines both histories. Version 13 (SEP 16) lets StepperRecord.method name the Adams method’s formulation: the exponential one appears in its representation, while the default interaction formulation is left out, so a version-12 method string and its configuration hash are unchanged. Version 14 records the controller’s norm region; a version-13 record had none, and scored the requested band. Version 15 names the formulation in every Adams method string, the interaction one included, now that the exponential one is the default; a method string of an earlier version that names none was stepped in the interaction picture. Version 16 (SEP 18) records the field plasma column’s band integration rule: the configuration carries plasma_integration, its taps, band edge and policies, and the file’s root attributes repeat the rule, K, theta_b, the taps and the certified error. A run integrated with the plain trapezoid records neither, as before. Version 17 (SEP 19) records the transverse quadrature of a resolved modal run – its family, requested and realised counts, nodes and physical weights – in quadrature and its resume key, so that a run continued on another rule is refused. A record older than 17 has none, which means “not recorded”, never a field-scale rule. Version 18 (SEP 20) admits model="hacc" TDSE records, atomic windows and consistency provenance: a haCC or windowed response serialises its model and atomic_window and its descriptor is solprop.tdse-descriptor.v3, and the consistency record travels with the execution facts, outside the hash. An SAE record without a window keeps its keys, so its configuration and descriptor identities are those of version 17. Version 18 also admits an optional raw-archive reference (SEP 20 phase 2): the TDSE execution record, in backend.settings["tdse"]["archive"] and in each outcome’s execution, names the archive’s schema, run UUID and path, outside the hash; a run without an archive records none. Version 19 (SEP 24 phase 1) lets StepperRecord.method name the low-storage march, LowStorageRungeKutta(scheme='rk4ip'), which replays a schedule with no error estimate: its step history records the norm 0.0 on every step, meaning “not estimated”, never a step that met its tolerance exactly, and a record older than 19 never carries such a norm for that reason. Version 20 (SEP 22) makes the stored field optional: a run keeps it on every plane, on the planes it names or on none (FieldRetention), the file’s root attribute fields says which and a field_planes dataset lists the planes that carry one, and a field-free file has no field or noise_phase dataset. From version 20 an array input above INLINE_ARRAY_LIMIT elements is recorded by its content digest (ArrayDigest) rather than inline, and a field added to a recorded dataclass after its record existed was left out while it held its default. Version 21 records every field of a recorded dataclass, at its default too, and a scalar Raman response with all its fields: no record leaves out what an earlier record lacked, so every configuration hash moved. Loading older records preserves their original field presence and configuration hash. Version 22 (SEP 27) changes the field-plasma source to the held-current equation and records plasma_source in its configuration: the source form, certificate limit and taper sample counts. The measured full-transform steady-current certificate is execution evidence under backend.settings["plasma_source"], outside the hash. Every previous schema has a different resume key. A private gated benchmark carries _benchmark_plasma_source and refuses continuation even under this schema. Version 23 (SEP 34) measures every spatial energy, power, fluence and intensity as the axial Poynting flux of the medium at the plane: the spatial record names the flux_model and carries the run’s plane-measure coefficients (flux_plan), from which a saved plane’s admittance at its own position is rebuilt. Earlier spatial records used the vacuum measure and are refused by the processing that needs one. Version 24 (SEP 31) carries the plane-reference record (RunMetadata.reference): the antiderivatives of the frame quantities the run’s linear operator removed, the frequency origin of its linear term, the carrier, and the reference origin of its path, which a run launched from another run’s plane inherits with that parent’s identity (configuration["coupling"]). Any saved plane’s absolute phase and time reference is rebuilt from it alone. The tables are derived numbers, outside the hash; the inherited origin is in the configuration, inside it. Planes of an earlier file cannot be exchanged.

Version 25 (SEP 33) records every run’s precision in its hashed configuration, "double" included, so a resume in the other precision is refused through the hash. The saved arrays are binary64 either way.

solprop.output.schema.SPECTRAL_CONVENTION = 'optics'#: Sign convention of every spectrum a handler stores.

solprop.output.schema.STEP_STATS_INDEX: Mapping[str, type] = {'step': <class 'numpy.int64'>, 'z': <class 'numpy.float64'>}#: Dtype of each StepStats index column, beside the statistics themselves.

class solprop.output.schema.SavedPlane(*, z, field, stats=<factory>, step_history=<factory>, step_stats=<factory>, noise_phase=None)[source]#

Bases: Displayed

One saved position: the field there, its statistics and how it was reached.

The field is in the internal FFT convention, exactly as a backend produces it; a handler converts it on the way to storage. Equality is by identity (eq=False) because the field is an array. A field-free plane, of a run that keeps no field there (FieldRetention), carries field=None and no noise phase: its position, statistics and step history only.

Attributes:

zfloat: Position along the waveguide, m.
fieldnumpy.ndarray or None: Stored spectrum, shape (n_modes, n_pol, n_omega), complex, in the square root of watts, internal convention; None on a field-free plane.
statsdict: Statistics evaluated at this plane, keyed by catalogue name; the values are the arrays solprop.stats.collect() returns.
step_historyStepHistory: Every attempt the stepper made since the previous plane.
step_statsStepStats: The rows a periodic statistics subset produced since the previous plane, empty unless the run asked for one.
noise_phasenumpy.ndarray or None: Accumulated unitary phase of the noise field at this plane, shape (n_modes, n_pol, n_omega), rad, internal convention, so that processing can restore the background; None when no noise model is active, and on a field-free plane.

Examples

>>> import numpy as np
>>> from solprop.output.schema import SavedPlane, StepHistory
>>> plane = SavedPlane(
...     z=0.0,
...     field=np.ones((1, 1, 5), dtype=complex),
...     stats={"energy": np.ones((1, 1))},
...     step_history=StepHistory.empty(),
... )
>>> plane.shape
(1, 1, 5)

field: Complex[ndarray, 'n_modes n_pol n_omega'] | None#

noise_phase: Float[ndarray, 'n_modes_phase n_pol_phase n_omega_phase'] | None = None#

property shape: tuple[int, ...] | None[source]#

Return the field’s shape, or None on a field-free plane.

Returns:

tuple or None: (n_modes, n_pol, n_omega).

stats: Mapping[str, ndarray]#

step_history: StepHistory#

step_stats: StepStats#

z: float#

solprop.output.schema.SimulationKind#

Which transform the run used.

alias of Literal[‘mode_averaged’, ‘multimode’, ‘radial’, ‘free’, ‘slab’]

class solprop.output.schema.StepHistory(*, z, h, norm, accepted, wall_time)[source]#

Bases: Displayed

What the stepper did between two saved planes.

Recorded whether the run was adaptive or replayed a schedule, because the embedded error estimate costs nothing extra and a fixed run still needs it: a largest scaled norm above one means a step was too coarse for the tolerance the norm is scaled by (docs/dev/stepper.md §8).

Equality is by identity (eq=False) because every field is an array.

Attributes:

znumpy.ndarray: Position at the start of each attempt, shape (n_steps,), m.
hnumpy.ndarray: Length of each attempt, shape (n_steps,), m.
normnumpy.ndarray: Scaled error norm each attempt scored as executed, shape (n_steps,), dimensionless; the controller’s rescaled equivalent of a landing step is its own and is not recorded. The low-storage march estimates none and records 0.0, “not estimated” (output schema 19).
acceptednumpy.ndarray: Whether each attempt was accepted, shape (n_steps,), boolean.
wall_timenumpy.ndarray: Wall-clock time each attempt took, shape (n_steps,), s.

Raises:

OutputError: If the five arrays do not have equal shapes.

Examples

>>> import numpy as np
>>> from solprop.output.schema import StepHistory
>>> history = StepHistory(
...     z=np.array([0.0, 0.5]),
...     h=np.array([0.5, 0.5]),
...     norm=np.array([0.3, 0.4]),
...     accepted=np.array([True, True]),
...     wall_time=np.array([1e-3, 1e-3]),
... )
>>> history.n_steps, history.n_rejected
(2, 0)

accepted: Bool[ndarray, 'n_history_accepted']#

classmethod empty()[source]#

Return the history of a plane no step led to, such as the starting plane.

Returns:

StepHistory: Five empty arrays.

Examples

>>> from solprop.output.schema import StepHistory
>>> StepHistory.empty().n_steps
0

h: Float[ndarray, 'n_history_h']#

property n_rejected: int[source]#

Return the number of rejected attempts.

Returns:

int: How many attempts the controller threw away.

property n_steps: int[source]#

Return the number of attempts recorded.

Returns:

int: Accepted and rejected attempts together.

norm: Float[ndarray, 'n_history_norm']#

wall_time: Float[ndarray, 'n_history_wall']#

z: Float[ndarray, 'n_history_z']#

class solprop.output.schema.StepLimits(dz_min=0.0, dz_max=inf, dz_init=0.0, stretch=1.1, interval_cap=True, auto_init=False, interpolate_planes=False, step_planes=0)[source]#

Bases: object

The caps a run put on the controller’s step proposal.

Grouped into one record because they travel together and because a constructor that took them one by one would need more arguments than the project allows.

Attributes:

dz_minfloat: Smallest step the controller was allowed to propose, m. Zero when the driver left it to solprop.stepper.controller.default_dz_min().
dz_maxfloat: Largest step, m; infinite when uncapped.
dz_initfloat: First step, m; zero when the driver chose it.
stretchfloat: Landing stretch factor, dimensionless; 1 switches landing stretch off.
interval_capbool: Whether a step was capped at its operator-table interval’s width (output schema 10; every earlier run was).
auto_initbool: Whether the start-up rule took the step after the first from the first step’s estimate (output schema 10; no earlier run did).
interpolate_planesbool: Whether the save planes but the end and the hard kinks were interpolated inside accepted steps (output schema 11); False when every plane was landed on, as in every earlier run.
step_planesint: A positive k when the end of every k-th accepted step was saved too (output schema 11); 0 for the requested planes only.

Examples

>>> from solprop.output.schema import StepLimits
>>> StepLimits(dz_max=0.01).dz_max
0.01

auto_init: bool = False#

dz_init: float = 0.0#

dz_max: float = inf#

dz_min: float = 0.0#

interpolate_planes: bool = False#

interval_cap: bool = True#

step_planes: int = 0#

stretch: float = 1.1#

class solprop.output.schema.StepStats(*, z, step, values=<factory>)[source]#

Bases: Displayed

The statistics a backend evaluated between two saved planes.

A run whose solprop.stats.StatsSpec names a periodic subset gets one row every period accepted steps, evaluated inside the backend’s step loop and flushed here at each plane. Empty for a run that asked for none, which is the default and what every file written before schema version 3 holds.

Stored the way the step history is – flat columns concatenated over planes with a per-plane counts beside them – so that a resume appends and a scan needs no NaN padding, which growing datasets would force on it.

Equality is by identity (eq=False) because every field is an array.

Attributes:

znumpy.ndarray: Position of each row, m, float64, shape (n_rows,).
stepnumpy.ndarray: Index of the accepted step each row was taken after, into this plane’s StepHistory, int64, shape (n_rows,). The history counts rejected attempts, so this is not row * period.
valuesdict: The statistics, keyed by catalogue name, each shaped (n_rows, *shape) with the statistic’s own shape after the row axis – the batch axis a backend carries is dropped here, as it is for the plane statistics.

Raises:

OutputError: If the columns do not all have the same number of rows.

Examples

>>> import numpy as np
>>> from solprop.output.schema import StepStats
>>> rows = StepStats(z=np.array([0.1, 0.2]), step=np.array([0, 2]),
...                  values={"energy": np.ones((2, 1, 1))})
>>> rows.n_rows, rows.names
(2, ('energy',))

classmethod empty()[source]#

Return the channel a run without a periodic subset carries.

Returns:

StepStats: No rows and no statistics.

property n_rows: int[source]#

Return how many rows belong to this plane.

Returns:

int: The row count.

property names: tuple[str, ...][source]#

Return the statistics the channel carries, sorted.

Returns:

tuple of str: The catalogue names.

step: Int[ndarray, 'n_rows_step']#

values: Mapping[str, ndarray]#

z: Float[ndarray, 'n_rows_z']#

class solprop.output.schema.StepperRecord(*, tableau, controller, limits=<factory>, schedule=None, method=None)[source]#

Bases: object

The solver settings a run used, including the schedule it actually took.

Equality is by identity (eq=False) because schedule is an array. Schedules are first class and re-loadable (see docs/dev/decisions.md): a replayed run is bit-identical, and every ensemble member then carries the same numerical error as a common systematic.

Build one from the objects a run held with from_settings(), and from a stored document with from_record().

Attributes:

tableaustr: Name of the Runge-Kutta scheme.
methodstr or None: The integrator that stepped the run instead of the tableau, as its repr: a low-call Adams method (SEP 13, recorded from output schema 9) or the low-storage march (SEP 24, from schema 19); None for the Runge-Kutta march.
controllerdict: The step controller as serialise() records it: tolerances, safety and gain factors, lag and the norm weights.
limitsStepLimits: The caps on the controller’s proposal.
schedulenumpy.ndarray or None: Step lengths of shape (n_steps,), m, when a fixed schedule was replayed or recorded; None for a purely adaptive run.

Examples

>>> import numpy as np
>>> from solprop.output.schema import StepLimits, StepperRecord
>>> from solprop.stepper import DOPRI5, FixedSchedule, PIController
>>> record = StepperRecord.from_settings(
...     DOPRI5,
...     PIController(rtol=1e-8, atol=0.0),
...     schedule=FixedSchedule(np.array([0.5, 0.5])),
...     limits=StepLimits(dz_max=0.1),
... )
>>> record.tableau, record.limits.dz_max
('DOPRI5', 0.1)

controller: Mapping[str, JsonValue]#

classmethod from_record(document)[source]#

Return the record a stored document describes.

Parameters:

documentcollections.abc.Mapping: The form serialise() produced from a record.

Returns:

StepperRecord: The record.

Examples

>>> from solprop.output.schema import StepperRecord, serialise
>>> from solprop.stepper import TSIT5, PIController
>>> one = StepperRecord.from_settings(TSIT5, PIController(rtol=1e-6, atol=0.0))
>>> StepperRecord.from_record(serialise(one)).tableau
'Tsit5'

classmethod from_settings(tableau, controller, *, schedule=None, limits=None, method=None)[source]#

Return the record of the objects a run stepped with.

Parameters:

tableausolprop.stepper.tableau.Tableau: The scheme.
controllersolprop.stepper.controller.PIController: The step controller, which carries the tolerances and the norm choice.
schedulesolprop.stepper.schedule.FixedSchedule, optional: The fixed schedule replayed, if any.
limitsStepLimits, optional: The caps on the controller’s proposal; the defaults when omitted.
methodstr or None, optional: The Adams method that stepped the run instead of the tableau, as its repr; None for the Runge-Kutta march.

Returns:

StepperRecord: The record.

Examples

>>> from solprop.output.schema import StepperRecord
>>> from solprop.stepper import TSIT5, PIController
>>> record = StepperRecord.from_settings(
...     TSIT5, PIController(rtol=1e-6, atol=0.0)
... )
>>> record.tableau, record.schedule is None
('Tsit5', True)

limits: StepLimits#

method: str | None = None#

schedule: Float[ndarray, 'n_steps'] | None = None#

tableau: str#

class solprop.output.schema.TerminalCheckpoint(plane, next_save_index, landed=True)[source]#

Bases: object

Keep an accepted field between scheduled saves without adding an observation.

Attributes:

planeSavedPlane: Accepted field in the internal FFT convention and the output amplitude convention, with already available history and periodic statistics. Creating a checkpoint never requests fresh material statistics. It always carries the field, also in a run that keeps none at its planes, so that a stopped run resumes (SEP 22, Q1).
next_save_indexint: Index of the first scheduled save strictly beyond this field.
landedbool: Whether the field is an accepted endpoint rather than a dense observation.

landed: bool = True#

next_save_index: int#

plane: SavedPlane#

solprop.output.schema.field_retention_refusal(fields, field_planes)[source]#

Say what is wrong with a field-retention request, or that nothing is.

The one validator behind solprop.driver.simulation.SimulationSettings, solprop.driver.run.run_problem(), solprop.driver.scan.run_scan() and FieldRetention; each door raises its own layer’s error with the message.

Parameters:

fieldsobject: The request, one of "none", "planes" and "all".
field_planesobject: Saved-plane indices, the entrance as 0 and negative from the end; required exactly when fields is "planes".

Returns:

str or None: Why the request is refused, or None when it is valid.

Examples

>>> from solprop.output.schema import field_retention_refusal
>>> field_retention_refusal("planes", [0, -1]) is None
True
>>> field_retention_refusal("planes", [])
"field_planes is required exactly when fields='planes'"

solprop.output.schema.package_versions()[source]#

Return the versions of solprop and of the packages a stored file depends on.

Recorded so that a file can be read years later with the same numerics in view. The versions are deliberately outside the configuration hash: upgrading NumPy must not refuse to resume a run. optmat’s version is here too, for a reader; a run that takes a material from optmat carries it again in its material manifest (RunMetadata.materials), where it is part of the key.

Returns:

dict: Version strings keyed by 'solprop', 'numpy', 'scipy', 'h5py', 'optmat' (its full version string) and 'python'.

Examples

>>> from solprop.output.schema import package_versions
>>> sorted(package_versions())
['h5py', 'numpy', 'optmat', 'python', 'scipy', 'solprop']

solprop.output.schema.recorded_input(value)[source]#

Return an input as the run record carries it: inline, or by its digest.

Parameters:

valueobject: A pulse, a spatial pulse or an array input.

Returns:

object: value itself, unless it is an array of more than INLINE_ARRAY_LIMIT elements, which is recorded as its ArrayDigest.

Examples

>>> import numpy as np
>>> from solprop.output.schema import recorded_input
>>> recorded_input(np.zeros(4)).shape
(4,)
>>> type(recorded_input(np.zeros(2**20 + 1))).__name__
'ArrayDigest'

solprop.output.schema.resolve_metadata_write(existing, new, *, force=False)[source]#

Decide whether a handler may write metadata, under one rule for both handlers.

Metadata is written once. A second write describing the same run is a no-op; one describing a different run is a mistake and raises, naming the fields that changed, unless the caller insists with force, in which case the record is replaced and the replacement is logged.

“The same run” means the same RunMetadata.configuration_hash(), so a rerun of identical physics under a newer NumPy, or from a checkout that has since gained a commit, is still the same run.

Parameters:

existingRunMetadata or None: What the handler already holds, or None when it holds nothing.
newRunMetadata: What the caller is offering.
forcebool, optional: Whether to overwrite a different record instead of refusing.

Returns:

bool: Whether the caller should write new.

Raises:

OutputError: If a different record is already held and force is false.

Examples

>>> from solprop.output.schema import _example_metadata, resolve_metadata_write
>>> metadata = _example_metadata()
>>> resolve_metadata_write(None, metadata)
True
>>> resolve_metadata_write(metadata, metadata)
False

solprop.output.schema.serialise(value)[source]#

Return any configuration object as a JSON-safe structure, dropping nothing.

Dataclasses become tagged dictionaries of their constructor fields (less a live handle whose field metadata says record=False), enumerations their qualified name and value, NumPy arrays a nested list with their dtype and shape, cubic splines their knots, coefficients, axis and extrapolation setting, complex numbers a real/imaginary pair, and mappings and sequences are recursed. Anything that cannot be reduced – a function, a lambda, an object with no structure – is recorded as its repr and marked reproducible: False, because a configuration that is silently incomplete is worse than one that says where it stops.

The function is idempotent: applying it to its own output returns that output, so a round-tripped RunMetadata hashes exactly as the original does.

Parameters:

valueobject: Anything at all.

Returns:

NoneType or bool or int or float or str or list or dict: A structure of JSON primitives, lists and string-keyed dictionaries.

Examples

>>> import numpy as np
>>> from solprop.output.schema import serialise
>>> serialise({"radius": 125e-6})
{'radius': 0.000125}
>>> serialise(np.array([1.0, 2.0]))["dtype"]
'float64'
>>> serialise(abs)["reproducible"]
False