solprop.driver.run#

The outer loop of a run: save planes in, saved planes out.

The inner loop – many adaptive or replayed steps between two save planes – lives inside a backend and is compiled; this is the Python loop around it, which runs a few hundred times at most (docs/dev/architecture.md section 5). It asks the backend to land on each save plane in turn, takes the observation and the statistics there, and hands them to an output handler.

What this module does not do. It never conjugates a spectrum and never touches the state between planes: the backend owns the step loop and the output handler owns the sign convention on disk (docs/dev/conventions.md section 2, boundary place 2). A plane is carried across as a solprop.output.schema.SavedPlane in the internal FFT convention, with modal amplitudes converted to true-power units.

Progress is reported through solprop.log.ProgressReporter – one call at the start, one per attempted step from inside the backend, one per save plane and one at the end. The protocol and the default solprop.log.LogReporter live in solprop.log, and are re-exported here because this is where a run reaches them; nothing is printed, and solprop.log.configure_logging() is how a script sees it (docs/dev/architecture.md section 14).

The manual page is docs/guide/setup/running.md.

Module Attributes

MARCH_SNAPSHOT_AUTO_BYTES

The largest Adams march snapshot a run keeps when resume_history is None.

Functions

`check_snapshot_request`(problem, output, ...)	Refuse unsupported snapshot policies before allocating native resources.
`field_retention`(problem, fields, field_planes)	Return a run's field retention, resolved against its saved planes.
`open_run_record`(output, metadata, backend, ...)	Open an output with the run's record and execution facts, and save nothing.
`plane_from_observation`(observation[, stats, ...])	Turn one observation and its statistics into the plane an output handler stores.
`run_problem`(problem, backend, output, *, ...)	Run one translated problem to its save planes, writing each one out.

class solprop.driver.run.LogReporter(*, period=10.0, logger=None, clock=<built-in function perf_counter>, n_steps=None, level=20, estimated=True)#

Bases: object

Report progress as log records: a periodic line, one per plane, one at the end.

Three kinds of line, all at level (INFO by default) on this module’s logger. A periodic line, at most one per period seconds however fast the steps come, says where the run is and when it expects to finish. A plane line says what the last save plane cost. The closing line says what the run achieved against what it asked for, and how far off its own ETAs turned out to be. Nothing is printed; call solprop.log.configure_logging() to see the lines.

The ETA is from the recent rate, and it is measured. Two estimates are formed at every periodic line: the whole-run one, remaining z / (z done / elapsed), and one from the rate of z over a trailing window of max(5 periods, 10 % of the elapsed time). The windowed one is reported, because the whole-run one is linear in z from the first step and most runs are not: the steps are short while a pulse compresses and long after it, a gradient fill changes the operator and the plasma cost along z, and a plane-dense run pays its planes unevenly. When a fixed schedule is replayed the total step count is known, so the estimate is the remaining steps at the recent cost per step. Both estimates’ median absolute relative errors are kept, and finished() logs the reported one, so the estimator is measured on every run rather than assumed.

Parameters:

periodfloat, optional: Seconds between periodic lines, s. 0 disables them and leaves the plane and closing lines.
loggerlogging.Logger or None, optional: Logger to use; defaults to this module’s.
clockcallable, optional: Returns a monotonic time in seconds. Injected so that the period and the ETA can be tested without waiting.
n_stepsint or None, optional: Total steps of a replayed fixed schedule, which turns the ETA into a remaining-steps estimate. None for an adaptive run.
levelint, optional: Level the three lines are logged at.
estimatedbool, optional: Whether the steps carry an error estimate. False for the low-storage march, whose 0.0 norms mean “not estimated”: the closing line then prints solprop.log.NOT_ESTIMATED in place of the tolerance ratio.

Examples

>>> import logging
>>> from types import SimpleNamespace
>>> reporter = LogReporter(level=logging.DEBUG)
>>> reporter.plane(0.5, SimpleNamespace(n_steps=40, n_rejected=1, max_norm=0.7,
...                                     wall_time=0.03))
>>> reporter.finished(0.06)
>>> reporter.n_steps, reporter.tolerance_ratio
(40, 0.7)

property eta_error: float#

Return how far off the reported ETAs were.

Returns:

float: Median absolute relative error of the windowed estimate over every periodic line that published one; NaN before finished(), or when no line did.

property eta_error_whole_run: float#

Return how far off the whole-run estimator would have been.

Formed on the same lines as eta_error but never reported, so that the two can be compared on every run.

Returns:

float: Median absolute relative error of the whole-run estimate; NaN as for eta_error.

finished(total)#

Log the closing line, with the ETA error the run’s own estimates made.

Parameters:

totalfloat: See ProgressReporter.finished().

interrupted(z, error)#

Log one warning saying where the run stopped and what stopped it.

Parameters:

zfloat: See solprop.log.InterruptibleReporter.interrupted().
errorBaseException: See solprop.log.InterruptibleReporter.interrupted().

property n_rejected: int#

Return the rejected steps of the run so far.

Returns:

int: Counted from whichever channel is live; see n_steps.

property n_steps: int#

Return the attempted steps of the run so far.

Returns:

int: The larger of the two channels that count: the per-step calls when a driver is attached, and the plane summaries otherwise. Both count every attempted step, so when both are live they agree.

plane(z, summary)#

Log one line for a save plane.

Parameters:

zfloat: See ProgressReporter.plane().
summaryPlaneSummary: See ProgressReporter.plane().

started(z_start, z_end, n_planes)#

Take the run’s extent and start the clock.

Emits nothing: the promise is one periodic line per period, and a run shorter than its period says nothing until its first plane.

Parameters:

z_start, z_endfloat: See ProgressReporter.started().
n_planesint: See ProgressReporter.started().

step(z, dz, accepted)#

Count one attempted step and emit a periodic line if one is due.

Parameters:

z, dzfloat: See ProgressReporter.step().
acceptedbool: See ProgressReporter.step().

property tolerance_ratio: float#

Return what the run achieved against what it asked for.

Returns:

float: The largest scaled error norm any accepted step scored (solprop.stepper.controller.ControlRecord.tolerance_ratio); 0 before the first plane.

solprop.driver.run.MARCH_SNAPSHOT_AUTO_BYTES = 268435456#

The largest Adams march snapshot a run keeps when resume_history is None.

The snapshot – the state and the history’s rows, a few to sixty copies of the state – is rewritten at every save plane; a state too large for that to be cheap keeps none unless asked to, and its resume starts the history afresh.

class solprop.driver.run.ProgressReporter(*args, **kwargs)#

Bases: Protocol

Receive progress events from the propagation driver.

Four calls, in this order: started() once, step() once per attempted step including the rejected ones, plane() once per save plane, and finished() once at the end. A reporter may also implement the fifth call of solprop.log.InterruptibleReporter, which the driver makes instead of finished() when the loop raises, KeyboardInterrupt included, so that a reporter holding a terminal can give it back; a reporter without it is still complete. The driver holds no state on a reporter’s behalf and never inspects what it returns.

step() is on the hot path – one attribute lookup and one call inside the step loop – so a reporter that does real work there must do it on a wall-clock period, not on every call. Nothing a reporter does may change the propagation: a run with a reporter is bitwise the run without one.

Subclass solprop.log.NullReporter to implement only the calls you care about.

Examples

>>> isinstance(NullReporter(), ProgressReporter)
True

finished(total)#

Record that the run ended.

Parameters:

totalfloat: Wall time of the whole run, s.

plane(z, summary)#

Record that a save plane was reached.

Parameters:

zfloat: Position of the plane, m.
summaryPlaneSummary: What the call to the plane cost and achieved.

started(z_start, z_end, n_planes)#

Record that a run has begun.

Parameters:

z_startfloat: Position the run starts from, m; non-zero on a resume.
z_endfloat: Position the run ends at, m.
n_planesint: Save planes the run will write.

step(z, dz, accepted)#

Record one attempted step.

Parameters:

zfloat: Position at the end of the attempt, m. On a rejected step this is where the attempt would have landed, which is ahead of the run’s true position.
dzfloat: Length of the attempt, m.
acceptedbool: Whether the step was kept.

solprop.driver.run.check_snapshot_request(problem, output, snapshots, resume_history, fields)[source]#

Refuse unsupported snapshot policies before allocating native resources.

No backend is refused by name: every backend’s fused and exponential Adams engines, the native backend’s resident one included, export their history (SEP 29 section 6.6).

Parameters:

problemKernelProblem: Translated run and observation policy.
outputOutput: Requested optical writer.
snapshotsSnapshotController or None: Optional snapshot request controller.
resume_historybool or None: Whether the caller explicitly disabled Adams history retention.
fieldsFieldsChoice: Saved-field retention policy.

Raises:

DriverError: If snapshots are requested outside their supported scope.

solprop.driver.run.field_retention(problem, fields, field_planes)[source]#

Return a run’s field retention, resolved against its saved planes.

The driver’s door to solprop.output.schema.field_retention_refusal(), the validator SimulationSettings and run_scan() share.

Parameters:

problemsolprop.translate.problem.KernelProblem: The translated problem: its save planes, and the entrance, are the planes.
fieldsobject: "all", "planes" or "none".
field_planescollections.abc.Sequence of int: Saved-plane indices for "planes", negative from the end.

Returns:

solprop.output.schema.FieldRetention: The rule, its indices counted from the entrance.

Raises:

DriverError: If the request is not valid, names a plane outside the run, asks for fields='planes' while step_planes saves an unknown number of planes.

Examples

>>> from solprop.driver import Simulation
>>> from solprop.driver.run import field_retention
>>> simulation = Simulation.from_gnlse(
...     0.11, 0.01, [0.0, 0.0, -1.183e-26], lambda0=835e-9,
...     wavelength_limits=(500e-9, 2e-6), trange=1e-12,
...     width=100e-15, peak_power=9.8, pulseshape="sech", n_save=3,
... )
>>> field_retention(simulation.translate(), "planes", [-1]).field_planes
(2,)

solprop.driver.run.open_run_record(output, metadata, backend, binding, memory)[source]#

Open an output with the run’s record and execution facts, and save nothing.

Used when a run is refused after its resources were prepared and before its first step, such as a failed haCC consistency check (SEP 20 section 5.4): the record, with the binding’s provenance, is written without a plane or a terminal checkpoint, so it cannot be mistaken for a propagation state.

Parameters:

outputOutput: Handler to open.
metadataRunMetadata: Scientific description of the run.
backendBackend: Prepared optical backend.
bindingTDSEBinding or None: Optional atomic service supplying execution provenance.
memoryMemoryAdmission or None: Optional run admission record.

solprop.driver.run.plane_from_observation(observation, stats=None, *, member=0, field=True)[source]#

Turn one observation and its statistics into the plane an output handler stores.

The only shape change on the way: the seam’s arrays carry a leading batch axis and a stored plane does not, so one member is taken from the field, the noise phase and every statistic (docs/dev/conventions.md section 6). Nothing is conjugated – the field stays in the internal FFT convention, and the handler converts it on the way to storage (docs/dev/conventions.md section 2). Nothing is rescaled either: the field is whatever amplitude the observation carries, the backend’s vacuum-normalised one unless the caller has applied the interface conversion to true-power amplitudes (_power_observation; see docs/dev/decisions.md).

Parameters:

observationsolprop.backends.protocol.Observation

The state at a save plane, from solprop.backends.protocol.Backend.observe().

statscollections.abc.Mapping, optional

What solprop.backends.protocol.Backend.stats() returned there, keyed by statistic name, each with a leading batch axis. Empty when omitted.

The observation’s between-planes rows are taken across the same way: their batch axis is the one after the row axis, so the member is taken from axis one rather than axis zero.

memberint, optional

Which batch member to store; 0, the only member of an unbatched run, by default. solprop.driver.batch.split_observation() stores each member of a lockstep batch in turn.

fieldbool, optional

Whether the plane keeps the field and its noise phase; False, or an observation that carries no field, builds a field-free plane (SEP 22), which copies neither.

Returns:

solprop.output.schema.SavedPlane: The position, the field, the statistics and the step history.

Examples

>>> import numpy as np
>>> from solprop.backends.protocol import Observation, StepHistory
>>> from solprop.driver.run import plane_from_observation
>>> history = StepHistory(np.array([0.5]), np.array([0.5]), np.array([0.3]),
...                       np.array([True]), np.array([0.02]))
>>> observation = Observation(0.5, np.ones((1, 1, 1, 4), complex), None, history,
...                           1, 0)
>>> plane = plane_from_observation(observation, {"energy": np.ones((1, 1, 1))})
>>> plane.z, plane.shape, plane.stats["energy"].shape
(0.5, (1, 1, 4), (1, 1))

solprop.driver.run.run_problem(problem, backend, output, *, metadata, stats_spec=None, progress=None, progress_period=10.0, start_plane=0, memory=None, tdse_binding=None, resume_history=None, fields='all', field_planes=(), snapshots=None, tdse_archive=None)[source]#

Run one translated problem to its save planes, writing each one out.

The loop of docs/dev/architecture.md section 5, usable without a solprop.driver.simulation.Simulation. The backend must already hold its initial state (solprop.backends.protocol.Backend.set_initial()); the handler is opened here, saved to plane by plane, and closed in a finally so that an interrupted run leaves a readable file.

The plane at problem.initial.z is saved first, with an empty step history, and then each save plane in turn: propagate, observe, collect statistics, save. Steps land on the planes exactly, so a saved field is an exact step endpoint and there is no interpolation (docs/dev/stepper.md section 3), unless the problem asks for dense output. With stepper.step_planes the march also stops at the end of every step_planes-th accepted step, which is saved too before the same plane is asked for again. A plane whose propagation did not meet its tolerance is logged as a warning naming that plane (docs/dev/stepper.md section 8).

Parameters:

problemsolprop.translate.problem.KernelProblem: The translated problem, whose stepper.save_planes is the itinerary.
backendsolprop.backends.protocol.Backend: The backend stepping it, already loaded with its initial state.
outputsolprop.output.schema.Output: Where the planes go. Closed when the loop ends, whether or not it succeeded.
metadatasolprop.output.schema.RunMetadata: The record of the run, written when the handler opens. solprop.driver.simulation.metadata_for() builds one.
stats_specsolprop.stats.StatsSpec, optional: Which statistics to collect at each plane. Defaults to the contract’s own problem.stats_spec (section 8 of docs/dev/kernel-contract.md).
progressProgressReporter, {‘auto’, ‘bar’, ‘log’, ‘none’} or None, optional: Where progress goes. None takes the process default (solprop.log.default_progress(), "auto" unless solprop.log.configure_logging() was told otherwise): a solprop.log.BarReporter drawing one redrawing line on standard error when that is a terminal or the process is a Jupyter kernel, and a solprop.log.LogReporter – a periodic line, one line per plane and one at the end, all at INFO – otherwise, so redirected, piped and captured output gets lines and never a bar. The four names force a choice; a reporter is used as given.
progress_periodfloat, optional: Seconds between the line reporter’s periodic lines, s; 0 leaves only the plane and closing lines. A bar has its own redraw period and a given reporter its own business, so it is ignored for both.
start_planeint, optional: Index of the first save plane strictly beyond the loaded state. A resume loads either the last regular plane or a separate terminal checkpoint at its actual absolute position. The launch plane is written when the handler holds no regular observations, so a resumed run never stores it twice.
memorysolprop.driver.memory.MemoryAdmission or None, optional: The run’s memory admission (solprop.driver.memory.admit_memory()), recorded as memory in the backend record’s settings; None records none.
tdse_bindingsolprop.translate.tdse.TDSEBinding or None, optional: Explicit live atomic runtime owned by the caller. Its ledgers and execution facts are captured before the caller closes it. TDSE requires an output supporting the separate checkpoint capability.
resume_historybool or None, optional: Whether an output that can keep it (save_march) keeps the Adams march’s state at every save plane and terminal checkpoint, so that a resume continues the history instead of starting it afresh. None keeps it while it is at most MARCH_SNAPSHOT_AUTO_BYTES.
fields{‘all’, ‘planes’, ‘none’}, optional: Which saved planes keep their field (SEP 22): every one (the default), those of field_planes, or none. A plane that keeps none stores its position, its statistics and its step history, and is not copied, converted or stored; the march is the same either way. A terminal checkpoint, written when the run stops, carries the field whatever this says, so a stopped run resumes.
field_planessequence of int, optional: Saved-plane indices for fields='planes', the entrance as 0 and negative from the end.
snapshotssolprop.output.SnapshotController or None, optional: Requests immutable disk copies at committed accepted steps. Requires HDF5, step_planes=1, all fields and non-interpolated observations.
tdse_archivesolprop.output.tdse_archive.TDSEArchiveWriter or None, optional: Live archive to suspend and copy beside the optical writer.

Raises:

DriverError: If the statistics given here disagree with the problem’s own about the between-planes channel; if the problem carries a batch axis, which the output schema has no room for yet; if the record and the problem disagree about whether this is a general modal run; if start_plane is not a plane of this problem; or if the field retention is not valid, names a plane outside the run, asks for fields='planes' beside step_planes (whose plane count is not known in advance), or is not 'all' for an output that cannot store field-free planes.

Examples

>>> from solprop.backends.registry import select
>>> from solprop.driver.run import run_problem
>>> from solprop.driver.simulation import metadata_for
>>> from solprop.grid import EnvelopeGrid
>>> from solprop.nonlinear.gnlse import problem_from
>>> from solprop.output import MemoryOutput
>>> from solprop.pulses import SechPulse
>>> from solprop.stepper import PIController
>>> from solprop.translate.problem import assemble
>>> grid = EnvelopeGrid.from_limits(0.01, 835e-9, (500e-9, 2e-6), 1e-12)
>>> gnlse = problem_from([0.0, 0.0, -1.183e-26], omega_ref=grid.omega0,
...                      gamma=0.11, shock=False)
>>> start = SechPulse(wavelength=835e-9, width=100e-15, peak_power=9.8).build(grid)
>>> problem = assemble(grid, gnlse.mode, [gnlse.kerr], start,
...                    controller=PIController(rtol=1e-6, atol=1e-16),
...                    save_planes=[0.005, 0.01], dz_max=0.01, dz_init=1e-3,
...                    density=1.0, shock=False, prefactor_index=1.0)
>>> backend = select("numpy_ref", problem, grid=grid, modes=[gnlse.mode])
>>> backend.set_initial(problem.initial.field0, None)
>>> handler = MemoryOutput()
>>> run_problem(problem, backend, handler,
...             metadata=metadata_for(problem, grid, modes=[gnlse.mode]))
>>> backend.close()
>>> handler.n_saved, handler["z"].tolist()
(3, [0.0, 0.005, 0.01])