solprop.driver.executors.queue#

Drain durable point work across controllers sharing a filesystem.

Functions

collect(path, output)

Idempotently collect validated artifacts through one locked HDF5 writer.

Classes

Queue(path[, stale_age, pool, chunk])

Drain a shared durable queue using a local persistent process pool.

class solprop.driver.executors.queue.Queue(path, stale_age=600.0, pool=<factory>, chunk=None)[source]#

Drain a shared durable queue using a local persistent process pool.

Attributes:

pathpathlib.Path, str or None: Queue directory. None derives it from the collected output path for the CLI.
stale_agefloat: Positive remote lease duration, s. Live local owners remain protected.
poolPool: Resource policy for workers within this controller.
chunktuple of int or None: Optional zero-based round-robin chunk number and count.

prepare(request)[source]#

Create or validate the manifest under an automatically released lock.

Parameters:

requestScanRequest: Scan identity, retention settings and worker to bind before dispatch.

run(points, work)[source]#

Run claimed points while a separate thread refreshes their leases.

Parameters:

pointsIterable[T]: Lazily supplied point descriptions retaining global indices.
workCallable[[T], R]: Pickleable point worker with its memory probe.

Yields:

validate(work)[source]#

Validate serialisation before preparing the durable queue.

Parameters:

solprop.driver.executors.queue.collect(path, output)[source]#

Idempotently collect validated artifacts through one locked HDF5 writer.

Parameters:

pathpathlib.Path or str: Queue directory containing its manifest and published point artifacts.
outputpathlib.Path or str: Collected HDF5 destination.

Returns:

Raises: