solprop.driver.executors.slurm

Emit and submit quoted Slurm array jobs with explicit controller resources.

Classes

Slurm(script[, ntasks, cpus_per_task, mem, ...])

Submit an array of queue or chunk controllers without changing their backend.

class solprop.driver.executors.slurm.Slurm(script, ntasks=1, cpus_per_task=1, mem='4G', time='01:00:00', partition=None, gres=None, mode='queue', instances=0, queue_path=None, workers=None, threads=1, fresh=False, workdir=None)

Bases: object

Submit an array of queue or chunk controllers without changing their backend.

Attributes:

scriptpathlib.Path or str

Python scan script with its ordinary main guard.

ntasksint

Positive array task count; Slurm task IDs run from one through ntasks.

cpus_per_taskint

Positive allocated CPUs per controller task.

memstr

Positive Slurm memory allocation per task; no suffix means MiB.

timestr

Slurm wall-clock limit, for example 01:00:00.

partitionstr or None

Optional scheduler partition.

gresstr or None

Optional generic resources, for example gpu:h200:2.

mode{‘queue’, ‘chunk’}

Shared claims or disjoint global round-robin subsets.

instancesint

Zero means one controller; positive values split each task into instances.

queue_pathpathlib.Path, str or None

Shared queue directory, derived from the output when unspecified.

workersint or None

Maximum local worker processes per instance.

threadsint

Threads per local worker, positive.

freshbool

Whether every point uses a fresh process.

workdirpathlib.Path, str or None

Shared working directory; defaults to the submitting process’s directory.

job_idstr or None

Scheduler identifier after a successful submission.

cpus_per_task: int = 1

fresh: bool = False

gres: str | None = None

instances: int = 0

job_id: str | None = None

mem: str = '4G'

mode: Literal['queue', 'chunk'] = 'queue'

ntasks: int = 1

partition: str | None = None

prepare(request)

Bind the submission or task to the same durable manifest.

Parameters:

requestScanRequest

Scan identity, retention settings and worker to bind before dispatch.

queue_path: Path | str | None = None

render()

Return a Bash array script with quoted paths and explicit resource limits.

Returns:

str

Complete script suitable for sbatch or a local task-body test.

run(points, work)

Submit once, or drain this array task’s prepared inner queue.

Parameters:

pointsIterable[T]

Lazily supplied point descriptions retaining global indices.

workCallable[[T], R]

Pickleable point worker with its memory probe.

Yields:

object

Completed result from the prepared execution policy.

script: Path | str

submit(destination=None)

Write the script and submit it with sbatch.

Parameters:

destinationpathlib.Path, str or None, optional

Generated script path; defaults to submission.slurm in the queue.

Returns:

str

Scheduler job identifier returned by sbatch –parsable.

Raises:

DriverError

If sbatch is unavailable or refuses the submission.

threads: int = 1

time: str = '01:00:00'

validate(work)

Refuse non-serialisable work before submitting a job.

Parameters:

workCallable

Pickleable point worker with its memory probe.

workdir: Path | str | None = None

workers: int | None = None