solprop.driver.executors.pool

Run bounded spawned workers with explicit threads and memory admission control.

Functions

peak_rss()

Return this process's peak resident memory in bytes.

Classes

MemoryBudget(limit, controller, per_worker, ...)	Record the admission calculation without treating it as a hard RSS limit.
Pool([workers, memory_limit, threads, pin, ...])	Execute independent points in memory-sized spawned processes.

class solprop.driver.executors.pool.MemoryBudget(limit, controller, per_worker, cores, threads)

Bases: object

Record the admission calculation without treating it as a hard RSS limit.

Attributes:

limitint

Effective memory allocation, bytes, positive.

controllerint

Controller resident memory and output allowance, bytes, non-negative.

per_workerint

Worker estimate, bytes, positive; backend RSS is counted once.

coresint

CPUs available to this controller, positive.

threadsint

Threads per worker, positive.

controller: int

cores: int

limit: int

per_worker: int

threads: int

workers(requested=None)

Choose a count within both CPU and memory allocations.

Parameters:

requestedint or None, optional

Optional upper bound on workers, positive.

Returns:

int

Admitted workers after reserving 20 percent of memory.

Raises:

DriverError

If inputs are invalid or one worker cannot fit.

class solprop.driver.executors.pool.Pool(workers=None, memory_limit=None, threads=1, pin=True, fresh=False)

Bases: object

Execute independent points in memory-sized spawned processes.

Attributes:

workersint or None

Positive maximum workers; None uses the CPU and memory admission rule.

memory_limitint or None

Positive memory cap, bytes; None reads the allocation and physical memory.

threadsint

Threads per worker, positive and one by default.

pinbool

Pin workers within Linux CPU affinity; unavailable on macOS.

freshbool

Start a fresh interpreter for every point instead of persistent workers.

budgetMemoryBudget or None

Last measured admission calculation, absent before execution.

budget: MemoryBudget | None = None

fresh: bool = False

memory_limit: int | None = None

pin: bool = True

run(points, work)

Probe memory and run bounded work, yielding results as they finish.

Parameters:

pointsiterable

Independent point descriptions with global indices.

workcallable

Pickleable worker operation; an optional probe reports RSS and ledger.

Yields:

object

Results in completion order, each retaining its input identity.

Raises:

DriverError

If resource admission fails or a worker exits abnormally.

threads: int = 1

validate(work)

Reject unpickleable builders before creating output or claiming work.

Parameters:

workcallable

Worker operation, including its builder and reducers.

Raises:

DriverError

If standard-library pickle cannot transfer the callable.

workers: int | None = None

solprop.driver.executors.pool.peak_rss()

Return this process’s peak resident memory in bytes.

Returns:

int

Peak RSS, converting Linux KiB and retaining Darwin bytes; on Windows the peak working set.