snake.core.parallel

Utilities for running parallel computations with processes and shared memory.

Module Contents

Classes

ArrayProps	Properties of an array stored in shared memory.
SHM_Wrapper	Wrapper for function to be call with parallel shared memory.

Functions

run_parallel	Run a function in parallel with shared memory.
array_from_shm	Get arrays from shared memory.
array_to_shm	Move an array to shared memory.

Data

log

API

snake.core.parallel.log = 'getLogger(...)'

class snake.core.parallel.ArrayProps

Bases: typing.NamedTuple

Properties of an array stored in shared memory.

name: str = None

shape: tuple[int, ...] = None

dtype: numpy.typing.DTypeLike = None

class snake.core.parallel.SHM_Wrapper(func: collections.abc.Callable)

Wrapper for function to be call with parallel shared memory.

Parameters:

func (Callable) – Function to be called with shared memory arrays.

Initialization

__call__(input_props: snake.core.parallel.ArrayProps, output_props: snake.core.parallel.ArrayProps, *args: Any, **kwargs: Any) → None

Run in parallel with shared memory.

snake.core.parallel.run_parallel(func: collections.abc.Callable, input_array: numpy.typing.NDArray, output_array: numpy.typing.NDArray, n_jobs: int = -1, parallel_axis: int = 0, *args: Any, **kwargs: Any) → numpy.typing.NDArray

Run a function in parallel with shared memory.

snake.core.parallel.array_from_shm(*array_props: snake.core.parallel.ArrayProps) → collections.abc.Generator[list[numpy.typing.NDArray], None, None]

Get arrays from shared memory.

snake.core.parallel.array_to_shm(array: numpy.typing.NDArray, smm: multiprocessing.managers.SharedMemoryManager) → tuple[snake.core.parallel.ArrayProps, numpy.typing.NDArray, multiprocessing.shared_memory.SharedMemory]

Move an array to shared memory.