BioSimSpace.Process.ProcessRunner

class BioSimSpace.Process.ProcessRunner(processes, name='runner', work_dir=None)

A class for managing and running multiple simulation processes, e.g. a free energy simulation at multiple lambda values.

Since BioSimSpace handles its own background processes it is unsuitable for use with Python modules such as concurrent.futures, where use of objects like a ProcessPoolExecutor would lead to redundant processes, i.e. a process would be created, from which BioSimSpace would fork its own background process. Instead, we recommend using a ProcessRunner, which can handle the running of processes for you, both in serial and parallel.

At present there is no way to allocate specific compute resources to individual processes. As such, unless you have access to a large amount of compute, when executing the runner in parallel we recommend that the individual processes are serial in nature. BioSimSpace is not intended to be a workflow manager and the ProcessRunner is only meant to help facilitate running of more complex, multi-leg simulation processes. If you desire more fine-grained resource control we recommend breaking your workflow into separate nodes, which can be run independently and allocated their own specific resources.

__init__(processes, name='runner', work_dir=None)

Constructor.

Parameters
  • processes ([Process]) – A list of process objects.

  • name (str) – The name of the of processes.

  • work_dir (str) – The working directory for the processes.

Methods

__init__(processes[, name, work_dir])

Constructor.

addProcess(process)

Add a process to the runner.

errored()

Return the indices of the errored processes.

getName()

Return the process runner name.

isError()

Return whether each process is in an error state.

isQueued()

Return whether each process is queued.

isRunning()

Return whether each process is running.

kill(index)

Kill a specific process.

killAll()

Kill all of the processes.

nError()

Return the number of errored processes.

nProcesses()

Return the number of processes.

nQueued()

Return the number of queued processes.

nRunning()

Return the number of running processes.

processes()

Return the list of processes.

queued()

Return the indices of the queued processes.

removeProcess(index)

Remove a process from the runner.

restartFailed()

Restart any jobs that are in an error state.

runTime()

Return the run time for each process.

running()

Return the indices of the running processes.

setName(name)

Set the process runner name.

start(index)

Start a specific process.

startAll([serial, batch_size, max_retries])

Start all of the processes.

wait()

Wait for any running processes to finish.

workDir()

Return the working directory.

addProcess(process)

Add a process to the runner.

Parameters

process (Process, [Process]) – The process/processes to add.

errored()

Return the indices of the errored processes.

Returns

idx_errored – A list containing the indices of the errored processes.

Return type

[int]

getName()

Return the process runner name.

Returns

name – The name of the process.

Return type

str

isError()

Return whether each process is in an error state.

Returns

is_error – A list indicating whether each process is in an error state.

Return type

[bool]

isQueued()

Return whether each process is queued.

Returns

is_queued – A list indicating whether each process is queued.

Return type

[ bool ]

isRunning()

Return whether each process is running.

Returns

is_running – A list indicating whether each process is running.

Return type

[ bool ]

kill(index)

Kill a specific process. The same can be achieved using:

runner.processes()[index].kill()

Parameters

index (int) – The index of the process.

killAll()

Kill all of the processes.

nError()

Return the number of errored processes.

Returns

n_error – The number of processes that are in an error state.

Return type

int

nProcesses()

Return the number of processes.

Returns

n_processes – The number of processes managed by the runner.

Return type

int

nQueued()

Return the number of queued processes.

Returns

n_queued – The number of processes that are queued.

Return type

int

nRunning()

Return the number of running processes.

Returns

n_running – The number of processes that are running.

Return type

int

processes()

Return the list of processes.

Returns

processes – The list of processes.

Return type

[Process]

queued()

Return the indices of the queued processes.

Returns

idx_queued – A list containing the indices of the queued processes.

Return type

[int]

removeProcess(index)

Remove a process from the runner.

Parameters

index (int) – The index of the process.

restartFailed()

Restart any jobs that are in an error state.

runTime()

Return the run time for each process.

Returns

run_time – A list containing the run time of each process.

Return type

[ BioSimSpace.Types.Time ]

running()

Return the indices of the running processes.

Returns

idx_running – A list containing the indices of the running processes.

Return type

[ int ]

setName(name)

Set the process runner name.

Parameters

name (str) – The process runner name.

start(index)

Start a specific process. The same can be achieved using:

runner.processes()[index].start()

Parameters

index (int) – The index of the process.

startAll(serial=False, batch_size=None, max_retries=5)

Start all of the processes.

Parameters
  • serial (bool) – Whether to start the processes in serial, i.e. wait for a process to finish before starting the next. When running in parallel (serial=False) care should be taken to ensure that each process doesn’t consume too many resources. We normally intend for the ProcessRunner to be used to manage single core processes.

  • batch_size (int) – When running in parallel, how many processes to run at any one time. If set to None, then the batch size will be set to the output of multiprocess.cpu_count().

  • max_retries (int) – How many times to retry a process if it fails.

wait()

Wait for any running processes to finish.

workDir()

Return the working directory.

Returns

work_dir – The working directory.

Return type

str