Skip to content

how to distribute stitching over slurm

You can launch the stitching on a computer having access to the slurm sbatch command, then distribute the stitching over slurm.

The stitching configuration file contains a dedicated slurm section which allows you to define how the work should be distributed.

It contains the following keys:

  • partition: name of the partition to use for launching the jobs
  • mem: memory to allocate for each job.
  • n_jobs: define into how many jobs the stitching should be split.
  • python_venv: python virtual environment to source before applying stitching. It must contain nabu.
  • modules: modules to be loaded before applying stitching. One must contain nabu. If provided then python_venv should be empty. Examples:

    modules = "tomotools"
    
    modules = ("tomotools/dev", "second_module")
    

Distribution will be done as follows (pseudo-code):

compute_shifts [locally or remotely] (option locally will enforce doing this calculation locally)
for each part of the stitching:
    apply_stitching_on_this_part()
concatenate_the_different_parts [locally or remotely] (option locally will enforce doing this calculation locally)