.. _job-parameters:

Job parameters
==============

When instantiating a SLURM client or executor, you can provide the :code:`parameters` argument

.. code-block:: python

   from pyslurmutils.client import SlurmScriptRestClient
   
   parameters={"time_limit": "02:00:00"}
   client = SlurmScriptRestClient(
       url=url,
       user_name=user_name,
       token=token,
       log_directory=log_directory,
       parameters=parameters,
   )

The parameters can also be overridden when submitting jobs

.. code-block:: python

   future = executor.submit(..., slurm_arguments={"parameters": parameters})

Documentation on all available SLURM job parameters can be found `here <https://slurm.schedmd.com/rest_api.html>`_.

Environment variables
---------------------

SLURM job environment variables can be passed in two ways.

Either by passing the ``environment`` parameter. For example:

.. code-block:: python

  parameters = {"environment": {"MYVAR1":"MYVALUE1"}}

Or by defining local environment variables that start with ``SLURM_ENV_``. For example:

.. code-block:: bash

  export SLURM_ENV_MYVAR2=MYVALUE2

In this example, the final SLURM job parameter ``environment`` will be:

.. code-block:: python

    environment = {
        "MYVAR1": "MYVALUE1",   # from 'environment' parameter
        "MYVAR2": "MYVALUE2",   # from SLURM_ENV_*
    }

Note that the ``SLURM_ENV_`` prefix is stripped before passing it to SLURM.

Priority of environment variables from high to low:

1. ``parameters["environment"]`` (explicit SLURM environment parameter)
2. ``SLURM_ENV_*`` (local environment variables prefixed with `SLURM_ENV_`)

Common SLURM Job Parameters
----------------------------

.. list-table:: SLURM Job Parameters Mapping
   :header-rows: 1
   :widths: 20 20 50

   * - **srun/sbatch**
     - **REST**
     - **Description**
   * - | :code:`--array=0,6,16-32`
     - | :code:`array="0,6,16-32"`
     - Job array index specification
   * - | :code:`--job-name=my_process`
     - | :code:`name="my_process"`
     - Specify a name for the job
   * - | :code:`--input=input.txt`
     - | :code:`standard_input="input.txt"`
     - Path to stdin file
   * - | :code:`--output=output.txt`
     - N/A
     - Path to stdout file. Cannot be used directly from the API. Use the parameters :code:`log_directory` and :code:`std_split` of :code:`SlurmRestExecutor` instead.
   * - | :code:`--error=error.txt`
     - N/A
     - Path to stderr file. Cannot be used directly from the API. Use the parameters :code:`log_directory` and :code:`std_split` of :code:`SlurmRestExecutor` instead.
   * - | :code:`--time-min=30`
       | :code:`--time-min=10:02:03`
     - | :code:`time_minimum=30`
       | :code:`time_minimum="10:02:03"`
     - Minimum run time for the job (minutes or HH:MM:SS)
   * - | :code:`--time=30`
       | :code:`--time=10:02:03`
     - | :code:`time_limit=30`
       | :code:`time_limit="10:02:03"`
     - Maximum run time for the job
   * - | :code:`--begin=16:00:00`
     - | :code:`begin_time=1735689600 (Unix timestamp)`
     - Defer job allocation until specified time
   * - | :code:`--deadline=18:00:00`
     - | :code:`end_time=1735689600 (Unix timestamp)`
     - Expected end time for the job
   * - | :code:`--gres=gpu:2`
     - N/A
     - Generic consumable resources (legacy)
   * - | :code:`--gpus=3`
       | :code:`--gpus=volta:3`
     - | :code:`tres_per_job="gres/gpu=2"`
     - Number of GPUs required for the job
   * - | :code:`--gpus-per-node=3`
       | :code:`--gpus-per-node=volta:3`
     - | :code:`tres_per_node="gres/gpu=2"`
     - Number of GPUs required per node
   * - | :code:`--gpus-per-task=3`
       | :code:`--gpus-per-task=volta:3`
     - | :code:`tres_per_task="gres/gpu=2"`
     - Number of GPUs required per task
   * - | :code:`--cpus-per-task=4`
     - | :code:`cpus_per_task=4`
     - Number of CPUs required per task
   * - | :code:`--cpus-per-gpu=8`
     - | :code:`cpus_per_tres="gres/gpu=8"`
     - Number of CPUs allocated per GPU
   * - | :code:`--nodes=4`
       | :code:`--nodes=1,4,8`
     - | :code:`nodes="4"`
       | :code:`nodes="1,4,8"`
     - Node count or range specification
   * - | :code:`--exclude=hpc1,hpc8`
     - | :code:`excluded_nodes="hpc1,hpc8"`
     - Exclude specific nodes from allocation
   * - | :code:`--exclusive`
     - | :code:`exclusive="true"`
     - Request exclusive node allocation
   * - | :code:`--nodelist=node042`
     - | :code:`allocation_node_list="node042"`
     - Request specific nodes
   * - | :code:`--partition=gpu_long`
     - | :code:`partition="gpu_long"`
     - Request specific partition
   * - | :code:`--mem-per-cpu=1024`
       | :code:`--mem-per-cpu=1G`
     - | :code:`memory_per_cpu=1024`
       | :code:`memory_per_cpu="1G"`
     - Memory required per CPU
   * - | :code:`--mem-per-gpu=1024`
       | :code:`--mem-per-gpu=1G`
     - | :code:`memory_per_tres="gres/gpu=1024"`
       | :code:`memory_per_tres="gres/gpu=1G"`
     - Memory required per GPU
   * - | :code:`--mem=1024`
       | :code:`--mem=1G`
     - | :code:`memory_per_node=1024`
       | :code:`memory_per_node="1G"`
     - Total memory per node


Example Usage
-------------

For example this script

.. code-block:: python

   from pyslurmutils.client import SlurmScriptRestClient

   SCRIPT = """#!/usr/bin/env python3
   import os
   host = os.gethostname()
   pid = os.getpid()
   ncpus = len(os.sched_getaffinity(pid))
   print(f'{host=}, {pid=}, {ncpus=}')
   """

   parameters = {"cpus_per_task": 4}
   client = SlurmScriptRestClient(
       url=url,
       user_name=user_name,
       token=token,
       log_directory=log_directory,
       parameters=parameters,
   )

The parameters can also be overridden when submitting jobs

.. code-block:: python

   job_id = client.submit_script(SCRIPT, parameters=parameters)
   try:
       print(client.wait_finished(job_id))
       client.print_stdout_stderr(job_id)
   finally:
       client.clean_job_artifacts(job_id)

The output confirms that 4 CPUs are available to the job

.. code-block:: bash

   COMPLETED
   STDOUT/STDERR: /tmp_14_days/<username>/slurm_logs/pyslurmutils.<hostname>.15120368.outerr
   -------------------------------------------------------------------------------------
   host='<slurmnode>', pid=1774392, ncpus=4