Python Using Anaconda Evnironment

Step 1: Installing Python Packages with Anaconda

Load Environment Module: anaconda

Install Anaconda Package: Python 3.10, pip, Numpy

Installation Path (Anaconda Environment): /home/$USER/.conda/envs/<environment name>

To instal python packages with Anaconda, run below command in the SSH terminal (CLI), for tutorial in accessing CLI, please refer to Shell Access and Useful Command

# Load Anaconda Environment Module
$ module load anaconda

# Add conda-forge Channels
$ conda config --add channels conda-forge

# Create Anaconda Virtual Environment
$ conda create –n <environment name> <packages to install>

# Create Anaconda Virtual Environment (Example)
$ conda create –n numpy_test_env python=3.10 pip numpy

# Purge Module
$ module purge

Step 2: Prepare Python program source code

Example source code path -> /home/$USER/job_template/python/numpy_test.py

import numpy as np

# Create a 1-dimensional array (vector)
array_1d = np.array([1, 2, 3, 4, 5])
print("1D Array:", array_1d)

# Create a 2-dimensional array (matrix)
array_2d = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
print("\n2D Array:\n", array_2d)

# Perform element-wise operations
sum_array = array_1d + 10
print("\n1D Array after adding 10 to each element:", sum_array)

product_array = array_2d * 2
print("\n2D Array after multiplying each element by 2:\n", product_array)

# Access elements using indexing and slicing
print("\nElement at index 2 of 1D array:", array_1d[2])
print("First row of 2D array:", array_2d[0])
print("Elements from index 1 to 3 of 1D array:", array_1d[1:4])

# Get array attributes
print("\nShape of 2D array:", array_2d.shape)
print("Number of dimensions of 2D array:", array_2d.ndim)
print("Data type of elements in 1D array:", array_1d.dtype)

Step 3: Prepare job template script

Pre-configured template script path -> /home/$USER/job_template/slurm_job/numpy_test_conda.sh

Example template script

#!/bin/bash
#SBATCH --job-name=numpy_test_conda ## Job Name
#SBATCH --partition=shared_cpu ## Partition for Running Job
#SBATCH --nodes=1 ## Number of Compute Node
#SBATCH --ntasks-per-node=1 # Number of Task per Compute Node
#SBATCH --cpus-per-task=2 ## Number of CPU per task
#SBATCH --time=60:00 ## Job Time Limit (i.e. 60 Minutes)
#SBATCH --mem=10GB ## Total Memory for Job
#SBATCH --output=./%x%j.out ## Output File Path
#SBATCH --error=./%x%j.err ## Error Log Path

## Initiate Environment Module
source /usr/share/modules/init/profile.sh

## Reset the Environment Module components
module purge

## Load Module
module load anaconda

## Run user command in conda environemnt
## Do not use conda activate command, use absolute path instead
${CONDA_ENV_PATH}/numpy_test_env/bin/python3 \ /home/${USER}/job_template/python/numpy_test.py

## Clear Environment Module components
module purge

Step 4: Create Template (Web Interface Feature)

To submit HPC via web interface a job template is required, details please refer to: Create Template (Web Interface Feature)

For Job submission via CLI Terminal, please skip this step.

Step 5: Submit HPC Job

Guides for submitting HPC job, please refer to: HPC Job Submission

Step 6: Remove user’s Conda (Python) environment (Optional)

# Remove user’s Conda (Python) environment
$ rm –rf /home/$USER/.conda/envs/<environment name>

# Remove user’s Conda (Python) environment (Example)
$ rm –rf /home/$USER/.conda/envs/numpy_test_env