Running Matlab

Running Matlab on ARCHIE-WeSt

.
This is a simple example of Matlab which should run on ARCHIE-WeSt and take 2 minutes. The input files and job script are downloadable below.
This example contains code and jobs scripts to execute 120,000 hands of blackjack in serial and in parallel.
.

1. Load the appropriate environment modules

.
To run MATLAB load the appropriate module:

module load apps/bin/matlab/R2017a
.

2. Running MATLAB in parallel

.
Built-in Multithreading
By default, MATLAB will attempt to use all available cores on the host computer on which it is run. Many internal routines are multi-threaded (e.g. linear algebra and fft routines) and will run automatically on multiple cores. Therefore, it may be the case that you can run your MATLAB program on ARCHIE and achieve some increased performance for no additional effort.

.
Parallelism Using MATLAB Workers

You can, however, run multiple MATLAB workers (MATLAB computational engines) explicitly to execute applications in parallel, with Parallel Computing Toolbox. This approach allows you more control over the parallelism than with built-in multithreading, and is often used for coarser grained problems such as running parameter sweeps in parallel.
For example, be replacing a for loop with a parfor loop in your code allows that block of code to be parallelised over the available workers. You can run on up to 12 workers on a standard ARCHIE node. This type of parallelism first requires a matlab paralell pool to be initialised. For more information, see: http://www.mathworks.co.uk/help/distcomp/index.html .

Forcing MATLAB to run in serial mode

If you plan to run MATLAB programs on the serial queue, you should ensure that you force MATLAB to run in single core mode. This is done by supplying the  -singleCompThread switch, e.g.

  • matlab -nodisplay -nodesktop -singleCompThread -r “blackjack;exit”

If the  -singleCompThread is not supplied, then each instance of MATLAB will run in muti-threaded mode resulting in the nodes becoming oversubscribed.

3. Prepare input files

.
The following files are required to be present in the working directory:

  • blackjack.m
  • parallel_blackjack.m
  • pctdemo_task_blackjack.m
  • job scripts start-matlab-serial.sh and  start-matlab-parallel.sh

These can be downloaded as a tar file from here.
The job script start-matlab-serial.sh will use a single core:

#!/bin/bash
# Simple serial job submission script
# Specifies that all environment variables active within the qsub
# utility be exported to the context of the job.
#$ -V
# Execute the job from the current working directory. Standard output and
# standard error files will be written to this directory
#$ -cwd
# Submit to the queue called serial-low.q
#$ -q serial-low.q
#Specify Project identifier
#$ -P training.prj
# Merges standard error stream with standard output
#$ -j y
# Specifies the name of the file containing the standard output
#$ -o out.$JOB_ID
# Add runtime indication
#$ -l h_rt=1:00:00
matlab -nodisplay -nodesktop -singleCompThread -r "blackjack;exit"

The job script (start-matlab-parallel.sh) will use up to 12 cores:

#!/bin/bash
# ************* SGE qsub options ****************
#Export env variables and keep current working directory
#$ -V -cwd
#Select parallel environment and number of parallel queue slots (nodes)
#$ -pe mpi-verbose 1
#$ -q parallel-low.q
#$ -P training.prj
#Combine STDOUT/STDERR
#$ -j y
#Specify output file
#$ -o out.$JOB_ID
#Request resource reservation (reserve slots on each scheduler run until enough have been gathered to run the job
#$ -R y
#Indicate runtime
#$ -l h_rt=02:00:00
# ************** END SGE qsub options ************
matlab -nodisplay -nodesktop -r "parallel_blackjack;exit"
.

4. Job submission

.
To submit the job, type:

qsub start-matlab-serial.sh
.

or

qsub start-matlab-parallel.sh
.

As a result, the following files will be produced (output files):

  • out.$jobID$

The output files can be downloaded as a tar file from here  for comparison.