MATLAB

From UMass GHPCC User Wiki
Jump to: navigation, search

Getting Started with Serial and Parallel MATLAB on GHPCC

The scripts required to submit jobs through your local MATLAB install are available here: GHPCC MATLAB Scripts (zip)

The scripts are also available in (tar gz) format. A PDF version of this wiki page is available.

CONFIGURATION - Running MATLAB on GHPCC

For those running MATLAB on the cluster, GHPCC (e.g. via job script or interactively on a compute node)

Start MATLAB. Configure MATLAB to run parallel jobs on your cluster by calling configCluster. For each cluster, configCluster only needs to be called once per version of MATLAB.

>> configCluster

Jobs will now default to the cluster rather than submit to the local machine.

CONFIGURATION - Running MATLAB on local desktop

For those running MATLAB on the local desktop or work machine.

Add the MATLAB integration scripts to your MATLAB Path by placing the integration scripts into $MATLAB/toolbox/local. Windows users can place the scripts in their Documents\MATLAB folder. Once placed in the folder, the scripts can be added to the PATH from within MATLAB. The pane on the left side of the application should have a folder named "toolbox". Right click on this folder and select "Add to Path -> Selected Folder and Subfolders".

Configure MATLAB to run parallel jobs on your cluster by calling configCluster from within MATLAB. For each cluster, configCluster only needs to be called once per version of MATLAB.

Submission to the remote cluster requires SSH credentials. You will be prompted for your ssh username and password or identity file (private key). The username and location of the private key will be cached in MATLAB for future sessions.

CONFIGURING JOBS

Prior to submitting the job, we can specify various parameters to pass to our jobs, such as queue, walltime, e-mail, etc. Other than MemUsage and WallTime, none of these are required to be set.

NOTE: Any parameters specified using the below workflow will be persistent between MATLAB sessions.

>> % Get a handle to the cluster
>> c = parcluster;

>> % Specify the memory usage (per core in MB) [REQUIRED]
>> c.AdditionalProperties.MemUsage = ‘2048’;

>> % Specify the walltime (e.g. 1 hour) [REQUIRED]
>> c.AdditionalProperties.WallTime = ‘01:00:00';

>> % Specify a particular queue to use for MATLAB jobs
>> c.AdditionalProperties.QueueName = ‘queue-name’;

>> % Specify e-mail address to receive notifications about your job
>> c.AdditionalProperties.EmailAddress = ‘user-id@umassmed.edu’;

>> % Set the project name
>> c.AdditionalProperties.ProjectName = ‘project-name’;

>> % Request 2 GPUs per node
>> c.AdditionalProperties.GpusPerNode = 2;

>> % Request the Tesla GPU card
>> c.AdditionalProperties.GpuCard = ‘tesla;

Save changes after modifying AdditionalProperties fields.

>> c.saveProfile

To see the values of the current configuration options, call the specific AdditionalProperties name.

>> % To view current properties
>> c.AdditionalProperties

To clear a value, assign the property an empty value (‘’, [], or false).

>> % Turn off email notifications
>> c.AdditionalProperties.EmailAddress = ‘ ’;

INTERACTIVE JOBS

[Only for MATLAB instances running on the cluster, not remotely on the user’s desktop machine]

To run an interactive pool job on the cluster, continue to use parpool as before.

>> % Open a pool of 64 workers on the cluster
>> p = parpool(64);

Rather than running local on the host machine, the pool can now run across multiple nodes on the cluster.

>> % Run a parfor over a 1000 iterations
>> parfor idx = 1:1000
      a(idx) = …
   end

Once we’re done with the pool, delete it.

>> % Delete the pool
>> p.delete

SERIAL JOBS

Rather than running interactively, use the batch command to submit asynchronous jobs to the cluster.

The batch command will return a job object which is used to access the output of the submitted job.

See the MATLAB documentation for more help on batch.

>> % Get a handle to the cluster
>> c = parcluster;

>> % Submit job to query where MATLAB is running on the cluster
>> j = c.batch(@pwd, 1, {});
>> % Query job for state
>> j.State

>> % If state is finished, fetch results
>> j.fetchOutputs{:}

>> % Delete the job after results are no longer needed
>> j.delete

To retrieve a list of currently running or completed jobs, call parcluster to retrieve the cluster object. The cluster object stores an array of jobs that were run, are running, or are queued to run. This allows us to fetch the results of completed jobs. Retrieve and view the list of jobs as shown below.

>> c = parcluster;
>> jobs = c.Jobs

Once we’ve identified the job we want, we can retrieve the results as we’ve done previously.

fetchOutputs is used to retrieve function output arguments; if using batch with a script, use load instead. Data that has been written to files on the cluster needs be retrieved directly from the file system.

To view results of a previously completed job:

>> % Get a handle on job with ID 2
>> j2 = c.Jobs(2);

NOTE: You can view a list of your jobs, as well as their IDs, using the above c.Jobs command.

>> % Fetch results for job with ID 2
>> j2.fetchOutputs{:}

>> % If the job produces an error view the error log file
>> c.getDebugLog(j.Tasks(1))

NOTE: When submitting independent jobs, with multiple tasks, you will have to specify the task number.

PARALLEL JOBS

Users can also submit parallel workflows with batch. Let’s use the following example for a parallel job, parallel_example.m:

function t = parallel_example(ter)

if nargin==0, iter = 16; end

disp('Start sim')

t0 = tic;
parfor idx = 1:iter
   A(idx) = idx;
   pause(2)
end
t = toc(t0);

disp('Sim completed.')

We’ll use the batch command again, but since we’re running a parallel job, we’ll also specify a MATLAB Pool.

>> % Get a handle to the cluster
>> c = parcluster;

>> % Submit a batch pool job using 4 workers for 16 simulations
>> j = c.batch(@parallel_example, 1, {}, ‘Pool’, 4);

>> % View current job status
>> j.State

>> % Fetch the results after a finished state is retrieved
>> j.fetchOutputs{:}

ans =

      8.8872

The job ran in 8.89 seconds using 4 workers. Note that these jobs will always request N+1 CPU cores, since one worker is required to manage the batch job and pool of workers. For example, a job that needs eight workers will consume nine CPU cores.

We’ll run the same simulation, but increase the Pool size. This time, to retrieve the results later, we’ll keep track of the job ID.

NOTE: For some applications, there will be a diminishing return when allocating too many workers, as the overhead may exceed computation time.

>> % Get a handle to the cluster
>> c = parcluster;

>> % Submit a batch pool job using 8 workers for 16 simulations
>> j = c.batch(@parallel_example, 1, {}, ‘Pool’, 8);

>> % Get the job ID
>> id = j.ID

Id =

      4

>> % Clear workspace, as though we quit MATLAB
>> clear j

Once we have a handle to the cluster, we’ll call the findJob method to search for the job with the specified job ID.

>> % Get a handle to the cluster
>> c = parcluster;

>> % Find the old job
>> j = c.findJob(‘ID’, 4);

>> % Retrieve the state of the job
>> j.State
ans

      finished

>> % Fetch the results
>> j.fetchOutputs{:};

ans =

      4.7270

>> % If necessary, retrieve output/error log file
>> c.getDebugLog(j)

The job now runs 4.73 seconds using 8 workers. Run code with different number of workers to determine the ideal number to use.

Alternatively, to retrieve job results via a graphical user interface, use the Job Monitor (Parallel > Monitor Jobs). The Parallel menu is located in between Layout and Add-Ons'.

DEBUGGING

If a serial job produces an error, we can call the getDebugLog method to view the error log file.

>> j.Parent.getDebugLog(j.Tasks(1))

When submitting independent jobs, with multiple tasks, you will have to specify the task number. For Pool jobs, do not deference into the job object.

>> j.Parent.getDebugLog(j)

The scheduler ID can be derived by calling schedID

>> schedID(j)
ans

      25539

TO LEARN MORE

To learn more about the MATLAB Parallel Computing Toolbox, check out these resources:

Running MATLAB Directly on the Cluster

The instructions below are for running MATLAB scripts without MATLAB's cluster functionality.

Interactive Job

MATLAB can be ran interactively on the cluster by submitting an interactive job and then typing the following commands:

module load matlab/R2017a
matlab

It's also possible to run a script directly after loading the MATLAB module by using the following command:

matlab < /path/to/myFile.m

Batch Job

Below is an example job submission script:

#!/bin/bash

#BSUB -n 1
#BSUB -R rusage[mem=1024] # ask for 1GB of RAM
#BSUB -W 1:0 # run for 1 hour
#BSUB -q short # which queue we want to run in

module load matlab/R2017a
matlab < /path/to/myFile.m

For additional information on submitting jobs on GHPCC see Submitting Cluster Jobs.