MATLAB on GHPCC

From UMass GHPCC User Wiki
Jump to: navigation, search

We currently have a 128 cpu license for running MATLAB on GHPCC. We have worked with Mathworks to create a set of configuration files that allows you to submit tasks directly to the GHPCC LSF scheduler from within the Matlab R2014a application.

Please Note

We have reports of some issues with the Parallel Cluster licensing on the ghpcc, which in certain circumstances may cause jobs to hang. We’ve implemented a workaround while we wait on Mathworks to resolve the issue. Please notify hpcc-support@umassmed.edu if you see license issues using the Parallel Toolbox on ghpcc.

Using Matlab's Parallel Computing Toolbox

We recommend watching the Mathworks provided webinar online on using the Parallel Computing Toolbox

Configuring your local system to submit Matlab jobs to the GHPCC cluster

In order to submit Matlab jobs to the GHPCC cluster you will need Matlab R2014a installed and functioning on your local system. Other versions of Matlab will not work. Once it's installed

  • Copy /share/pkg/matlab/R2014a/clientconfig/sharedcluster_remote_r2014a.zip from GHPCC (available via ghpcc06) to the local machine you have Matlab R2014a installed on.
  • Unzip the sharedcluster_remote_r2014a.zip archive under your matlab/toolbox/local path or your "My Documents/matlab" path.
  • In Matlab run "configCluster"
    • You will be asked for a directory to use on the cluster. This may be your home directory, a path in project space you have access to, or a subdirectory under either place.
      • Matlab will not create the directory you specify, so make sure it already exists, or that you create it manually by logging in to the cluster and running mkdir before attempting to submit a job.
      • Matlab will create a directory under this path called MdcsDataLocation which is used to transfer data files and hold job files.
    • You will be asked for your ghpcc cluster username.

Ex:

>> configCluster
Clearing all ClusterInfo settings.
Home directory on GHPCC (e.g. /home/js72b or /project/umb_joe_smith) : /home/ru18w
Username on GHPCC (e.g. js72b) : ru18w

Before sumbitting a job to GHPCC, you must specify the memory usage per worker.

	>> % E.g. 2GB 

	>> ClusterInfo.setMemUsage('2048')


Walltime defaults to 1 hour. To provide a walltime:
	>> % E.g. 10 minutes 

	>> ClusterInfo.setWallTime('00:10')

>> 
  • In Matlab run "c=parcluster()"
  • Try a simple test, e.g.:\
j = batch(c,@rand,1,{10,10})
or
j=c.batch(@pwd,1,{},'Pool',3)

Job parameters within Matlab

Matlab jobs running on GHPCC are subject to the same rules as jobs submitted directly with the bsub command. Matlab has a number of parameters which will define how it tells LSF to run the job. Some of these are:

  • ClusterInfo.setMemUsage('X')
    • Defines how much memory is needed for each Matlab worker, where X is the number of megabytes.
    • If left undefined, LSF sets a default of 1gb.
    • The equivalent of "-R rusage[mem=X]" for the bsub command.
  • ClusterInfo.setWallTime('XX:YY')
    • Defines the maximum runtime of the job, where XX is hours, YY is minutes.
    • If left undefined, LSF sets a default of 1 hour.
    • The equivalent of "-W XX:YY" for the bsub command.
  • ClusterInfo.setQueueName('X')
    • Defines which queue on the cluster to submit the job to.
    • If left undefined, LSF sets to the 'long' queue.
    • Other queue options currently include 'short' and 'gpu'.
    • The equivalent of "-q X" for the bsub command.
  • ClusterInfo.setUserDefinedOptions('X')
    • Allows you to specify other bsub options.
    • Common examples might be where you replace 'X' with:
      • '-R span[ptile=4]' to request workers be grouped four to a host.
      • '-R span[hosts=1]' to request all workers be run on the same host.
      • '-m ghpcc-sgi' to request that workers be run specifically on the SGI UV2 node.
    • Setting this and the setMemUsage parameter may cause a conflict, if so, try clearing the ClusterInfo.setMemUsage and including the "-R rusage[mem=X]" as part of this parameter.

Submitting a job to GHPCC

How do I submit a job?

According to Mathworks the easiest way to submit a job to the cluster from Matlab is using the batch() command.

An example command might look like:

j=c.batch(@functionName or ‘ScriptName’,numberOfOutputArguments, {inputArguments,go,here},’Pool’,NumberOfWorkers)

Mathworks documentation online has information about the batch() command here: http://www.mathworks.com/help/distcomp/batch.html

You can also get information batch() in Matlab by running:

help batch

Identity File Prompt

When you submit a job to ghpcc the first time, Matlab will prompt to see if you want to use an identity file (ssh key file) to login to ghpcc06:

User credentials.PNG

Choosing 'No' will then have Matlab prompt you for your GHPCC user account password.

Error Messages

ghpcc_remote_r2014a does not support calling

ghpcc_remote_r2014a does not support calling

    >> matlabpool open ghpcc_remote_r2014a 12

Instead, use batch()

    >> job = batch(...);

When you directly open a pool at the matlab prompt, it asks the workers on the cluster to directly connect to the MATLAB client you started the parpool, but due to the network architecture the workers will not be able to ping the MATLAB client.

You can run a function or a script within a batch call. So make a script that loads or sets your variables then runs your parfor or spmd code block and pass it in to batch to be run on the cluster.