# Matlab-parallel

### From FarmShare

(→MATLAB MDCS) |
|||

(12 intermediate revisions not shown) | |||

Line 1: | Line 1: | ||

+ | == reference == | ||

+ | |||

+ | *CLI options for MATLAB: http://www.mathworks.com/help/matlab/ref/matlabunix.html | ||

+ | |||

+ | *http://www.mathworks.com/help/distcomp/parpool.html#btybrqa-1 | ||

+ | |||

== parallel matlab == | == parallel matlab == | ||

Line 4: | Line 10: | ||

* Batch style where many matlab jobs are submitted and run on the Barley cluster. [[foo]] is an example of this. | * Batch style where many matlab jobs are submitted and run on the Barley cluster. [[foo]] is an example of this. | ||

- | * Parallelism within matlab by use of matlabpools and parallel matlab constructs such as parfor. See [[Matlab-parallel# | + | * Parallelism within matlab by use of matlabpools and parallel matlab constructs such as parfor. See [[Matlab-parallel#batch_matlab_with_matlabpools|below]] for an example. |

Of course, these two can be combined to run many batch jobs, each with a matlabpool of workers. | Of course, these two can be combined to run many batch jobs, each with a matlabpool of workers. | ||

Line 141: | Line 147: | ||

- | === matlab with matlabpools on the barley === | + | === batch matlab with matlabpools on the barley === |

A good way to speed up your matlab code on barley is to submit your matlab job with the shm parallel environment. The parallel environment is specified with -pe shm 4 below. The number you specify after shm will be the number of cores allocated to your job (all from the same host). In this case we are asking for 4 cores, which then become workers in a matlab pool. | A good way to speed up your matlab code on barley is to submit your matlab job with the shm parallel environment. The parallel environment is specified with -pe shm 4 below. The number you specify after shm will be the number of cores allocated to your job (all from the same host). In this case we are asking for 4 cores, which then become workers in a matlab pool. | ||

Line 217: | Line 223: | ||

1232753 0.36667 matlabfft bishopj r 11/28/2013 09:21:06 raring.q@barley09.Stanford.EDU 4 | 1232753 0.36667 matlabfft bishopj r 11/28/2013 09:21:06 raring.q@barley09.Stanford.EDU 4 | ||

</source> | </source> | ||

+ | |||

+ | cat matlabfft.o1232753 | ||

<source lang="m"> | <source lang="m"> | ||

Line 352: | Line 360: | ||

</source> | </source> | ||

+ | |||

+ | == MATLAB MDCS == | ||

+ | |||

+ | The example above shows how to submit a job in grid engine to run MATLAB on multiple cores but on one compute node. | ||

+ | |||

+ | There is also a way to write MATLAB code to split work across multiple machines simultaneously. The example code is in MATLAB help for 'parallel.Cluster/createJob.'. Please contact us if your code looks like the example there and we can help set up the MATLAB cluster parameters for your parcluster. It needs to be configured to run under a "generic scheduler" and your code needs to define the tasks and jobs. | ||

+ | |||

+ | We can use the "shared" example code from here: https://www.mathworks.com/matlabcentral/fileexchange/52816-parallel-computing-toolbox-integration-for-matlab-distributed-computing-server-with-grid-engine | ||

= tomlab = | = tomlab = | ||

Line 359: | Line 375: | ||

# startup | # startup | ||

+ | Same with R2013b, inside matlab: | ||

+ | cd /farmshare/software/non-free/MATLAB-R2013b/tomlab | ||

+ | startup | ||

example session: | example session: | ||

Line 428: | Line 447: | ||

Search the farmshare-discuss archives for posts about [https://mailman.stanford.edu/mailman/swish?query=Matlab&submit=Search+farmshare-discuss+%21&listname=farmshare-discuss&metaname=swishdefault&sort=unixdate Matlab]. | Search the farmshare-discuss archives for posts about [https://mailman.stanford.edu/mailman/swish?query=Matlab&submit=Search+farmshare-discuss+%21&listname=farmshare-discuss&metaname=swishdefault&sort=unixdate Matlab]. | ||

+ | |||

+ | =links= | ||

+ | *http://www.psc.edu/index.php/user-resources/software/matlab/mdcs-r2013a | ||

+ | *http://help.unc.edu/help/mathematical-and-statistical-application-matlab/#P91_3753 | ||

+ | *http://web.stanford.edu/group/proclus/cgi-bin/mediawiki/index.php/Software-MATLAB#Matlab_as_a_job_with_matlab_pool_-_many_machines |

## Latest revision as of 14:03, 1 June 2016

## Contents |

## reference

- CLI options for MATLAB: http://www.mathworks.com/help/matlab/ref/matlabunix.html

## parallel matlab

Parallel matlab comes in two forms.

- Batch style where many matlab jobs are submitted and run on the Barley cluster. foo is an example of this.
- Parallelism within matlab by use of matlabpools and parallel matlab constructs such as parfor. See below for an example.

Of course, these two can be combined to run many batch jobs, each with a matlabpool of workers.

### batch

#### example single matlab file run via qsub

Here's our helloworld.m:

disp('Hello World');

Here's a command to run that non-interactively. (note we pass -singleCompThread for best performance):

matlab -nodesktop -singleCompThread < helloworld.m

We want to run this same command via the job scheduling system. Let's write a job script (save this as matlab_example.script)

#!/bin/bash #$ -N matlab_example #$ -m bes #$ -M chekh@stanford.edu module load matlab matlab -nodesktop -singleCompThread < /farmshare/software/examples/matlab/helloworld.m

Submit the script:

qsub matlab_example.script

Look at the job status:

qstat

You should get output file like matlab_example.oXXXXX

Warning: No display specified. You will not be able to display graphics on the screen. Warning: No window system found. Java option 'MWT' ignored < M A T L A B (R) > Copyright 1984-2011 The MathWorks, Inc. R2011b (7.13.0.564) 64-bit (glnxa64) August 13, 2011 To get started, type one of these: helpwin, helpdesk, or demo. For product information, visit www.mathworks.com. >> Hello World

#### example second matlab file run via qsub

Now we want to make sure we avoid AFS.

Here's our helloworld.m:

disp('Hello World');

Here's a command to run that non-interactively:

matlab -nodesktop < helloworld.m

We want to run this same command via the job scheduling system. Let's write a job script.

#!/bin/bash #$ -N matlab_example #$ -m bes #$ -M chekh@stanford.edu #$ -cwd module load matlab matlab -nodesktop -singleCompThread < /farmshare/software/examples/matlab/helloworld.m

Submit the script:

cd /farmshare/user_data/$USER qsub -cwd matlab_example.script

Look at the job status:

qstat

You should get output file like matlab_example.oXXXXX in your current directory

Warning: No display specified. You will not be able to display graphics on the screen. Warning: No window system found. Java option 'MWT' ignored < M A T L A B (R) > Copyright 1984-2011 The MathWorks, Inc. R2011b (7.13.0.564) 64-bit (glnxa64) August 13, 2011 To get started, type one of these: helpwin, helpdesk, or demo. For product information, visit www.mathworks.com. >> Hello World

### PCT (Parallel Computing Toolbox aka matlabpool)

We have the Parallel Computing Toolbox, you can use that to parallelize your job across multiple cores in a single machine.

Here's how to write a job using MDCS: http://docs.uabgrid.uab.edu/wiki/MatLab_CLI#Parallel_MATLAB

You can use the "maxNumCompThreads" command (deprecated) to see how many parallel threads you can run. I get "24" on barley, or "8" on corn.

#### simple PCT run on corn

matlab -nodesktop -r 'maxNumCompThreads' >> matlabpool ( 'open', 'local', 8) Starting matlabpool using the 'local' configuration ... connected to 8 labs. >> >> matlabpool size ans = 8 >> feature('numCores') ans = 8 Then use 'parfor' instead of 'for'.

matlabpool of size 0 and size 1 are effectively the same, except the latter uses a PCT toolbox license.

Here are some training slides and example code that I copied from http://www.osc.edu/~samsi/sc11edu/

### batch matlab with matlabpools on the barley

A good way to speed up your matlab code on barley is to submit your matlab job with the shm parallel environment. The parallel environment is specified with -pe shm 4 below. The number you specify after shm will be the number of cores allocated to your job (all from the same host). In this case we are asking for 4 cores, which then become workers in a matlab pool.

There is a problem that frequently occurs with matlab if you run many jobs which all use matlab pools. Matlab is naive and does not handle having multiple instances running at the same time. To allow many matlabpools to be run simultaneously, use the following code to setup your pool. You may substitute any number between 1 and 12 for the number of cores (12 being the max matlab will let us use with current license).

#!/bin/bash #$ -N matlabfft #$ -S /bin/bash #$ -pe shm 4 #$ -l mem_free=500M #$ -cwd hostname echo "" echo "" echo "" module load matlab matlab -nodesktop <<EOF cluster = parcluster('local') tmpdirforpool = tempname mkdir(tmpdirforpool) cluster.JobStorageLocation = tmpdirforpool msg = sprintf('setting matlabpool to %s', getenv('NSLOTS')) cluster.NumWorkers = str2num(getenv('NSLOTS')) matlabpool(cluster) cluster a = ones(1000,1000) disp('begin running parfor') parfor i = 1:10000 b=ones./a end disp('done running parfor') matlabpool close EOF

To see if the matlabpools are running correctly, we will submit some jobs and check the output. In this case, its worth noting that 6 jobs ran at same time on 6 different hosts. They all ran to completion and gave same output.

$ qsub ~/matlabfftparcluster.submit Your job 1232748 ("matlabfft") has been submitted $ qsub ~/matlabfftparcluster.submit Your job 1232749 ("matlabfft") has been submitted $ qsub ~/matlabfftparcluster.submit Your job 1232750 ("matlabfft") has been submitted $ qsub ~/matlabfftparcluster.submit Your job 1232751 ("matlabfft") has been submitted $ qsub ~/matlabfftparcluster.submit Your job 1232752 ("matlabfft") has been submitted $ qsub ~/matlabfftparcluster.submit Your job 1232753 ("matlabfft") has been submitted $ qstat job-ID prior name user state submit/start at queue slots ja-task-ID ----------------------------------------------------------------------------------------------------------------- 1232748 0.36667 matlabfft bishopj r 11/28/2013 09:20:51 raring.q@barley18.Stanford.EDU 4 1232749 0.36667 matlabfft bishopj r 11/28/2013 09:21:06 raring.q@barley01.Stanford.EDU 4 1232750 0.36667 matlabfft bishopj r 11/28/2013 09:21:06 raring.q@barley12.Stanford.EDU 4 1232751 0.36667 matlabfft bishopj r 11/28/2013 09:21:06 raring.q@barley02.Stanford.EDU 4 1232752 0.36667 matlabfft bishopj r 11/28/2013 09:21:06 raring.q@barley07.Stanford.EDU 4 1232753 0.36667 matlabfft bishopj r 11/28/2013 09:21:06 raring.q@barley09.Stanford.EDU 4

cat matlabfft.o1232753

barley09.stanford.edu Warning: No display specified. You will not be able to display graphics on the screen. Warning: No window system found. Java option 'MWT' ignored. < M A T L A B (R) > Copyright 1984-2013 The MathWorks, Inc. R2013a (8.1.0.604) 64-bit (glnxa64) February 15, 2013 To get started, type one of these: helpwin, helpdesk, or demo. For product information, visit www.mathworks.com. >> cluster = Local Cluster Properties: Profile: local Modified: false Host: barley09.stanford.edu NumWorkers: 12 JobStorageLocation: /afs/ir/users/b/i/bishopj/.matlab/local_cluster_jobs/R2013a RequiresMathWorksHostedLicensing: false Associated Jobs: Number Pending: 0 Number Queued: 0 Number Running: 0 Number Finished: 1 >> tmpdirforpool = /tmp/1232749.1.raring.q/tpdeab63e3_5ca0_46b5_86e0_5df6c5b655de >> >> cluster = Local Cluster Properties: Profile: local Modified: true Host: barley09.stanford.edu NumWorkers: 12 JobStorageLocation: /tmp/1232749.1.raring.q/tpdeab63e3_5ca0_46b5_86e0_5df6c5b655de RequiresMathWorksHostedLicensing: false Associated Jobs: Number Pending: 0 Number Queued: 0 Number Running: 0 Number Finished: 0 >> >> msg = setting matlabpool to 4 >> cluster = Local Cluster Properties: Profile: local Modified: true Host: barley09.stanford.edu NumWorkers: 4 JobStorageLocation: /tmp/1232749.1.raring.q/tpdeab63e3_5ca0_46b5_86e0_5df6c5b655de RequiresMathWorksHostedLicensing: false Associated Jobs: Number Pending: 0 Number Queued: 0 Number Running: 0 Number Finished: 0 >> >> Starting matlabpool ... connected to 4 workers. >> >> cluster = Local Cluster Properties: Profile: local Modified: true Host: barley09.stanford.edu NumWorkers: 4 JobStorageLocation: /tmp/1232749.1.raring.q/tpdeab63e3_5ca0_46b5_86e0_5df6c5b655de RequiresMathWorksHostedLicensing: false Associated Jobs: Number Pending: 0 Number Queued: 0 Number Running: 1 Number Finished: 0 >> >> >> a = Columns 1 through 13 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 . . . >> >> begin running parfor >> >> >> done running parfor >> >> Sending a stop signal to all the workers ... stopped. >>

## MATLAB MDCS

The example above shows how to submit a job in grid engine to run MATLAB on multiple cores but on one compute node.

There is also a way to write MATLAB code to split work across multiple machines simultaneously. The example code is in MATLAB help for 'parallel.Cluster/createJob.'. Please contact us if your code looks like the example there and we can help set up the MATLAB cluster parameters for your parcluster. It needs to be configured to run under a "generic scheduler" and your code needs to define the tasks and jobs.

We can use the "shared" example code from here: https://www.mathworks.com/matlabcentral/fileexchange/52816-parallel-computing-toolbox-integration-for-matlab-distributed-computing-server-with-grid-engine

# tomlab

We have a TOMLAB license. To start it up:

- launch matlab r2013a
- cd to /farmshare/software/non-free/MATLAB-R2013a/tomlab
- startup

Same with R2013b, inside matlab:

cd /farmshare/software/non-free/MATLAB-R2013b/tomlab startup

example session:

$ module load matlab $ matlab -nodesktop Warning: No display specified. You will not be able to display graphics on the screen. Warning: No window system found. Java option 'MWT' ignored. < M A T L A B (R) > Copyright 1984-2013 The MathWorks, Inc. R2013a (8.1.0.604) 64-bit (glnxa64) February 15, 2013 No window system found. Java option 'MWT' ignored. To get started, type one of these: helpwin, helpdesk, or demo. For product information, visit www.mathworks.com. >> cd /farmshare/software/non-free/MATLAB-R2013a/tomlab >> startup The TOMLAB v7.9 directory is /srv/zfs01/software/non-free/MATLAB-R2013a/tomlab Found the base path as /srv/zfs01/software/non-free/MATLAB-R2013a/tomlab/base Found the cgo path as /srv/zfs01/software/non-free/MATLAB-R2013a/tomlab/cgo Found the testprob path as /srv/zfs01/software/non-free/MATLAB-R2013a/tomlab/testprob Found the examples path as /srv/zfs01/software/non-free/MATLAB-R2013a/tomlab/examples Found the optim path as /srv/zfs01/software/non-free/MATLAB-R2013a/tomlab/optim Warning - optimization toolbox drop-in replacements Remove tomlab\optim from path to disable Found the mex path as /srv/zfs01/software/non-free/MATLAB-R2013a/tomlab/mex Found the splines path as /srv/zfs01/software/non-free/MATLAB-R2013a/tomlab/splines Found the quickguide path as /srv/zfs01/software/non-free/MATLAB-R2013a/tomlab/quickguide Found the MAD path as /srv/zfs01/software/non-free/MATLAB-R2013a/tomlab/mad Found the lib path as /srv/zfs01/software/non-free/MATLAB-R2013a/tomlab/lib Found the usersguide path as /srv/zfs01/software/non-free/MATLAB-R2013a/tomlab/usersguide Found the ampl path as /srv/zfs01/software/non-free/MATLAB-R2013a/tomlab/ampl Found the common path as /srv/zfs01/software/non-free/MATLAB-R2013a/tomlab/common Found the finance path as /srv/zfs01/software/non-free/MATLAB-R2013a/tomlab/finance Found the modellib path as /srv/zfs01/software/non-free/MATLAB-R2013a/tomlab/modellib MAD successfully installed Found the PROPT path as /srv/zfs01/software/non-free/MATLAB-R2013a/tomlab/propt Found the TOMSYM path as /srv/zfs01/software/non-free/MATLAB-R2013a/tomlab/tomsym Found the modellib path as /srv/zfs01/software/non-free/MATLAB-R2013a/tomlab/modellib Run tomlablic to display license information Done with setting TOMLAB paths >> cd quickguide >> nllsQG ===== * * * =================================================================== * * * TOMLAB - Stanford University Ac. department 602044. Valid to 2100-01-01 ===================================================================================== Problem: --- 1: Gisela f_k 103.648860373704110316 f(x_0) 103.652176276833472457 Solver: clsSolve. EXIT=0. INFORM=1. Fletcher-Xu Hybrid Method. Modified Gauss-Newton - BFGS Optimal solution found Iteration points are close ResEv 126 JacEv 118 Iter 11 CPU time: 0.480000 sec. Elapsed time: 0.932415 sec. >>

Search the farmshare-discuss archives for posts about Matlab.

# links

- http://www.psc.edu/index.php/user-resources/software/matlab/mdcs-r2013a
- http://help.unc.edu/help/mathematical-and-statistical-application-matlab/#P91_3753
- http://web.stanford.edu/group/proclus/cgi-bin/mediawiki/index.php/Software-MATLAB#Matlab_as_a_job_with_matlab_pool_-_many_machines