Can I run multi-threaded jobs?

If you would like to use multiple CPU cores simultaneously, please be sure to reserve the appropriate number of CPU cores when submitting a job. This will ensure that your jobs do not compete with other jobs for a CPU, and are allocated resources for your exclusive use.

Many applications and programming libraries can make use of multiple CPU cores simultaneously by running multiple active threads or processes.

Jobs in the RCE are guaranteed access to as many CPU cores as are reserved for the job. More CPU cores may be available, but if there is competition for CPU cores between jobs, then each job will be limited to only the number of CPU cores requested. Available cluster resources may be lower during periods of intense utilization. This applies to both Interactive RCE Powered Applications and Batch parallel computing jobs.

  • Batch cluster jobs are allocated as much memory and as many CPU cores per job instance as requested in the .submit file, and you can queue an unlimited number of job instances for high-throughput parallel computing. Batch jobs must run disconnected from a display, although they can write to log and output files.
  • For RCE Powered (interactive) jobs, you can request an allocation of up to 24 CPU cores (and up to 250GB RAM), but each job submission queues only one instance.
    • When running an multi-threaded RCE Powered Application from the menu, you will be prompted for the number of CPU cores.
    • To request a different number of CPU cores from the command line, use -c num_cpu -x command

Once your application is running, it will be unaware of the number of CPU cores reserved for its use, and may try to spawn as many threads or processes as there are CPU cores on the back-end RCE cluster server. This is undesirable, as it can cause your jobs threads to compete against each other. To improve job performance, you can use these commands to set the number of CPU cores used by the app to be equal to the number of CPU cores you reserved (N):

  • Matlab
    • Option 1: 1 CPU core
      • For an Interactive RCE Powered job, run this command in an RCE Powered Shell rather than launching the application directly.
      • For Batch parallel computing jobs, run this command as your job executable, or submit your job with the script which adds these parameters automatically.
        • matlab -nodisplay -singleCompThread
    • Option 2: N CPU cores
      • Within the application, run:
        • maxNumCompThreads( N )
          • This command is available as of Matlab R2014b but may be removed in a future release.
    • See:
  • R
    • Each multi-threaded R library has a different mechanism for limiting the number of threads. Consult the documentation for the libraries you use for specific information.
    • library(doMC)
    • Any library using OpenMP for threading:
      • Sys.setenv( OMP_NUM_THREADS = "N" )
  • Shell
    • Each multi-threaded application has a different mechanism for limiting the number of threads. Consult the documentation for the application you use for specific information.
    • For any application using OpenMP for threading, set the shell variable OMP_NUM_THREADS before running the application.
      • If you are using the bash shell, run:
        • export OMP_NUM_THREADS=N
      • If you are using the tcsh shell, run:
        • setenv OMP_NUM_THREADS N