Edit detail for Running jobs on the servers revision 2 of 1

2
Editor: joseph.r.kline.1
Time: 2023/05/12 15:35:59 GMT-4
Note:

changed:
-
Starting Jobs on Servers from Windows machines with **'ssh'**

1 **'ssh'** into a server.

2 Please use <a href="https://engineering.purdue.edu/AAE/AboutUs/Help/presentation/screen.html">screen</a>, you'll find it much easier.

3 Once you have a screen session running you are ready to proceed.

4 You will need to where you want to run your job

  cd /home/roger/a/gizmo/dev/bground_on_jvoss

5 Then you will need to run your job:

  module load matlab
  matlab -nodisplay < program.m

This will startup matlab without the gui. Then run program.m

This <a href="https://engineering.purdue.edu/AAE/AboutUs/Help/server_status/compute_servers_html">page</a> lists the department servers.
  
  **Useful Commands**

  To check the load on the server you will run on use:
  
  **/usr/local/etc/top** or **/usr/bin/top**

  This will show the processes using the most CPU and memory.

  **/usr/bin/uptime**

  This returns one line simply summarizing the load on the server over the last 1 minute,
  5 minutes, and 15 minutes. A load of 1.0 is a single cpu being used 100%.

  **Killing Jobs**

  In the event you need to stop a job do the following:

  1 Get the process id
  
      Use the **top** command to list the jobs on the server. 
      You should see your username next to any jobs you have on the server.
      The display looks something like this:

      <pre>
      PID   USER     PRI  NI  SIZE  RSS SHARE STAT %CPU %MEM   TIME CPU COMMAND
      20509 kakoi     25   0 68348  12M 10700 R    36.5  0.3 131:51   0 MATLAB
      20852 kakoi     25   0 80236  58M 11252 R    30.6  1.4 123:13   0 MATLAB
      21112 kakoi     25   0 76088  54M 11212 R    29.6  1.4 119:36   0 MATLAB
      21920 jfrommer  25   0 81904  74M 15676 R    19.7  1.9 101:33   0 MATLAB
      1     root      15   0   512  464   440 S     0.0  0.0   0:05   0 init
      2     root      15   0     0    0     0 SW    0.0  0.0   0:00   0 keventd
      </pre>

      The left hand column lists the process id (PID) number
      for the jobs.

      Say you are jfrommer and would like to kill your job because the data it is dumping
      is wrong and the run is useless. The PID for it is 21920.


  2 Kill the job
  
      Now that you have the pid for the job now it's time to stop it. This rather
      straightforward. Run the command 'kill -15 21920'.
      This will "politely" stop the job.
      If that doesn't work after once or twice we need to get mean.

      Run the command 'kill -9 21920'. This should do the trick.


  **Things to be careful of**

    * Creating/Edit documents with Windows editors
    
      Windows text files are a little different in that they have an extra
      character at the end of each line. So, unless you edit the files on Linux/Unix
      you will need to run the the **dos2unix** command to strip out those extra characters.

    * Unix vs. Windows paths

      Make sure that you use unix paths when running on the servers.


Also, here's a very crude, but nifty example of a **'startup.m'** script if
you want to load up some stuff in matlab before your script runs::

  startup.m (for matlab 6.5.1)
  ================================
  os = regexp(pwd,{':'})
  if os == 2
      disp('In Windows');
      addpath N:\Personal\RTBP_models N:\Personal\RTBP_models\Functions
      cd N:\Personal\RTBP_models
  else
      disp('In *nix');
      addpath /home/roger/a/cwagner/Personal/RTBP_models /home/roger/a/cwagner/Personal/RTBP_models/Functions
      cd /home/roger/a/cwagner/Personal/RTBP_models
  end
  ================================


  startup.m (for matlab 7)
  ================================
  match = regexp(pwd,{':'})
  os = cell2mat(match)
  if os == 2
      disp('In Windows');
      addpath N:\Personal\RTBP_models N:\Personal\RTBP_models\Functions
      cd N:\Personal\RTBP_models
  else
      disp('In *nix');
      addpath /home/roger/a/cwagner/Personal/RTBP_models /home/roger/a/cwagner/Personal/RTBP_models/Functions
      cd /home/roger/a/cwagner/Personal/RTBP_models
  end
  ================================

<strong><a name="user_interaction">If you get a buffer overflow error</a></strong>

<a href="http://www.mathworks.com/support/bugreports/details.html?rp=310077">Matlab bug</a>

<dl>
  <dt>Summary</dt>
    <dd>Redirecting an M-file into matlab -nodisplay can cause "Type-ahead Buffer Overflow" errors</dd>


<dt>Description</dt>

<dd>
  <p>One way to have MATLAB execute an M-file on a UNIX system is to redirect that M-file into MATLAB with the command</p>

<p>matlab -nodisplay < myMfile.m</p>

<p>at the shell prompt.</p>

<p>If myMfile.m contains a call to the DCT function waitForState, it is possible that you will see "Type-ahead Buffer Overflow" error messages in the standard output stream from MATLAB.</p></dd>

<dt>Workaround</dt>

<dd><p>If you start MATLAB with the -r flag, which also executes the specified M-file, rather than use input redirection, you should not see this behavior. For example,</p>

<p>matlab -nodisplay -r myMfile</p>
<p>Note that you don't add the file extension (if there is one). So if the file name is MyFile.m, you don't have the .m when using the -r option.</p></dd>

</dl>

From joseph.r.kline.1 Fri Apr 16 11:08:22 -0500 2004
From: joseph.r.kline.1
Date: Fri, 16 Apr 2004 11:08:22 -0500
Subject: 
Message-ID: <20040416110822-0500@https://engineering.purdue.edu>



Starting Jobs on Servers from Windows machines with ssh

  1. ssh into a server.
  2. Please use screen, you'll find it much easier.
  3. Once you have a screen session running you are ready to proceed.
  4. You will need to where you want to run your job

    cd /home/roger/a/gizmo/dev/bground_on_jvoss

  5. Then you will need to run your job:

    module load matlab matlab -nodisplay < program.m

This will startup matlab without the gui. Then run program.m

This page lists the department servers.

Useful Commands

To check the load on the server you will run on use:

/usr/local/etc/top or /usr/bin/top

This will show the processes using the most CPU and memory.

/usr/bin/uptime

This returns one line simply summarizing the load on the server over the last 1 minute, 5 minutes, and 15 minutes. A load of 1.0 is a single cpu being used 100%.

Killing Jobs

In the event you need to stop a job do the following:

  1. Get the process id

    Use the top command to list the jobs on the server. You should see your username next to any jobs you have on the server. The display looks something like this:

          PID   USER     PRI  NI  SIZE  RSS SHARE STAT %CPU %MEM   TIME CPU COMMAND
          20509 kakoi     25   0 68348  12M 10700 R    36.5  0.3 131:51   0 MATLAB
          20852 kakoi     25   0 80236  58M 11252 R    30.6  1.4 123:13   0 MATLAB
          21112 kakoi     25   0 76088  54M 11212 R    29.6  1.4 119:36   0 MATLAB
          21920 jfrommer  25   0 81904  74M 15676 R    19.7  1.9 101:33   0 MATLAB
          1     root      15   0   512  464   440 S     0.0  0.0   0:05   0 init
          2     root      15   0     0    0     0 SW    0.0  0.0   0:00   0 keventd
          

    The left hand column lists the process id (PID) number for the jobs.

    Say you are jfrommer and would like to kill your job because the data it is dumping is wrong and the run is useless. The PID for it is 21920.

  2. Kill the job

    Now that you have the pid for the job now it's time to stop it. This rather straightforward. Run the command kill -15 21920. This will "politely" stop the job. If that doesn't work after once or twice we need to get mean.

    Run the command kill -9 21920. This should do the trick.

Things to be careful of

  • Creating/Edit documents with Windows editors

    Windows text files are a little different in that they have an extra character at the end of each line. So, unless you edit the files on Linux/Unix you will need to run the the dos2unix command to strip out those extra characters.

  • Unix vs. Windows paths

    Make sure that you use unix paths when running on the servers.

Also, here's a very crude, but nifty example of a startup.m script if you want to load up some stuff in matlab before your script runs:

  startup.m (for matlab 6.5.1)
  ================================
  os = regexp(pwd,{':'})
  if os == 2
      disp('In Windows');
      addpath N:\Personal\RTBP_models N:\Personal\RTBP_models\Functions
      cd N:\Personal\RTBP_models
  else
      disp('In *nix');
      addpath /home/roger/a/cwagner/Personal/RTBP_models /home/roger/a/cwagner/Personal/RTBP_models/Functions
      cd /home/roger/a/cwagner/Personal/RTBP_models
  end
  ================================

  startup.m (for matlab 7)
  ================================
  match = regexp(pwd,{':'})
  os = cell2mat(match)
  if os == 2
      disp('In Windows');
      addpath N:\Personal\RTBP_models N:\Personal\RTBP_models\Functions
      cd N:\Personal\RTBP_models
  else
      disp('In *nix');
      addpath /home/roger/a/cwagner/Personal/RTBP_models /home/roger/a/cwagner/Personal/RTBP_models/Functions
      cd /home/roger/a/cwagner/Personal/RTBP_models
  end
  ================================

If you get a buffer overflow error

Matlab bug

Summary
Redirecting an M-file into matlab -nodisplay can cause "Type-ahead Buffer Overflow" errors

Description

One way to have MATLAB execute an M-file on a UNIX system is to redirect that M-file into MATLAB with the command

matlab -nodisplay < myMfile.m

at the shell prompt.

If myMfile.m contains a call to the DCT function waitForState, it is possible that you will see "Type-ahead Buffer Overflow" error messages in the standard output stream from MATLAB.

Workaround

If you start MATLAB with the -r flag, which also executes the specified M-file, rather than use input redirection, you should not see this behavior. For example,

matlab -nodisplay -r myMfile

Note that you don't add the file extension (if there is one). So if the file name is MyFile?.m, you don't have the .m when using the -r option.


comments:

... --joseph.r.kline.1, Fri, 16 Apr 2004 11:08:22 -0500 reply