Nohup problem with MPI and remote ssh

English Applications
1

I am trying to run an application on multiple processors using openMPI. The command is as follows:

$mpirun -np 2 <executable>

For all those unfamiliar with the mpirun-command: -np specifies the number of processors to be used.
From time to time I log in using a remote machine and ssh. I can start my mpi-process normally. The downside is that I have to keep my remote terminal open and my remote machine running until the process is finished. I’d like to be able to log off and shut down the remote machine though.

I tried using the nohup-command as follows:

$nohup mpirun -np 2 <executable> > run.out &

Which starts the process just fine (and writes the output to <run.out>). Once I close the terminal thought, the process is killed despite the ‘nohup’ command.

I also tried

$nohup bash -c “mpirun -np 2 <executable> > run.out” &

and

$nohup ksh -c “mpirun -np 2 <executable> > run.out” &

Unfortunately without success.
Can anyone help with this?
Thanks!

2

Strange, nohup should work.

In any case, try GNU Screen instead. Here’s a cheat sheet. To emulate nohup, use:

$ screen
% mpirun ...
% Ctrl-A D
$ exit
3

Indeed strange.

Can you check, after you started and before you log out, if the parent PID of the mpirun program is 1? (it should be).

ps -ef | grep mpirun
4

The $screen approach works fine.

When I start the mpi-process with $mpirun I get
[1] <process ID>

Does that help?
As the screen-command works fine for me, I can close this thread.

5

It should help you, not me.
I did suggest a statement to you. When you have questions about the output of it, I am willing to look into it, but then you must:
a) execute it
b) post the output here (between CODE tags!).

But as you are satisfied this is a bit academic now.

6

Dear Henk,
yes here is the result of the statement in my case. What should I do?

$ nohup mpirun -np 4 ./c.e &
[1] 19473
$ nohup: ignorando entrada e anexando saída a “nohup.out”
$ ps -ef |grep mpirun
daniel 19473 18924 0 16:09 pts/11 00:00:00 mpirun -np 4 ./c.e
daniel 19536 18924 0 16:10 pts/11 00:00:00 grep --color mpirun
$

Thank you very much,
Daniel