Regarding parallelization of exciting code calculation

exciting
1

Hii users, I have installed exciting code on a server. There are files in the bin folder exciting, exciting_smp, and exciting_mpismp. I have to run EXECUTE-single.sh k444 command.
for this i add command in my file
module load exciting-neon/v21

exciting_mpismp -np 4 /home/apps/exciting/tools/tutorial_scripts/EXECUTE-single.sh k444
but its still showing running on 1 processors in INFO.out file and taking a lot of time.
Kindly help me how can i run it fastly.

2

Hi Sakshi,

The command you’re running doesn’t make any sense.
Without calling mpirun, a parallel binary will run with a single
process. exciting_mpismp is a binary, not an mpi job launcher.

You should just call

export OMP_NUM_THREADS=<n_threads>
./EXECUTE-single.sh k444

where n_threads is the total available to you.

Or if you want MPI and OMP support, edit these lines of the bash file:

Line 4: EXECUTABLE=$EXCITINGROOT/bin/exciting_mpismp
Line: 47: time mpirun -np 4 $EXECUTABLE | tee output.screen

There’s also no reason to be putting run commands in a module
file. Modules for setting up environment variables, such that
exciting is in the PATH, for example.

Cheers,
Alex

3

Thanku abuccheri. I am running now with following commands
#!/bin/bash
#SBATCH -N 4
#SBATCH --ntasks-per-node=48
module load compiler/intel/2018_4 gnu8/8.3.0
module load exciting-neon/v21
time mpirun -np 40 /home/apps/exciting/bin/exciting_mpismp
But its still showing
MPI version using 1 processor(s)
| | using MPI-2 features in INFO.out file.

The calculation is running but its giving some error, i have attached below error files.
Also, what is the use of EXECUTE-single.sh k444 command if calculation is running without it.
I am doing MOKE calculations of a 2d material, input.xml is also attached. its taking too much time is something wrong in my file?
input.xml (1.9 KB)
job.555290.err (18.2 KB)
job.555290.out (30.6 KB)

4

Hii abuccheri I
I try to change the following commands in EXECUTE-single.sh
Or if you want MPI and OMP support, edit these lines of the bash file:

Line 4: EXECUTABLE=$EXCITINGROOT/bin/exciting_mpismp
Line: 47: time mpirun -np 4 $EXECUTABLE | tee output.screen

but its showing permission denied. I can not create bashrc file as i m running exciting on a cluster.
please help me how can i do it.

5

In response to the first question, there are some inconsistencies in this slurm script:

  • You’re asking for 4 nodes with 48 MPI processes per node, but you’re running exciting with a total of 40 MPI processes - most of the processes will just idle.
  • You should run the MPI command like:
    mpirun -np $SLURM_NTASKS /home/apps/exciting/bin/exciting_mpismp
    (no need to time)
  • Note, you do not allocate any resources for threading, or set OMP_NUM_THREADS
  • There also shouldn’t be a reason to load both intel and GNU, unless you’ve compiled with Intel plus non-Intel MPI library (not recommended). Use Intel OR Gfortran

If you check the job*.out file, it’s telling you the problem:

No PMIx server was reachable, but a PMI1/2 was detected.
If srun is being used to launch application, 40 singletons will be started.

This implies an issue with your environment. You should speak to your system admin, or a fellow user for help on how to run MPI programs on the cluster.

W.r.t. using the bash script, it’s only meant for tutorial purposes. Many tutorials have migrated to Jupyter notebooks, which make the commands more transparent.

W.r.t. editing the bash file, why would you create a bashrc? I’m just talking about editing the shell script with a text editor, for example:

emacs EXECUTE-single.sh
or
vim EXECUTE-single.sh

If you can’t edit this script, it means the file permissions are wrong. Changing file permissions and submitting slurm jobs are best informed by someone sitting next to you. I recommend trying to take an input from an exciting example (or its test farm) and submitting that first, as one knows the expected output.

Cheers,
Alex

6

Thank you very much.