CGNS, HDF5, OpenMPI, Fortran... need help getting it configured/installed properly

English Applications
#1

What do I want to do:
I need to use CGNS in combination with HDF5. My programming language is Fortran, and I want OpenMPI for an MPI implementation. The end goal is writing (and optional reading) the CGNS databases in MPI parallel.

What I tried so far:

  1. configure/compile the the packages from source. I got as far as having a functioning Fortran compiler with OpenMPI 4.0.3. I was still able to compile the HDF5 library without errors. Not sure if the configuration was correct, hard to tell for me at this point. But then compiling the CGNS library while linking to HDF5 and OpenMPI turned out to be beyond my capabilities. Sure, I don’t really know what I’m doing, but the “documentation” full of outdated sample configuration scripts and code examples didn’t help either. After confiiguring CGNS (successfully? at least I got no errors), make falis with this message:
    …
    mpicc -O2 -I… -o cgnslist cgnslist.o getargs.o …/lib/libcgns.a /usr/local/hdf5/lib64/libhdf5.a -lm -lz
    /usr/bin/ld: /usr/local/hdf5/lib64/libhdf5.a(H5PLint.o): undefined reference to symbol ‘dlclose@@GLIBC_2.2.5’
    /usr/bin/ld: /lib64/libdl.so.2: error adding symbols: DSO missing from command line
    …

  2. tried to use the provided packages from OpenSUSE Leap 15.3. I got OpenMPI wrappers to work after reading this: https://stackoverflow.com/questions/58230700/how-to-get-mpif90-compiler-for-opensuse-leap-15-0
    Then I installed all HDF5 packages that seemed to be useful (hdf5-gnu-openmpi4-hpc, hdf5-gnu-openmpi4-hpc-devel + dependencies that were installed automatically). And finally, I installed all 4 available packages for CGNS (cgns, cgns-devel, cgns-devel-static, libcgns4_1).
    No idea if I need to take additional steps similar to sourcing /usr/lib64/mpi/gcc/openmpi4/bin/mpivars.sh for getting the MPI wrappers. But quickly trying to compile one of the serial CGNS examples from the documentation leads to errors like these:
    /usr/lib64/gcc/x86_64-suse-linux/7/…/…/…/…/x86_64-suse-linux/bin/ld: /tmp/cc1zFKeP.o: in function MAIN__': /home/as01449/development/cgns/test_cgns/write_grid_unst.f90:59: undefined reference to cg_open_f_’
    /usr/lib64/gcc/x86_64-suse-linux/7/…/…/…/…/x86_64-suse-linux/bin/ld: /home/as01449/development/cgns/test_cgns/write_grid_unst.f90:60: undefined reference to `cg_error_exit_f’
    …

Questions:
So I’m stuck again :shame:

  1. Which route do you recommend for the goal I have in mind? Can I even get there using the packages from the official repositories, or are they configured in a way that renders them useless for my purpose?
  2. And if I have to go down the “compile from source” route again, does anyone have some tips? Or would be willing to walk me through the questions and errors I get along the way. Maybe someone already had to do something similar
#2

Hi
Usually missing linker flags and order, in that first command, add -ldl after -lz

#3

Try install all of hpc patterns first:

zypper in --type pattern hpc_compute_node hpc_development_node hpc_libraries hpc_workload_server
#4

Thanks for the hint. I probably should have stressed this even more in my initial post, but I don’t know how to do that.
Currently my configuration script looks like this, how can I get -ldl in there?


#!/bin/bash

export HDF5=/usr/local/hdf5 #that's where I configured HDF5, contains bin, include, lib64
export CFLAGS=-O2
export CC=mpicc
export FC=mpifort
export F77=mpifort
export FFLAGS=-O2

./configure --verbose \
--prefix="$HOME/cgns" \
--enable-parallel \
--with-fortran \
--with-hdf5=$HDF5 \
--enable-64bit \
--disable-cgnstools \
--enable-gcc \
--enable-lfs

Try install all of hpc patterns first

Again, thanks for your time. I guess this would be a step for getting everything from the official repositories, and not for compiling the software from source?
I don’t want to mix things up here, i.e. going through compiling the packages, only to have it work by accident, because it happened to find some missing dependency -that I failed to configure properly- from one of the packages I installed via this command. I need to start learning how to to these things properly.

#5

Sorry for double posting, I can’t edit my posts after 10 minutes.
This is the output I get from running the configure script above:


configure: loading site script /usr/share/site/x86_64-unknown-linux-gnu
checking CGNS version... 4.20
checking if gcc is requested... yes
checking for gcc... mpicc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether mpicc accepts -g... yes
checking for mpicc option to accept ISO C89... none needed
checking if compiler supports prototyping... yes
checking if debugging is enabled... no
checking for ranlib... ranlib
checking for strip... strip
checking how to run the C preprocessor... mpicc -E
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for ANSI C header files... yes
checking for an ANSI C-conforming const... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for size_t... yes
checking for mpicc option to accept ISO C99... none needed
checking for Complex Gnu Extension... yes
checking for a BSD-compatible install... /usr/bin/install -c
checking where to install library... /home/as01449/cgns/lib64
checking where to install headers... /home/as01449/cgns/include
checking where to install executables... /home/as01449/cgns/bin
checking where to install Tcl scripts... /home/as01449/cgns/share/cgnstools
checking if system was specified... no
checking if 64bit support is requested... yes
checking cgsystem... LINUX64
checking size of void *... 8
checking size of char *... 8
checking if legacy build is enabled... no
checking if scope enumeration values is enabled... no
checking if base scope for family/zone ref-to enabled... no
checking size of long... 8
checking if Fortran interface is enabled... yes
checking if Fortran compiler meets 2003 standard... yes
checking if Fortran compiler meets Technical Specification 29113... yes
checking if Fortran compiler meets the Fortran 2008 Standard... yes
checking size of default Fortran integer ... c_int32_t
checking Fortran to C interface... assuming LOWERCASE_
checking for concatenation operator... yes
checking if building a shared library... no
checking if using HDF5... /usr/local/hdf5
checking HDF5 library version... 1.12.0
checking if HDF5 is 1.8 or greater... yes
checking if HDF5 library has multi-dataset read/write APIs... no
checking if HDF5 library has collective metadata APIs... yes
checking if HDF5 library has H5Pset_file_space_strategy... yes
checking if HDF5 library has compression dependencies... no
checking HDF5 library LIB dependencies...  -lz
checking if parallel IO support is requested... yes
checking prefix for running in parallel... mpiexec -n $${NPROCS:=4}
checking for ceil in -lm... yes
checking if large file support is requested... yes
checking for large file support options... none
checking size of off_t... 8
checking for stat64... no
checking if building CGNStools... no
checking whether make sets $(MAKE)... yes
checking size of void *... (cached) 8
configure: creating ./config.status
config.status: creating cgnstypes.h
config.status: creating cgnstypes_f.h
config.status: creating cgnstypes_f03.h
config.status: creating cgnsconfig.h
config.status: creating cg_hash_types.h
config.status: creating Makefile
config.status: creating make.defs
config.status: creating cgnsBuild.defs
config.status: creating tests/Makefile
config.status: creating tools/Makefile
config.status: creating ptests/Makefile
config.status: creating cgnstools/Makefile
config.status: creating cgnstools/make.defs
config.status: creating cgnstools/cgconfig

And that’s the full output from running make afterwards:


mkdir lib
mpicc -O2   -I.  -o  lib/cgns_error.o -c cgns_error.c
mpicc -O2   -I.  -I/usr/local/hdf5/include -o  lib/cg_hashmap.o -c cg_hashmap.c
mpicc -O2   -I.  -I/usr/local/hdf5/include -o  lib/cgns_internals.o -c cgns_internals.c
mpicc -O2   -I.  -o  lib/cgns_io.o -c cgns_io.c
mpicc -O2   -I.  -I/usr/local/hdf5/include  -o  lib/cgnslib.o -c cgnslib.c
mpifort -O2 -Wno-c-binding-type  -I.  -o  lib/cgns_f.o -c cgns_f.F90
mpicc -O2   -I.  -Iadfh -I/usr/local/hdf5/include  -o  lib/ADFH.o -c adfh/ADFH.c
mpicc -O2   -I.  -I/usr/local/hdf5/include  -o  lib/pcgnslib.o -c pcgnslib.c
mpicc -O2   -I.  -Iadf -o  lib/ADF_interface.o -c adf/ADF_interface.c
mpicc -O2   -I.  -Iadf -o  lib/ADF_internals.o -c adf/ADF_internals.c
mpicc -O2   -I.  -I/usr/local/hdf5/include -DLOWERCASE_ -o  lib/cg_ftoc.o -c cg_ftoc.c
mpicc -O2   -I.  -I/usr/local/hdf5/include -DLOWERCASE_ -o  lib/cgio_ftoc.o -c cgio_ftoc.c
ar cr lib/libcgns.a lib/cgns_error.o lib/cg_hashmap.o lib/cgns_internals.o lib/cgns_io.o lib/cgnslib.o lib/cgns_f.o lib/ADFH.o lib/pcgnslib.o lib/ADF_interface.o lib/ADF_internals.o lib/cg_ftoc.o lib/cgio_ftoc.o
ranlib lib/libcgns.a
cd tools && make
make[1]: Entering directory '/home/as01449/development/cgns/sourcecode/CGNS/src/tools'
mpicc -O2  -I.. -c cgnslist.c
mpicc -O2  -I.. -c getargs.c
mpicc -O2  -I.. -o  cgnslist cgnslist.o getargs.o ../lib/libcgns.a /usr/local/hdf5/lib64/libhdf5.a     -lm  -lz
/usr/bin/ld: /usr/local/hdf5/lib64/libhdf5.a(H5PLint.o): undefined reference to symbol 'dlclose@@GLIBC_2.2.5'
/usr/bin/ld: /lib64/libdl.so.2: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
make[1]: *** [Makefile:25: cgnslist] Error 1
make[1]: Leaving directory '/home/as01449/development/cgns/sourcecode/CGNS/src/tools'
make: *** [Makefile:66: tools] Error 2
#6

https://forums.opensuse.org/showthread.php/540197-error-while-loading-shared-libraries-libmpi_cxx-so-1-cannot-open-shared-object-file-No-such-filehttps://www.pgroup.com/index.htm

#7

Addendum: found something that gets me through the installation of CGNS: https://www.cfd-online.com/Forums/main/210249-build-cgns-hdf5.html
I.e. I changed the line “./configure --verbose " to “LIBS=”-ldl -lz” ./configure --verbose "

Now make seems to run without errors:


mkdir lib
mpicc -O2   -I.  -o  lib/cgns_error.o -c cgns_error.c
mpicc -O2   -I.  -I/usr/local/hdf5/include -o  lib/cg_hashmap.o -c cg_hashmap.c
mpicc -O2   -I.  -I/usr/local/hdf5/include -o  lib/cgns_internals.o -c cgns_internals.c
mpicc -O2   -I.  -o  lib/cgns_io.o -c cgns_io.c
mpicc -O2   -I.  -I/usr/local/hdf5/include  -o  lib/cgnslib.o -c cgnslib.c
mpifort -O2 -Wno-c-binding-type  -I.  -o  lib/cgns_f.o -c cgns_f.F90
mpicc -O2   -I.  -Iadfh -I/usr/local/hdf5/include  -o  lib/ADFH.o -c adfh/ADFH.c
mpicc -O2   -I.  -I/usr/local/hdf5/include  -o  lib/pcgnslib.o -c pcgnslib.c
mpicc -O2   -I.  -Iadf -o  lib/ADF_interface.o -c adf/ADF_interface.c
mpicc -O2   -I.  -Iadf -o  lib/ADF_internals.o -c adf/ADF_internals.c
mpicc -O2   -I.  -I/usr/local/hdf5/include -DLOWERCASE_ -o  lib/cg_ftoc.o -c cg_ftoc.c
mpicc -O2   -I.  -I/usr/local/hdf5/include -DLOWERCASE_ -o  lib/cgio_ftoc.o -c cgio_ftoc.c
ar cr lib/libcgns.a lib/cgns_error.o lib/cg_hashmap.o lib/cgns_internals.o lib/cgns_io.o lib/cgnslib.o lib/cgns_f.o lib/ADFH.o lib/pcgnslib.o lib/ADF_interface.o lib/ADF_internals.o lib/cg_ftoc.o lib/cgio_ftoc.o
ranlib lib/libcgns.a
cd tools && make
make[1]: Entering directory '/home/as01449/development/cgns/sourcecode/CGNS/src/tools'
mpicc -O2  -I.. -c cgnslist.c
mpicc -O2  -I.. -c getargs.c
mpicc -O2  -I.. -o  cgnslist cgnslist.o getargs.o ../lib/libcgns.a /usr/local/hdf5/lib64/libhdf5.a     -lm -ldl -lz -lz
mpicc -O2  -I.. -c cgnscheck.c
mpicc -O2  -I.. -c hash.c
mpicc -O2  -I.. -c cgnames.c
mpicc -O2  -I.. -o  cgnscheck cgnscheck.o getargs.o hash.o cgnames.o ../lib/libcgns.a /usr/local/hdf5/lib64/libhdf5.a     -lm -ldl -lz -lz
mpicc -O2  -I.. -c cgnsconvert.c
mpicc -O2  -o  cgnsconvert cgnsconvert.o getargs.o ../lib/libcgns.a /usr/local/hdf5/lib64/libhdf5.a     -lm -ldl -lz -lz
mpicc -O2  -I.. -c cgnscompress.c
mpicc -O2  -o  cgnscompress cgnscompress.o ../lib/libcgns.a /usr/local/hdf5/lib64/libhdf5.a     -lm -ldl -lz -lz
mpicc -O2  -I.. -c cgnsdiff.c
mpicc -O2  -o  cgnsdiff cgnsdiff.o getargs.o ../lib/libcgns.a /usr/local/hdf5/lib64/libhdf5.a     -lm -ldl -lz -lz
mpicc -O2  -I.. -c cgnsnames.c
mpicc -O2  -I.. -o  cgnsnames cgnsnames.o cgnames.o ../lib/libcgns.a /usr/local/hdf5/lib64/libhdf5.a     -lm -ldl -lz -lz
make[1]: Leaving directory '/home/as01449/development/cgns/sourcecode/CGNS/src/tools'

make install afterwards:


Making directory /home/as01449/cgns/include
Making directory /home/as01449/cgns/lib64
/usr/bin/install -c -m 644 pcgnslib.h /home/as01449/cgns/include/pcgnslib.h
/usr/bin/install -c -m 644 cgnstypes.h /home/as01449/cgns/include/cgnstypes.h
/usr/bin/install -c -m 644 cgnstypes_f.h /home/as01449/cgns/include/cgnstypes_f.h
/usr/bin/install -c -m 644 cgnstypes_f03.h /home/as01449/cgns/include/cgnstypes_f03.h
/usr/bin/install -c -m 644 cgnslib.h /home/as01449/cgns/include/cgnslib.h
/usr/bin/install -c -m 644 cgnswin_f.h /home/as01449/cgns/include/cgnswin_f.h
/usr/bin/install -c -m 644 cgns_io.h /home/as01449/cgns/include/cgns_io.h
/usr/bin/install -c -m 644 cgnsconfig.h /home/as01449/cgns/include/cgnsconfig.h
/usr/bin/install -c -m 644 cgnsBuild.defs /home/as01449/cgns/include/cgnsBuild.defs
/usr/bin/install -c -m 644 lib/libcgns.a /home/as01449/cgns/lib64/libcgns.a

So far, so good? I add $HOME/cgns/lib64 to LD_LIBRARY_PATH via .bashrc

But then when trying to compile a short example from the documentation, it still fails:


$ gfortran-9 -ffree-line-length-none -I $HOME/cgns/include write_grid_unst.f90
/usr/bin/ld: /tmp/cc3iIRRL.o: in function `MAIN__':
write_grid_unst.f90:(.text+0x233): undefined reference to `cg_open_f_'
/usr/bin/ld: write_grid_unst.f90:(.text+0x23f): undefined reference to `cg_error_exit_f'
/usr/bin/ld: write_grid_unst.f90:(.text+0x2c3): undefined reference to `cg_base_write_f_'
/usr/bin/ld: write_grid_unst.f90:(.text+0x377): undefined reference to `cg_zone_write_f_'
/usr/bin/ld: write_grid_unst.f90:(.text+0x3b4): undefined reference to `cg_coord_write_f_'
/usr/bin/ld: write_grid_unst.f90:(.text+0x3f1): undefined reference to `cg_coord_write_f_'
/usr/bin/ld: write_grid_unst.f90:(.text+0x42e): undefined reference to `cg_coord_write_f_'
/usr/bin/ld: write_grid_unst.f90:(.text+0x71e): undefined reference to `cg_section_write_f_'
/usr/bin/ld: write_grid_unst.f90:(.text+0x950): undefined reference to `cg_section_write_f_'
/usr/bin/ld: write_grid_unst.f90:(.text+0xb90): undefined reference to `cg_section_write_f_'
/usr/bin/ld: write_grid_unst.f90:(.text+0x1120): undefined reference to `cg_section_write_f_'
/usr/bin/ld: write_grid_unst.f90:(.text+0x1137): undefined reference to `cg_close_f'
collect2: error: ld returned 1 exit status

The linker finds the cgns library (imported via “use cgns” in the fortran source code; if I omit -I $HOME/cgns/include the compiler complains about not being able to find the module cgns.mod), but doesn’t seem to find the cgns routines/interfaces for fortran.

#8

LD_LIBRARY_PATH is for resolving shared library location at run time, not for compiling.

but doesn’t seem to find the cgns routines/interfaces for fortran.

You need to tell it where libraries are located using -L$HOME/cgns/lib64 flag just as you need to tell it where headers are located using -I flag.

#9

Again what learned as we say in Germany.

Now while your hint did not immediately fix all my programming-related problems (errors stayed the same) it might have sent me in the right direction.
I can compile a simple example now using this:


gfortran-9 -ffree-line-length-none -I $HOME/cgns/include -I /usr/local/hdf5/include -I $HOME/openmpi_403/include -c write_grid_unst.f90
gfortran-9 -o write_grid_unst write_grid_unst.o -L $HOME/cgns/lib64 -lcgns -L /usr/local/hdf5/lib64 -lhdf5 -L $HOME/openmpi_403/lib64 -lmpi

Only including the cgns library leads to errors for mission routines from HDF5. including the HDF5 library lead to errors about missing MPI routines… and lastly including the OpenMPI library gets rid of all errors.
About the second line though… was there any way for me to know that I need to add -lcgns, -lhdf5 and -lmpi? I got there with a little bit of trial&error :open_mouth:

The code runs without errors and produces a cgns database. Trying to import that into Paraview causes it to crash immediately. But that’s about what I expected from PV for my first try.