Failed to compile wannier_tools with arpack-ng on Ubuntu 20.04.

[hongyi-zhao]

On Ubuntu 20.04, I try to compile git master version of wannier_tools which requires arpack library for cooperation with the intel MPI and mkl. I tried with the following steps but failed:

$ module load parallel_studio_xe/2020.1.102
$ git clone https://github.com/opencollab/arpack-ng.git opencollab/arpack-ng.git
$ cd opencollab/arpack-ng.git/
$ ./bootstrap
$ F77=mpiifort F90=mpiifort FC=mpiifort CC=mpiicc CXX=mpiicpc FFLAGS=-O3 FCFLAGS=-O3 CFLAGS=-O3 CXXFLAGS=-O3 ./configure --enable-mpi --enable-static
$ make -j44

$ find . -type f | egrep 'libp?arpack.a' | xargs realpath
/home/werner/Public/repo/github.com/opencollab/arpack-ng.git/SRC/.libs/libarpack.a
/home/werner/Public/repo/github.com/opencollab/arpack-ng.git/PARPACK/SRC/MPI/.libs/libparpack.a

Then I try to use either of the above generated static library to compile wannier_tools, but failed with the following info:

$  module load parallel_studio_xe/2020.1.102
$ git clone https://github.com/quanshengwu/wannier_tools.git wannier_tools.git 
$ cd wannier_tools.git/src

When using libparpack.a:

$ make -f Makefile.intel-mpi 
mpiifort -fpp -DMPI -DINTELMKL -fpe3 -O3 -static-intel module.o sparse.o wt_aux.o math_lib.o symmetry.o readHmnR.o inverse.o proteus.o eigen.o ham_qlayer2qlayer.o psi.o unfolding.o rand.o ham_slab.o ham_bulk.o ek_slab.o ek_bulk_polar.o ek_bulk.o readinput.o fermisurface.o surfgreen.o surfstat.o mat_mul.o ham_ribbon.o ek_ribbon.o fermiarc.o berrycurvature.o wanniercenter.o dos.o  orbital_momenta.o landau_level_sparse.o landau_level.o lanczos_sparse.o berry.o wanniercenter_adaptive.o effective_mass.o findnodes.o sigma_OHE.o sigma.o sigma_AHC.o main.o -o wt.x -Wl,--start-group /opt/intel/compilers_and_libraries_2020.1.217/linux/mkl/lib/intel64/libmkl_intel_lp64.a /opt/intel/compilers_and_libraries_2020.1.217/linux/mkl/lib/intel64/libmkl_sequential.a /opt/intel/compilers_and_libraries_2020.1.217/linux/mkl/lib/intel64/libmkl_core.a -Wl,--end-group -lpthread -lm -ldl /home/werner/Public/repo/github.com/opencollab/arpack-ng.git/PARPACK/SRC/MPI/.libs/libparpack.a
ld: sparse.o: in function `sparse_mp_zmat_arpack_zndrv2_':
sparse.f90:(.text+0xf74e): undefined reference to `znaupd_'
ld: sparse.f90:(.text+0xfb85): undefined reference to `zneupd_'
ld: sparse.o: in function `sparse_mp_zmat_arpack_zndrv1_':
sparse.f90:(.text+0x12f4c): undefined reference to `znaupd_'
ld: sparse.f90:(.text+0x1319b): undefined reference to `zneupd_'
make: *** [../../Makefile.intel-mpi:51: main] Error 1

When using libarpack.a:

$ make -f Makefile.intel-mpi 
mpiifort -fpp -DMPI -DINTELMKL -fpe3 -O3 -static-intel module.o sparse.o wt_aux.o math_lib.o symmetry.o readHmnR.o inverse.o proteus.o eigen.o ham_qlayer2qlayer.o psi.o unfolding.o rand.o ham_slab.o ham_bulk.o ek_slab.o ek_bulk_polar.o ek_bulk.o readinput.o fermisurface.o surfgreen.o surfstat.o mat_mul.o ham_ribbon.o ek_ribbon.o fermiarc.o berrycurvature.o wanniercenter.o dos.o  orbital_momenta.o landau_level_sparse.o landau_level.o lanczos_sparse.o berry.o wanniercenter_adaptive.o effective_mass.o findnodes.o sigma_OHE.o sigma.o sigma_AHC.o main.o -o wt.x -Wl,--start-group /opt/intel/compilers_and_libraries_2020.1.217/linux/mkl/lib/intel64/libmkl_intel_lp64.a /opt/intel/compilers_and_libraries_2020.1.217/linux/mkl/lib/intel64/libmkl_sequential.a /opt/intel/compilers_and_libraries_2020.1.217/linux/mkl/lib/intel64/libmkl_core.a -Wl,--end-group -lpthread -lm -ldl /home/werner/Public/repo/github.com/opencollab/arpack-ng.git/SRC/.libs/libarpack.a
ld: /home/werner/Public/repo/github.com/opencollab/arpack-ng.git/SRC/.libs/libarpack.a(znaupd.o): in function `znaupd_':
znaupd.f:(.text+0x211): undefined reference to `_gfortran_st_write'
ld: znaupd.f:(.text+0x219): undefined reference to `_gfortran_st_write_done'
ld: znaupd.f:(.text+0x248): undefined reference to `_gfortran_st_write'
ld: znaupd.f:(.text+0x25c): undefined reference to `_gfortran_transfer_integer_write'
ld: znaupd.f:(.text+0x270): undefined reference to `_gfortran_transfer_integer_write'
ld: znaupd.f:(.text+0x284): undefined reference to `_gfortran_transfer_integer_write'
ld: znaupd.f:(.text+0x298): undefined reference to `_gfortran_transfer_integer_write'
ld: znaupd.f:(.text+0x2ac): undefined reference to `_gfortran_transfer_integer_write'
ld: /home/werner/Public/repo/github.com/opencollab/arpack-ng.git/SRC/.libs/libarpack.a(znaupd.o):znaupd.f:(.text+0x2c0): more undefined references to `_gfortran_transfer_integer_write' follow
ld: /home/werner/Public/repo/github.com/opencollab/arpack-ng.git/SRC/.libs/libarpack.a(znaupd.o): in function `znaupd_':
znaupd.f:(.text+0x2d4): undefined reference to `_gfortran_transfer_real_write'
ld: znaupd.f:(.text+0x2e8): undefined reference to `_gfortran_transfer_real_write'
ld: znaupd.f:(.text+0x2fc): undefined reference to `_gfortran_transfer_real_write'
ld: znaupd.f:(.text+0x310): undefined reference to `_gfortran_transfer_real_write'
ld: znaupd.f:(.text+0x324): undefined reference to `_gfortran_transfer_real_write'
ld: /home/werner/Public/repo/github.com/opencollab/arpack-ng.git/SRC/.libs/libarpack.a(znaupd.o):znaupd.f:(.text+0x338): more undefined references to `_gfortran_transfer_real_write' follow
ld: /home/werner/Public/repo/github.com/opencollab/arpack-ng.git/SRC/.libs/libarpack.a(znaupd.o): in function `znaupd_':
znaupd.f:(.text+0x3b8): undefined reference to `_gfortran_st_write_done'
ld: znaupd.f:(.text+0x796): undefined reference to `dlamch_'
ld: /home/werner/Public/repo/github.com/opencollab/arpack-ng.git/SRC/.libs/libarpack.a(zneupd.o): in function `zneupd_':
zneupd.f:(.text+0x8b): undefined reference to `dlamch_'
ld: zneupd.f:(.text+0x764): undefined reference to `zcopy_'
ld: zneupd.f:(.text+0x791): undefined reference to `zcopy_'
ld: zneupd.f:(.text+0x7bc): undefined reference to `zcopy_'
ld: zneupd.f:(.text+0xc58): undefined reference to `zcopy_'
ld: zneupd.f:(.text+0xca0): undefined reference to `zlaset_'
ld: zneupd.f:(.text+0xd01): undefined reference to `zlahqr_'
ld: zneupd.f:(.text+0xd4b): undefined reference to `zcopy_'
ld: zneupd.f:(.text+0xdde): undefined reference to `ztrsen_'
ld: zneupd.f:(.text+0xe4d): undefined reference to `zcopy_'
ld: zneupd.f:(.text+0xe9d): undefined reference to `zgeqr2_'
ld: zneupd.f:(.text+0xf08): undefined reference to `zunm2r_'
ld: zneupd.f:(.text+0xf3e): undefined reference to `zlacpy_'
ld: zneupd.f:(.text+0x1124): undefined reference to `ztrevc_'
ld: zneupd.f:(.text+0x13cc): undefined reference to `zscal_'
ld: zneupd.f:(.text+0x1412): undefined reference to `zscal_'
ld: zneupd.f:(.text+0x148e): undefined reference to `zscal_'
ld: zneupd.f:(.text+0x14b6): undefined reference to `zscal_'
ld: zneupd.f:(.text+0x1545): undefined reference to `zgeru_'
ld: zneupd.f:(.text+0x1656): undefined reference to `zcopy_'
ld: zneupd.f:(.text+0x1779): undefined reference to `zdscal_'
ld: zneupd.f:(.text+0x181f): undefined reference to `zcopy_'
ld: zneupd.f:(.text+0x1875): undefined reference to `ztrmm_'
ld: zneupd.f:(.text+0x1a27): undefined reference to `zcopy_'
ld: /home/werner/Public/repo/github.com/opencollab/arpack-ng.git/SRC/.libs/libarpack.a(ivout.o): in function `ivout_':
ivout.f:(.text+0xf2): undefined reference to `_gfortran_st_write'
ld: ivout.f:(.text+0x100): undefined reference to `_gfortran_transfer_character_write'
ld: ivout.f:(.text+0x117): undefined reference to `_gfortran_transfer_character_write'
ld: ivout.f:(.text+0x11f): undefined reference to `_gfortran_st_write_done'
ld: ivout.f:(.text+0x21c): undefined reference to `_gfortran_st_write'
ld: ivout.f:(.text+0x22e): undefined reference to `_gfortran_transfer_integer_write'
ld: ivout.f:(.text+0x23e): undefined reference to `_gfortran_transfer_integer_write'
ld: ivout.f:(.text+0x2a6): undefined reference to `_gfortran_transfer_array_write'
ld: ivout.f:(.text+0x2ae): undefined reference to `_gfortran_st_write_done'
ld: ivout.f:(.text+0x318): undefined reference to `_gfortran_st_write'
ld: ivout.f:(.text+0x320): undefined reference to `_gfortran_st_write_done'
ld: ivout.f:(.text+0x43c): undefined reference to `_gfortran_st_write'
ld: ivout.f:(.text+0x44e): undefined reference to `_gfortran_transfer_integer_write'
ld: ivout.f:(.text+0x45e): undefined reference to `_gfortran_transfer_integer_write'
ld: ivout.f:(.text+0x4c6): undefined reference to `_gfortran_transfer_array_write'
ld: ivout.f:(.text+0x4ce): undefined reference to `_gfortran_st_write_done'
ld: ivout.f:(.text+0x5cc): undefined reference to `_gfortran_st_write'
ld: ivout.f:(.text+0x5de): undefined reference to `_gfortran_transfer_integer_write'
ld: ivout.f:(.text+0x5ee): undefined reference to `_gfortran_transfer_integer_write'
ld: ivout.f:(.text+0x656): undefined reference to `_gfortran_transfer_array_write'
ld: ivout.f:(.text+0x65e): undefined reference to `_gfortran_st_write_done'
ld: ivout.f:(.text+0x77c): undefined reference to `_gfortran_st_write'
ld: ivout.f:(.text+0x78e): undefined reference to `_gfortran_transfer_integer_write'
ld: ivout.f:(.text+0x79e): undefined reference to `_gfortran_transfer_integer_write'
ld: ivout.f:(.text+0x808): undefined reference to `_gfortran_transfer_array_write'
ld: ivout.f:(.text+0x810): undefined reference to `_gfortran_st_write_done'
ld: ivout.f:(.text+0x8fc): undefined reference to `_gfortran_st_write'
ld: ivout.f:(.text+0x90e): undefined reference to `_gfortran_transfer_integer_write'
ld: ivout.f:(.text+0x91e): undefined reference to `_gfortran_transfer_integer_write'
ld: ivout.f:(.text+0x988): undefined reference to `_gfortran_transfer_array_write'
ld: ivout.f:(.text+0x990): undefined reference to `_gfortran_st_write_done'
ld: ivout.f:(.text+0xa98): undefined reference to `_gfortran_st_write'
ld: ivout.f:(.text+0xaaa): undefined reference to `_gfortran_transfer_integer_write'
ld: ivout.f:(.text+0xaba): undefined reference to `_gfortran_transfer_integer_write'
ld: ivout.f:(.text+0xb24): undefined reference to `_gfortran_transfer_array_write'
ld: ivout.f:(.text+0xb2c): undefined reference to `_gfortran_st_write_done'
ld: ivout.f:(.text+0xc34): undefined reference to `_gfortran_st_write'
ld: ivout.f:(.text+0xc46): undefined reference to `_gfortran_transfer_integer_write'
ld: ivout.f:(.text+0xc56): undefined reference to `_gfortran_transfer_integer_write'
ld: ivout.f:(.text+0xcbe): undefined reference to `_gfortran_transfer_array_write'
ld: ivout.f:(.text+0xcc6): undefined reference to `_gfortran_st_write_done'
ld: ivout.f:(.text+0xdbc): undefined reference to `_gfortran_st_write'
ld: ivout.f:(.text+0xdce): undefined reference to `_gfortran_transfer_integer_write'
ld: ivout.f:(.text+0xdde): undefined reference to `_gfortran_transfer_integer_write'
ld: ivout.f:(.text+0xe46): undefined reference to `_gfortran_transfer_array_write'
ld: ivout.f:(.text+0xe4e): undefined reference to `_gfortran_st_write_done'
ld: /home/werner/Public/repo/github.com/opencollab/arpack-ng.git/SRC/.libs/libarpack.a(zmout.o): in function `zmout_':
zmout.f:(.text+0x100): undefined reference to `_gfortran_st_write'
ld: zmout.f:(.text+0x116): undefined reference to `_gfortran_transfer_character_write'
ld: zmout.f:(.text+0x12b): undefined reference to `_gfortran_transfer_character_write'
ld: zmout.f:(.text+0x133): undefined reference to `_gfortran_st_write_done'
ld: zmout.f:(.text+0x291): undefined reference to `_gfortran_st_write'
ld: zmout.f:(.text+0x320): undefined reference to `_gfortran_transfer_array_write'
ld: zmout.f:(.text+0x335): undefined reference to `_gfortran_transfer_integer_write'
ld: zmout.f:(.text+0x358): undefined reference to `_gfortran_st_write_done'
ld: zmout.f:(.text+0x445): undefined reference to `_gfortran_st_write'
ld: zmout.f:(.text+0x45a): undefined reference to `_gfortran_transfer_integer_write'
ld: zmout.f:(.text+0x4e4): undefined reference to `_gfortran_transfer_array_write'
ld: zmout.f:(.text+0x4ec): undefined reference to `_gfortran_st_write_done'
ld: zmout.f:(.text+0x59d): undefined reference to `_gfortran_st_write'
ld: zmout.f:(.text+0x616): undefined reference to `_gfortran_transfer_array_write'
ld: zmout.f:(.text+0x628): undefined reference to `_gfortran_transfer_integer_write'
ld: zmout.f:(.text+0x630): undefined reference to `_gfortran_st_write_done'
ld: zmout.f:(.text+0x6cb): undefined reference to `_gfortran_st_write'
ld: zmout.f:(.text+0x6db): undefined reference to `_gfortran_transfer_integer_write'
ld: zmout.f:(.text+0x6eb): undefined reference to `_gfortran_transfer_complex_write'
ld: zmout.f:(.text+0x6f7): undefined reference to `_gfortran_st_write_done'
ld: zmout.f:(.text+0x78a): undefined reference to `_gfortran_st_write'
ld: zmout.f:(.text+0x792): undefined reference to `_gfortran_st_write_done'
ld: zmout.f:(.text+0x899): undefined reference to `_gfortran_st_write'
ld: zmout.f:(.text+0x930): undefined reference to `_gfortran_transfer_array_write'
ld: zmout.f:(.text+0x945): undefined reference to `_gfortran_transfer_integer_write'
ld: zmout.f:(.text+0x968): undefined reference to `_gfortran_st_write_done'
ld: zmout.f:(.text+0xa5e): undefined reference to `_gfortran_st_write'
ld: zmout.f:(.text+0xa73): undefined reference to `_gfortran_transfer_integer_write'
ld: zmout.f:(.text+0xaff): undefined reference to `_gfortran_transfer_array_write'
ld: zmout.f:(.text+0xb07): undefined reference to `_gfortran_st_write_done'
ld: zmout.f:(.text+0xda4): undefined reference to `_gfortran_st_write'
ld: zmout.f:(.text+0xe3d): undefined reference to `_gfortran_transfer_array_write'
ld: zmout.f:(.text+0xe52): undefined reference to `_gfortran_transfer_integer_write'
ld: zmout.f:(.text+0xe75): undefined reference to `_gfortran_st_write_done'
ld: zmout.f:(.text+0xf3f): undefined reference to `_gfortran_st_write'
ld: zmout.f:(.text+0xf4f): undefined reference to `_gfortran_transfer_integer_write'
ld: zmout.f:(.text+0xfcc): undefined reference to `_gfortran_transfer_array_write'
ld: zmout.f:(.text+0xfd4): undefined reference to `_gfortran_st_write_done'
ld: zmout.f:(.text+0x1104): undefined reference to `_gfortran_st_write'
ld: zmout.f:(.text+0x119d): undefined reference to `_gfortran_transfer_array_write'
ld: zmout.f:(.text+0x11b2): undefined reference to `_gfortran_transfer_integer_write'
ld: zmout.f:(.text+0x11d5): undefined reference to `_gfortran_st_write_done'
ld: zmout.f:(.text+0x1284): undefined reference to `_gfortran_st_write'
ld: zmout.f:(.text+0x1294): undefined reference to `_gfortran_transfer_integer_write'
ld: zmout.f:(.text+0x1311): undefined reference to `_gfortran_transfer_array_write'
ld: zmout.f:(.text+0x1319): undefined reference to `_gfortran_st_write_done'
ld: zmout.f:(.text+0x14a2): undefined reference to `_gfortran_st_write'
ld: zmout.f:(.text+0x1536): undefined reference to `_gfortran_transfer_array_write'
ld: zmout.f:(.text+0x154b): undefined reference to `_gfortran_transfer_integer_write'
ld: zmout.f:(.text+0x156e): undefined reference to `_gfortran_st_write_done'
ld: zmout.f:(.text+0x1656): undefined reference to `_gfortran_st_write'
ld: zmout.f:(.text+0x166b): undefined reference to `_gfortran_transfer_integer_write'
ld: zmout.f:(.text+0x16f5): undefined reference to `_gfortran_transfer_array_write'
ld: zmout.f:(.text+0x16fd): undefined reference to `_gfortran_st_write_done'
ld: zmout.f:(.text+0x17d4): undefined reference to `_gfortran_st_write'
ld: zmout.f:(.text+0x186d): undefined reference to `_gfortran_transfer_array_write'
ld: zmout.f:(.text+0x1882): undefined reference to `_gfortran_transfer_integer_write'
ld: zmout.f:(.text+0x18a5): undefined reference to `_gfortran_st_write_done'
ld: zmout.f:(.text+0x196f): undefined reference to `_gfortran_st_write'
ld: zmout.f:(.text+0x197f): undefined reference to `_gfortran_transfer_integer_write'
ld: zmout.f:(.text+0x19fc): undefined reference to `_gfortran_transfer_array_write'
ld: zmout.f:(.text+0x1a04): undefined reference to `_gfortran_st_write_done'
ld: zmout.f:(.text+0x1b34): undefined reference to `_gfortran_st_write'
ld: zmout.f:(.text+0x1bcd): undefined reference to `_gfortran_transfer_array_write'
ld: zmout.f:(.text+0x1be2): undefined reference to `_gfortran_transfer_integer_write'
ld: zmout.f:(.text+0x1c05): undefined reference to `_gfortran_st_write_done'
ld: zmout.f:(.text+0x1cb4): undefined reference to `_gfortran_st_write'
ld: zmout.f:(.text+0x1cc4): undefined reference to `_gfortran_transfer_integer_write'
ld: zmout.f:(.text+0x1d41): undefined reference to `_gfortran_transfer_array_write'
ld: zmout.f:(.text+0x1d49): undefined reference to `_gfortran_st_write_done'
ld: zmout.f:(.text+0x1dec): undefined reference to `_gfortran_st_write'
ld: zmout.f:(.text+0x1dfc): undefined reference to `_gfortran_transfer_integer_write'
ld: zmout.f:(.text+0x1e79): undefined reference to `_gfortran_transfer_array_write'
ld: zmout.f:(.text+0x1e81): undefined reference to `_gfortran_st_write_done'
ld: zmout.f:(.text+0x1efb): undefined reference to `_gfortran_st_write'
ld: zmout.f:(.text+0x1f0b): undefined reference to `_gfortran_transfer_integer_write'
ld: zmout.f:(.text+0x1f88): undefined reference to `_gfortran_transfer_array_write'
ld: zmout.f:(.text+0x1f90): undefined reference to `_gfortran_st_write_done'
ld: /home/werner/Public/repo/github.com/opencollab/arpack-ng.git/SRC/.libs/libarpack.a(zvout.o): in function `zvout_':
zvout.f:(.text+0xf0): undefined reference to `_gfortran_st_write'
ld: zvout.f:(.text+0xff): undefined reference to `_gfortran_transfer_character_write'
ld: zvout.f:(.text+0x11b): undefined reference to `_gfortran_transfer_character_write'
ld: zvout.f:(.text+0x123): undefined reference to `_gfortran_st_write_done'
ld: zvout.f:(.text+0x20d): undefined reference to `_gfortran_st_write'
ld: zvout.f:(.text+0x21f): undefined reference to `_gfortran_transfer_integer_write'
ld: zvout.f:(.text+0x231): undefined reference to `_gfortran_transfer_integer_write'
ld: zvout.f:(.text+0x2a5): undefined reference to `_gfortran_transfer_array_write'
ld: zvout.f:(.text+0x2ad): undefined reference to `_gfortran_st_write_done'
ld: zvout.f:(.text+0x31b): undefined reference to `_gfortran_st_write'
ld: zvout.f:(.text+0x323): undefined reference to `_gfortran_st_write_done'
ld: zvout.f:(.text+0x43d): undefined reference to `_gfortran_st_write'
ld: zvout.f:(.text+0x44f): undefined reference to `_gfortran_transfer_integer_write'
ld: zvout.f:(.text+0x461): undefined reference to `_gfortran_transfer_integer_write'
ld: zvout.f:(.text+0x4d5): undefined reference to `_gfortran_transfer_array_write'
ld: zvout.f:(.text+0x4dd): undefined reference to `_gfortran_st_write_done'
ld: zvout.f:(.text+0x53b): undefined reference to `_gfortran_st_write'
ld: zvout.f:(.text+0x54d): undefined reference to `_gfortran_transfer_integer_write'
ld: zvout.f:(.text+0x55f): undefined reference to `_gfortran_transfer_integer_write'
ld: zvout.f:(.text+0x5fb): undefined reference to `_gfortran_st_write'
ld: zvout.f:(.text+0x60d): undefined reference to `_gfortran_transfer_integer_write'
ld: zvout.f:(.text+0x61f): undefined reference to `_gfortran_transfer_integer_write'
ld: zvout.f:(.text+0x73d): undefined reference to `_gfortran_st_write'
ld: zvout.f:(.text+0x74f): undefined reference to `_gfortran_transfer_integer_write'
ld: zvout.f:(.text+0x761): undefined reference to `_gfortran_transfer_integer_write'
ld: zvout.f:(.text+0x7d7): undefined reference to `_gfortran_transfer_array_write'
ld: zvout.f:(.text+0x7df): undefined reference to `_gfortran_st_write_done'
ld: zvout.f:(.text+0x894): undefined reference to `_gfortran_st_write'
ld: zvout.f:(.text+0x8a4): undefined reference to `_gfortran_transfer_integer_write'
ld: zvout.f:(.text+0x8b6): undefined reference to `_gfortran_transfer_integer_write'
ld: zvout.f:(.text+0x912): undefined reference to `_gfortran_transfer_array_write'
ld: zvout.f:(.text+0x91a): undefined reference to `_gfortran_st_write_done'
ld: zvout.f:(.text+0x9bd): undefined reference to `_gfortran_st_write'
ld: zvout.f:(.text+0x9cd): undefined reference to `_gfortran_transfer_integer_write'
ld: zvout.f:(.text+0x9df): undefined reference to `_gfortran_transfer_integer_write'
ld: zvout.f:(.text+0xac4): undefined reference to `_gfortran_st_write'
ld: zvout.f:(.text+0xad4): undefined reference to `_gfortran_transfer_integer_write'
ld: zvout.f:(.text+0xae6): undefined reference to `_gfortran_transfer_integer_write'
ld: zvout.f:(.text+0xb42): undefined reference to `_gfortran_transfer_array_write'
ld: zvout.f:(.text+0xb4a): undefined reference to `_gfortran_st_write_done'
ld: zvout.f:(.text+0xbee): undefined reference to `_gfortran_st_write'
ld: zvout.f:(.text+0xbfe): undefined reference to `_gfortran_transfer_integer_write'
ld: zvout.f:(.text+0xc10): undefined reference to `_gfortran_transfer_integer_write'
ld: zvout.f:(.text+0xca3): undefined reference to `_gfortran_st_write'
ld: zvout.f:(.text+0xcb5): undefined reference to `_gfortran_transfer_integer_write'
ld: zvout.f:(.text+0xcc7): undefined reference to `_gfortran_transfer_integer_write'
ld: zvout.f:(.text+0xd73): undefined reference to `_gfortran_st_write'
ld: zvout.f:(.text+0xd85): undefined reference to `_gfortran_transfer_integer_write'
ld: zvout.f:(.text+0xd97): undefined reference to `_gfortran_transfer_integer_write'
ld: zvout.f:(.text+0xe7c): undefined reference to `_gfortran_st_write'
ld: zvout.f:(.text+0xe8c): undefined reference to `_gfortran_transfer_integer_write'
ld: zvout.f:(.text+0xe9e): undefined reference to `_gfortran_transfer_integer_write'
ld: zvout.f:(.text+0xefa): undefined reference to `_gfortran_transfer_array_write'
ld: zvout.f:(.text+0xf02): undefined reference to `_gfortran_st_write_done'
ld: zvout.f:(.text+0xfa5): undefined reference to `_gfortran_st_write'
ld: zvout.f:(.text+0xfb5): undefined reference to `_gfortran_transfer_integer_write'
ld: zvout.f:(.text+0xfc7): undefined reference to `_gfortran_transfer_integer_write'
ld: zvout.f:(.text+0x1055): undefined reference to `_gfortran_st_write'
ld: zvout.f:(.text+0x1067): undefined reference to `_gfortran_transfer_integer_write'
ld: zvout.f:(.text+0x1079): undefined reference to `_gfortran_transfer_integer_write'
ld: zvout.f:(.text+0x117e): undefined reference to `_gfortran_st_write'
ld: zvout.f:(.text+0x1190): undefined reference to `_gfortran_transfer_integer_write'
ld: zvout.f:(.text+0x11a2): undefined reference to `_gfortran_transfer_integer_write'
ld: zvout.f:(.text+0x1210): undefined reference to `_gfortran_transfer_array_write'
ld: zvout.f:(.text+0x1218): undefined reference to `_gfortran_st_write_done'
ld: zvout.f:(.text+0x1272): undefined reference to `_gfortran_st_write'
ld: zvout.f:(.text+0x1284): undefined reference to `_gfortran_transfer_integer_write'
ld: zvout.f:(.text+0x1296): undefined reference to `_gfortran_transfer_integer_write'
ld: zvout.f:(.text+0x132d): undefined reference to `_gfortran_st_write'
ld: zvout.f:(.text+0x133f): undefined reference to `_gfortran_transfer_integer_write'
ld: zvout.f:(.text+0x1351): undefined reference to `_gfortran_transfer_integer_write'
ld: zvout.f:(.text+0x13f8): undefined reference to `_gfortran_st_write'
ld: zvout.f:(.text+0x1408): undefined reference to `_gfortran_transfer_integer_write'
ld: zvout.f:(.text+0x1418): undefined reference to `_gfortran_transfer_integer_write'
ld: zvout.f:(.text+0x1428): undefined reference to `_gfortran_transfer_complex_write'
ld: zvout.f:(.text+0x1430): undefined reference to `_gfortran_st_write_done'
ld: /home/werner/Public/repo/github.com/opencollab/arpack-ng.git/SRC/.libs/libarpack.a(znaup2.o): in function `znaup2_':
znaup2.f:(.text+0x125): undefined reference to `zcopy_'
ld: znaup2.f:(.text+0x374): undefined reference to `dlamch_'
ld: znaup2.f:(.text+0x7c7): undefined reference to `zcopy_'
ld: znaup2.f:(.text+0x8a6): undefined reference to `zcopy_'
ld: znaup2.f:(.text+0x90c): undefined reference to `zcopy_'
ld: znaup2.f:(.text+0x94c): undefined reference to `zcopy_'
ld: /home/werner/Public/repo/github.com/opencollab/arpack-ng.git/SRC/.libs/libarpack.a(zneigh.o): in function `zneigh_':
zneigh.f:(.text+0x81): undefined reference to `zlacpy_'
ld: zneigh.f:(.text+0xb2): undefined reference to `zlaset_'
ld: zneigh.f:(.text+0xe9): undefined reference to `zlahqr_'
ld: zneigh.f:(.text+0x13e): undefined reference to `zcopy_'
ld: zneigh.f:(.text+0x1af): undefined reference to `ztrevc_'
ld: zneigh.f:(.text+0x24c): undefined reference to `zdscal_'
ld: zneigh.f:(.text+0x299): undefined reference to `zcopy_'
ld: zneigh.f:(.text+0x2b0): undefined reference to `zdscal_'
ld: zneigh.f:(.text+0x338): undefined reference to `zlacpy_'
ld: zneigh.f:(.text+0x36d): undefined reference to `zlaset_'
ld: zneigh.f:(.text+0x3a3): undefined reference to `zlahqr_'
ld: zneigh.f:(.text+0x3e4): undefined reference to `zcopy_'
ld: zneigh.f:(.text+0x427): undefined reference to `zcopy_'
ld: zneigh.f:(.text+0x478): undefined reference to `zcopy_'
ld: zneigh.f:(.text+0x48f): undefined reference to `zdscal_'
ld: /home/werner/Public/repo/github.com/opencollab/arpack-ng.git/SRC/.libs/libarpack.a(zgetv0.o): in function `zgetv0_':
zgetv0.f:(.text+0x118): undefined reference to `zgemv_'
ld: zgetv0.f:(.text+0x17b): undefined reference to `zgemv_'
ld: zgetv0.f:(.text+0x60b): undefined reference to `zlarnv_'
ld: zgetv0.f:(.text+0x66d): undefined reference to `zcopy_'
ld: zgetv0.f:(.text+0x6df): undefined reference to `zcopy_'
ld: zgetv0.f:(.text+0x705): undefined reference to `zcopy_'
ld: zgetv0.f:(.text+0x74a): undefined reference to `zcopy_'
ld: zgetv0.f:(.text+0x782): undefined reference to `zcopy_'
ld: /home/werner/Public/repo/github.com/opencollab/arpack-ng.git/SRC/.libs/libarpack.a(zgetv0.o):zgetv0.f:(.text+0x7a9): more undefined references to `zcopy_' follow
ld: /home/werner/Public/repo/github.com/opencollab/arpack-ng.git/SRC/.libs/libarpack.a(dvout.o): in function `dvout_':
dvout.f:(.text+0xf2): undefined reference to `_gfortran_st_write'
ld: dvout.f:(.text+0x100): undefined reference to `_gfortran_transfer_character_write'
ld: dvout.f:(.text+0x117): undefined reference to `_gfortran_transfer_character_write'
ld: dvout.f:(.text+0x11f): undefined reference to `_gfortran_st_write_done'
ld: dvout.f:(.text+0x21c): undefined reference to `_gfortran_st_write'
ld: dvout.f:(.text+0x22e): undefined reference to `_gfortran_transfer_integer_write'
ld: dvout.f:(.text+0x240): undefined reference to `_gfortran_transfer_integer_write'
ld: dvout.f:(.text+0x2a5): undefined reference to `_gfortran_transfer_array_write'
ld: dvout.f:(.text+0x2ad): undefined reference to `_gfortran_st_write_done'
ld: dvout.f:(.text+0x317): undefined reference to `_gfortran_st_write'
ld: dvout.f:(.text+0x31f): undefined reference to `_gfortran_st_write_done'
ld: dvout.f:(.text+0x438): undefined reference to `_gfortran_st_write'
ld: dvout.f:(.text+0x44a): undefined reference to `_gfortran_transfer_integer_write'
ld: dvout.f:(.text+0x45c): undefined reference to `_gfortran_transfer_integer_write'
ld: dvout.f:(.text+0x4c1): undefined reference to `_gfortran_transfer_array_write'
ld: dvout.f:(.text+0x4c9): undefined reference to `_gfortran_st_write_done'
ld: dvout.f:(.text+0x5bc): undefined reference to `_gfortran_st_write'
ld: dvout.f:(.text+0x5ce): undefined reference to `_gfortran_transfer_integer_write'
ld: dvout.f:(.text+0x5e0): undefined reference to `_gfortran_transfer_integer_write'
ld: dvout.f:(.text+0x645): undefined reference to `_gfortran_transfer_array_write'
ld: dvout.f:(.text+0x64d): undefined reference to `_gfortran_st_write_done'
ld: dvout.f:(.text+0x754): undefined reference to `_gfortran_st_write'
ld: dvout.f:(.text+0x766): undefined reference to `_gfortran_transfer_integer_write'
ld: dvout.f:(.text+0x778): undefined reference to `_gfortran_transfer_integer_write'
ld: dvout.f:(.text+0x7df): undefined reference to `_gfortran_transfer_array_write'
ld: dvout.f:(.text+0x7e7): undefined reference to `_gfortran_st_write_done'
ld: dvout.f:(.text+0x8d8): undefined reference to `_gfortran_st_write'
ld: dvout.f:(.text+0x8ea): undefined reference to `_gfortran_transfer_integer_write'
ld: dvout.f:(.text+0x8fc): undefined reference to `_gfortran_transfer_integer_write'
ld: dvout.f:(.text+0x963): undefined reference to `_gfortran_transfer_array_write'
ld: dvout.f:(.text+0x96b): undefined reference to `_gfortran_st_write_done'
ld: dvout.f:(.text+0xa7c): undefined reference to `_gfortran_st_write'
ld: dvout.f:(.text+0xa8e): undefined reference to `_gfortran_transfer_integer_write'
ld: dvout.f:(.text+0xaa0): undefined reference to `_gfortran_transfer_integer_write'
ld: dvout.f:(.text+0xb07): undefined reference to `_gfortran_transfer_array_write'
ld: dvout.f:(.text+0xb0f): undefined reference to `_gfortran_st_write_done'
ld: dvout.f:(.text+0xbf8): undefined reference to `_gfortran_st_write'
ld: dvout.f:(.text+0xc0a): undefined reference to `_gfortran_transfer_integer_write'
ld: dvout.f:(.text+0xc1c): undefined reference to `_gfortran_transfer_integer_write'
ld: dvout.f:(.text+0xc81): undefined reference to `_gfortran_transfer_array_write'
ld: dvout.f:(.text+0xc89): undefined reference to `_gfortran_st_write_done'
ld: dvout.f:(.text+0xd6c): undefined reference to `_gfortran_st_write'
ld: dvout.f:(.text+0xd7e): undefined reference to `_gfortran_transfer_integer_write'
ld: dvout.f:(.text+0xd90): undefined reference to `_gfortran_transfer_integer_write'
ld: dvout.f:(.text+0xdf5): undefined reference to `_gfortran_transfer_array_write'
ld: dvout.f:(.text+0xdfd): undefined reference to `_gfortran_st_write_done'
ld: /home/werner/Public/repo/github.com/opencollab/arpack-ng.git/SRC/.libs/libarpack.a(znaitr.o): in function `znaitr_':
znaitr.f:(.text+0x11e): undefined reference to `zgemv_'
ld: znaitr.f:(.text+0x17a): undefined reference to `zgemv_'
ld: znaitr.f:(.text+0x2ba): undefined reference to `zgemv_'
ld: znaitr.f:(.text+0x30c): undefined reference to `zgemv_'
ld: znaitr.f:(.text+0x55b): undefined reference to `zcopy_'
ld: znaitr.f:(.text+0x5d3): undefined reference to `zlascl_'
ld: znaitr.f:(.text+0x61b): undefined reference to `zlascl_'
ld: znaitr.f:(.text+0x681): undefined reference to `zcopy_'
ld: znaitr.f:(.text+0x72b): undefined reference to `zcopy_'
ld: znaitr.f:(.text+0x8cd): undefined reference to `dlamch_'
ld: znaitr.f:(.text+0x914): undefined reference to `dlabad_'
ld: znaitr.f:(.text+0x925): undefined reference to `dlamch_'
ld: znaitr.f:(.text+0x9b9): undefined reference to `zdscal_'
ld: znaitr.f:(.text+0x9e2): undefined reference to `zdscal_'
ld: znaitr.f:(.text+0xd14): undefined reference to `zlanhs_'
ld: znaitr.f:(.text+0x1064): undefined reference to `zcopy_'
ld: znaitr.f:(.text+0x1135): undefined reference to `zcopy_'
ld: znaitr.f:(.text+0x121f): undefined reference to `zcopy_'
ld: znaitr.f:(.text+0x125d): undefined reference to `zcopy_'
ld: /home/werner/Public/repo/github.com/opencollab/arpack-ng.git/SRC/.libs/libarpack.a(znapps.o): in function `znapps_':
znapps.f:(.text+0xe8): undefined reference to `zlaset_'
ld: znapps.f:(.text+0x352): undefined reference to `zlanhs_'
ld: znapps.f:(.text+0x4da): undefined reference to `zgemv_'
ld: znapps.f:(.text+0x61e): undefined reference to `zgemv_'
ld: znapps.f:(.text+0x636): undefined reference to `zcopy_'
ld: znapps.f:(.text+0x6b6): undefined reference to `zlacpy_'
ld: znapps.f:(.text+0x72d): undefined reference to `zcopy_'
ld: znapps.f:(.text+0x774): undefined reference to `zscal_'
ld: znapps.f:(.text+0xa93): undefined reference to `zlartg_'
ld: znapps.f:(.text+0x11db): undefined reference to `zlartg_'
ld: znapps.f:(.text+0x160c): undefined reference to `zscal_'
ld: znapps.f:(.text+0x1646): undefined reference to `zscal_'
ld: znapps.f:(.text+0x167f): undefined reference to `zscal_'
ld: znapps.f:(.text+0x179e): undefined reference to `zlanhs_'
ld: znapps.f:(.text+0x184e): undefined reference to `dlamch_'
ld: znapps.f:(.text+0x1895): undefined reference to `dlabad_'
ld: znapps.f:(.text+0x18a6): undefined reference to `dlamch_'
make: *** [../../Makefile.intel-mpi:51: main] Error 1

Any hints for solving this problem will be highly appreciated.

Regards,
HY

[RyanBernX]

1. PARPACK requires ARPACK. So libarpack.a or libarpack.so is needed in your first example.
2. undefined reference to _gfortran_xxx indicates that you are missing -lgfortran when linking.
3. Since you are using static libraries, the order of the libraries matters when linking. Try to put libarpack.a and/or libparpack.a before the intel MKL library group to see whether it works (keep in mind that libparpack.a should precede libarpack.a).

[hongyi-zhao]

@RyanBernX Thanks a lot. Based on your valuable instructions, I found out the following workable linking libraries setting in Makefile.intel-mpi for compiling wannier_tools:

ARPACK=/home/werner/Public/repo/github.com/opencollab/arpack-ng.git/SRC/.libs/libarpack.a

# blas and lapack libraries
# static linking
LIBS =-lgfortran ${ARPACK} -Wl,--start-group ${MKLROOT}/lib/intel64/libmkl_intel_lp64.a \
        ${MKLROOT}/lib/intel64/libmkl_sequential.a \
        ${MKLROOT}/lib/intel64/libmkl_core.a -Wl,--end-group -lpthread -lm -ldl

or

ARPACK=/home/werner/Public/repo/github.com/opencollab/arpack-ng.git/PARPACK/SRC/MPI/.libs/libparpack.a /home/werner/Public/repo/github.com/opencollab/arpack-ng.git/SRC/.libs/libarpack.a

# blas and lapack libraries
# static linking
LIBS =-lgfortran ${ARPACK} -Wl,--start-group ${MKLROOT}/lib/intel64/libmkl_intel_lp64.a \
        ${MKLROOT}/lib/intel64/libmkl_sequential.a \
        ${MKLROOT}/lib/intel64/libmkl_core.a -Wl,--end-group -lpthread -lm -ldl

But I still can't figure out why arpack-ng still need to be called with -lgfortran. As I've told, the arpack-ng library is compiled with intel tool chains with the following configuration:

$ F77=mpiifort F90=mpiifort FC=mpiifort CC=mpiicc CXX=mpiicpc FFLAGS=-O3 FCFLAGS=-O3 CFLAGS=-O3 CXXFLAGS=-O3 ./configure --enable-mpi --enable-static

Regards,
HY

[fghoussen]

Intel tool chain can not be tested here : it's not open source.
Intel open-sourced partially MKL so it's only possible to test linking with MKL and/or MKL-ILP64

arpack-ng/.travis.yml

Line 229 in 0a655d9

- stage: debian_interface64

As a starting point, did you compile OK with gnu compilers ?

[hongyi-zhao]

Yes, see following:

# The git commit hash of the repo for my case:
$ git log -1
commit 0a655d918d558ef97881db2760668b8d1e3b4bc1 (HEAD -> master, origin/master, origin/HEAD)
Author: Haoyang Liu <[email protected]>
Date:   Fri Apr 2 23:44:53 2021 +0800

    [TravisCI] Add support for centos:7 (#304)
    
    * CI: add support for centos:7
    * add wrapper for centos builds

$ ./bootstrap 
$ ./configure --enable-mpi --enable-static
[...]
--------------------------------------------------
Configuration summary for ARPACK-NG 3.9.0
--------------------------------------------------
Installation prefix : /usr/local
MPI enabled         : yes
ICB enabled         : no
INTERFACE64         : 0
F77                 : mpif77
FFLAGS              : -g -O2
FC                  : gfortran
FCFLAGS             : -g -O2
CC                  : gcc
CFLAGS              : -g -O2
CXX                 : g++
CXXFLAGS            : -g -O2
CPPFLAGS            :  -Dsgemm=sgemm_ -Dsnaupd=snaupd_ -Dsneupd=sneupd_ -Ddnaupd=dnaupd_ -Ddneupd=dneupd_ -Dcheev=cheev_
MPI_Fortran_LIBS    : -lmpi
MPI_C_LIBS          : 
MPI_CXX_LIBS        : 
BLAS                : -lopenblas
LAPACK              : 
EIGEN               : 
LIBS                : 
LDADD               : 
--------------------------------------------------
Configuration OK
--------------------------------------------------
$ make -j44

[fghoussen]

So it's Intel and/or MKL specific problem. Following exactly https://software.intel.com/content/www/us/en/develop/tools/oneapi/components/onemkl/link-line-advisor.html may help. You may try with cmake too.

[hongyi-zhao]

So it's Intel and/or MKL specific problem. Following exactly https://software.intel.com/content/www/us/en/develop/tools/oneapi/components/onemkl/link-line-advisor.html may help.

It generated the following link line and compiler options:

Use this link line:

-Wl,--start-group ${MKLROOT}/lib/intel64/libmkl_intel_ilp64.a ${MKLROOT}/lib/intel64/libmkl_sequential.a ${MKLROOT}/lib/intel64/libmkl_core.a -Wl,--end-group -lpthread -lm -ldl

Compiler options:

-i8 -I"${MKLROOT}/include"

So, I tried with the following configurations:

$ F77=ifort FC=ifort FCFLAGS="-i8 -I${MKLROOT}/include" LIBS="-Wl,--start-group ${MKLROOT}/lib/intel64/libmkl_intel_ilp64.a ${MKLROOT}/lib/intel64/libmkl_sequential.a ${MKLROOT}/lib/intel64/libmkl_core.a -Wl,--end-group -lpthread -lm -ldl" ./configure --enable-mpi --enable-static 
[...]
--------------------------------------------------
Configuration summary for ARPACK-NG 3.9.0
--------------------------------------------------
Installation prefix : /usr/local
MPI enabled         : yes
ICB enabled         : no
INTERFACE64         : 0
F77                 : mpif77
FFLAGS              : -g
FC                  : ifort
FCFLAGS             : -i8 -I/opt/intel/compilers_and_libraries_2020.1.217/linux/mkl/include
CC                  : gcc
CFLAGS              : -g -O2
CXX                 : g++
CXXFLAGS            : -g -O2
CPPFLAGS            :  -Dsgemm=sgemm_ -Dsnaupd=snaupd_ -Dsneupd=sneupd_ -Ddnaupd=dnaupd_ -Ddneupd=dneupd_ -Dcheev=cheev_
MPI_Fortran_LIBS    : -lmpi
MPI_C_LIBS          : 
MPI_CXX_LIBS        : 
BLAS                : 
LAPACK              : 
EIGEN               : 
LIBS                : -Wl,--start-group /opt/intel/compilers_and_libraries_2020.1.217/linux/mkl/lib/intel64/libmkl_intel_ilp64.a /opt/intel/compilers_and_libraries_2020.1.217/linux/mkl/lib/intel64/libmkl_sequential.a /opt/intel/compilers_and_libraries_2020.1.217/linux/mkl/lib/intel64/libmkl_core.a -Wl,--end-group -lpthread -lm -ldl
LDADD               : 
--------------------------------------------------
Configuration OK
--------------------------------------------------
$ make -j44

And the above method can also compile arpack-ng successfully. But I still need -lgfortran to compile wannier_tools, otherwise the compilation will complain with the similar errors.

You may try with cmake too.

At the bottom, cmake will call make, if I remember correctly. So, I don't think there is essential difference when make works.

[fghoussen]

cmake may end-up with different compile / link options than configure. AFAIR, when using intel, you need to make sure your env is intel-compliant : some scripts (ifortvars.sh or similar) must be ran before using intel compilers.

[RyanBernX]

@hongyi-zhao
Try this:

F77=ifort FC=ifort CC=icc CXX=icpc MPIF77=mpiifort LIBS="-Wl,--start-group ${MKLROOT}/lib/intel64/libmkl_intel_lp64.a ${MKLROOT}/lib/intel64/libmkl_sequential.a ${MKLROOT}/lib/intel64/libmkl_core.a -Wl,--end-group -lpthread -lm -ldl" ../configure --enable-mpi --enable-static

I believe there is an issue with mpif77 so you should replace it with mpiifort, too.

BTW, libmkl_intel_lp64.a (32-bit interface) should be used, instead of libmkl_intel_ilp64.a, since you are not enabling INTERFACE64.

You can verify the compilation with ldd SRC/.libs/libarpack.so. Normally there wouldn't be libgfortran.so in the output.

[hongyi-zhao]

@hongyi-zhao
Try this:

F77=ifort FC=ifort CC=icc CXX=icpc MPIF77=mpiifort LIBS="-Wl,--start-group ${MKLROOT}/lib/intel64/libmkl_intel_lp64.a ${MKLROOT}/lib/intel64/libmkl_sequential.a ${MKLROOT}/lib/intel64/libmkl_core.a -Wl,--end-group -lpthread -lm -ldl" ../configure --enable-mpi --enable-static

I believe there is an issue with mpif77 so you should replace it with mpiifort, too.

Wonderful analysis. It does the trick. The detailed configuration results are shown below:

$ F77=ifort FC=ifort CC=icc CXX=icpc MPIF77=mpiifort LIBS="-Wl,--start-group ${MKLROOT}/lib/intel64/libmkl_intel_lp64.a ${MKLROOT}/lib/intel64/libmkl_sequential.a ${MKLROOT}/lib/intel64/libmkl_core.a -Wl,--end-group -lpthread -lm -ldl" ./configure --enable-mpi --enable-static
[...]
--------------------------------------------------
Configuration summary for ARPACK-NG 3.9.0
--------------------------------------------------
Installation prefix : /usr/local
MPI enabled         : yes
ICB enabled         : no
INTERFACE64         : 0
F77                 : mpiifort
FFLAGS              : -g
FC                  : ifort
FCFLAGS             : -g
CC                  : icc
CFLAGS              : -g -O2
CXX                 : icpc
CXXFLAGS            : -g -O2
CPPFLAGS            :  -Dsgemm=sgemm_ -Dsnaupd=snaupd_ -Dsneupd=sneupd_ -Ddnaupd=dnaupd_ -Ddneupd=dneupd_ -Dcheev=cheev_
MPI_Fortran_LIBS    : -lmpi
MPI_C_LIBS          : 
MPI_CXX_LIBS        : 
BLAS                : 
LAPACK              : 
EIGEN               : 
LIBS                : -Wl,--start-group /opt/intel/compilers_and_libraries_2020.1.217/linux/mkl/lib/intel64/libmkl_intel_lp64.a /opt/intel/compilers_and_libraries_2020.1.217/linux/mkl/lib/intel64/libmkl_sequential.a /opt/intel/compilers_and_libraries_2020.1.217/linux/mkl/lib/intel64/libmkl_core.a -Wl,--end-group -lpthread -lm -ldl
LDADD               : 
--------------------------------------------------
Configuration OK
--------------------------------------------------

Why don't you use the following option suggested by the MKL Link Line Advisor:

FCFLAGS="-i8 -I${MKLROOT}/include"

BTW, libmkl_intel_lp64.a (32-bit interface) should be used, instead of libmkl_intel_ilp64.a, since you are not enabling INTERFACE64.

How to enable it, and should I enable it or not?

You can verify the compilation with ldd SRC/.libs/libarpack.so. Normally there wouldn't be libgfortran.so in the output.

Yep. As shown below:

$ ldd SRC/.libs/libarpack.so |grep -i fortran |wc
      0       0       0

[hongyi-zhao]

cmake may end-up with different compile / link options than configure. AFAIR, when using intel, you need to make sure your env is intel-compliant : some scripts (ifortvars.sh or similar) must be ran before using intel compilers.

I use lmod to manage the env, and the corresponding env setting for intel toolchain is generated from the intel official shipped script. So, it's unlikely that this will cause the problem for my case.

[RyanBernX]

Why don't you use the following option suggested by the MKL Link Line Advisor:

1. ARPACK only requires external libs to provide BLAS/LAPACK functionality. It does not need the headers in MKL.
2. i8 seems to be an option to cope with 64-bit integer interface. However, we are compiling with 32-bit libs.

How to enable it,

Add INTERFACE64=1 before configure to enable ILP64.

and should I enable it or not?

It depends on other libs in your projects. ARPACK supports both LP64 (32-bit integer) and ILP64 (64-bit integer) version of BLAS/LAPACK. However, some libraries may only support one of them. Either version can be used as long as it is consistent across your project.

[hongyi-zhao]

@RyanBernX Thank you very much for your in-depth explanation. Based on my rough intuition, if we use ILP64, then does it mean that LP64 will also be supported?

[RyanBernX]

does it mean that LP64 will also be supported?

Not sure what you are asking. As long as the BLAS/LAPACK functions are declared properly, it is OK to pass 32-bit integers to the ILP64 interface (but you are still using ILP64, not LP64. The compiler will perform implicit conversion for you). What you can't do is to declare an ILP64 function in your header and link with an LP64 library.

See this link for more details about LP64/ILP64.

[hongyi-zhao]

Not sure what you are asking.

I mean the backward compatibility, i.e., in this scenario, ILP64 should be compatible with LP64.

It seems that LP64/ILP64 belong to 64-bit data models. But I still can't figure out what terms they're abbreviated from.

[hongyi-zhao]

cmake may end-up with different compile / link options than configure. AFAIR, when using intel, you need to make sure your env is intel-compliant : some scripts (ifortvars.sh or similar) must be ran before using intel compilers.

I tried to compile with cmake, but failed, as shown below:

$ module load parallel_studio_xe/2020.1.102
$ mkdir build && cd $_
$ cmake -DCMAKE_Fortran_COMPILER=ifort ..
-- The C compiler identification is GNU 9.3.0
-- The Fortran compiler identification is Intel 19.1.0.20200306
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working Fortran compiler: /opt/intel/compilers_and_libraries_2020.1.217/linux/bin/intel64/ifort
-- Check for working Fortran compiler: /opt/intel/compilers_and_libraries_2020.1.217/linux/bin/intel64/ifort  -- works
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Checking whether /opt/intel/compilers_and_libraries_2020.1.217/linux/bin/intel64/ifort supports Fortran 90
-- Checking whether /opt/intel/compilers_and_libraries_2020.1.217/linux/bin/intel64/ifort supports Fortran 90 -- yes
-- Detecting Fortran/C Interface
-- Detecting Fortran/C Interface - Found GLOBAL and MODULE mangling
-- Verifying Fortran/C Compiler Compatibility
-- Verifying Fortran/C Compiler Compatibility - Failed
CMake Error at /usr/share/cmake-3.16/Modules/FortranCInterface.cmake:383 (message):
  The Fortran compiler:

    /opt/intel/compilers_and_libraries_2020.1.217/linux/bin/intel64/ifort

  and the C compiler:

    /usr/bin/cc

  failed to compile a simple test project using both languages.  The output
  was:

    Change Dir: /home/werner/Public/repo/github.com/opencollab/arpack-ng.git/build/CMakeFiles/FortranCInterface/VerifyC
    
    Run Build Command(s):/usr/bin/make VerifyFortranC && /usr/bin/cmake -S/usr/share/cmake-3.16/Modules/FortranCInterface/Verify -B/home/werner/Public/repo/github.com/opencollab/arpack-ng.git/build/CMakeFiles/FortranCInterface/VerifyC --check-build-system CMakeFiles/Makefile.cmake 0
    /usr/bin/make -f CMakeFiles/Makefile2 VerifyFortranC
    make[1]: Entering directory '/home/werner/Public/repo/github.com/opencollab/arpack-ng.git/build/CMakeFiles/FortranCInterface/VerifyC'
    /usr/bin/cmake -S/usr/share/cmake-3.16/Modules/FortranCInterface/Verify -B/home/werner/Public/repo/github.com/opencollab/arpack-ng.git/build/CMakeFiles/FortranCInterface/VerifyC --check-build-system CMakeFiles/Makefile.cmake 0
    /usr/bin/cmake -E cmake_progress_start /home/werner/Public/repo/github.com/opencollab/arpack-ng.git/build/CMakeFiles/FortranCInterface/VerifyC/CMakeFiles 5
    /usr/bin/make -f CMakeFiles/Makefile2 CMakeFiles/VerifyFortranC.dir/all
    make[2]: Entering directory '/home/werner/Public/repo/github.com/opencollab/arpack-ng.git/build/CMakeFiles/FortranCInterface/VerifyC'
    /usr/bin/make -f CMakeFiles/VerifyFortran.dir/build.make CMakeFiles/VerifyFortran.dir/depend
    make[3]: Entering directory '/home/werner/Public/repo/github.com/opencollab/arpack-ng.git/build/CMakeFiles/FortranCInterface/VerifyC'
    cd /home/werner/Public/repo/github.com/opencollab/arpack-ng.git/build/CMakeFiles/FortranCInterface/VerifyC && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /usr/share/cmake-3.16/Modules/FortranCInterface/Verify /usr/share/cmake-3.16/Modules/FortranCInterface/Verify /home/werner/Public/repo/github.com/opencollab/arpack-ng.git/build/CMakeFiles/FortranCInterface/VerifyC /home/werner/Public/repo/github.com/opencollab/arpack-ng.git/build/CMakeFiles/FortranCInterface/VerifyC /home/werner/Public/repo/github.com/opencollab/arpack-ng.git/build/CMakeFiles/FortranCInterface/VerifyC/CMakeFiles/VerifyFortran.dir/DependInfo.cmake
    Scanning dependencies of target VerifyFortran
    make[3]: Leaving directory '/home/werner/Public/repo/github.com/opencollab/arpack-ng.git/build/CMakeFiles/FortranCInterface/VerifyC'
    /usr/bin/make -f CMakeFiles/VerifyFortran.dir/build.make CMakeFiles/VerifyFortran.dir/build
    make[3]: Entering directory '/home/werner/Public/repo/github.com/opencollab/arpack-ng.git/build/CMakeFiles/FortranCInterface/VerifyC'
    [ 20%] Building Fortran object CMakeFiles/VerifyFortran.dir/VerifyFortran.f.o
    /opt/intel/compilers_and_libraries_2020.1.217/linux/bin/intel64/ifort  -I/home/werner/Public/repo/github.com/opencollab/arpack-ng.git/build/CMakeFiles/FortranCInterface/VerifyC  -fpp -extend-source -O3   -c /usr/share/cmake-3.16/Modules/FortranCInterface/Verify/VerifyFortran.f -o CMakeFiles/VerifyFortran.dir/VerifyFortran.f.o
    [ 40%] Linking Fortran static library libVerifyFortran.a
    /usr/bin/cmake -P CMakeFiles/VerifyFortran.dir/cmake_clean_target.cmake
    /usr/bin/cmake -E cmake_link_script CMakeFiles/VerifyFortran.dir/link.txt --verbose=1
    /usr/bin/ar qc libVerifyFortran.a  CMakeFiles/VerifyFortran.dir/VerifyFortran.f.o
    /usr/bin/ranlib libVerifyFortran.a
    make[3]: Leaving directory '/home/werner/Public/repo/github.com/opencollab/arpack-ng.git/build/CMakeFiles/FortranCInterface/VerifyC'
    [ 40%] Built target VerifyFortran
    /usr/bin/make -f CMakeFiles/VerifyFortranC.dir/build.make CMakeFiles/VerifyFortranC.dir/depend
    make[3]: Entering directory '/home/werner/Public/repo/github.com/opencollab/arpack-ng.git/build/CMakeFiles/FortranCInterface/VerifyC'
    cd /home/werner/Public/repo/github.com/opencollab/arpack-ng.git/build/CMakeFiles/FortranCInterface/VerifyC && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /usr/share/cmake-3.16/Modules/FortranCInterface/Verify /usr/share/cmake-3.16/Modules/FortranCInterface/Verify /home/werner/Public/repo/github.com/opencollab/arpack-ng.git/build/CMakeFiles/FortranCInterface/VerifyC /home/werner/Public/repo/github.com/opencollab/arpack-ng.git/build/CMakeFiles/FortranCInterface/VerifyC /home/werner/Public/repo/github.com/opencollab/arpack-ng.git/build/CMakeFiles/FortranCInterface/VerifyC/CMakeFiles/VerifyFortranC.dir/DependInfo.cmake
    Scanning dependencies of target VerifyFortranC
    make[3]: Leaving directory '/home/werner/Public/repo/github.com/opencollab/arpack-ng.git/build/CMakeFiles/FortranCInterface/VerifyC'
    /usr/bin/make -f CMakeFiles/VerifyFortranC.dir/build.make CMakeFiles/VerifyFortranC.dir/build
    make[3]: Entering directory '/home/werner/Public/repo/github.com/opencollab/arpack-ng.git/build/CMakeFiles/FortranCInterface/VerifyC'
    [ 60%] Building C object CMakeFiles/VerifyFortranC.dir/main.c.o
    /usr/bin/cc  -I/home/werner/Public/repo/github.com/opencollab/arpack-ng.git/build/CMakeFiles/FortranCInterface/VerifyC  -Dsgemm=sgemm -Dsnaupd=snaupd -Dsneupd=sneupd -Ddnaupd=dnaupd -Ddneupd=dneupd -Dcheev=cheev -DINCLUDE_FCMANGLE -O3 -DNDEBUG   -o CMakeFiles/VerifyFortranC.dir/main.c.o   -c /usr/share/cmake-3.16/Modules/FortranCInterface/Verify/main.c
    [ 80%] Building C object CMakeFiles/VerifyFortranC.dir/VerifyC.c.o
    /usr/bin/cc  -I/home/werner/Public/repo/github.com/opencollab/arpack-ng.git/build/CMakeFiles/FortranCInterface/VerifyC  -Dsgemm=sgemm -Dsnaupd=snaupd -Dsneupd=sneupd -Ddnaupd=dnaupd -Ddneupd=dneupd -Dcheev=cheev -DINCLUDE_FCMANGLE -O3 -DNDEBUG   -o CMakeFiles/VerifyFortranC.dir/VerifyC.c.o   -c /usr/share/cmake-3.16/Modules/FortranCInterface/Verify/VerifyC.c
    [100%] Linking C executable VerifyFortranC
    /usr/bin/cmake -E cmake_link_script CMakeFiles/VerifyFortranC.dir/link.txt --verbose=1
    /usr/bin/cc  -Dsgemm=sgemm -Dsnaupd=snaupd -Dsneupd=sneupd -Ddnaupd=dnaupd -Ddneupd=dneupd -Dcheev=cheev -DINCLUDE_FCMANGLE -O3 -DNDEBUG   CMakeFiles/VerifyFortranC.dir/main.c.o CMakeFiles/VerifyFortranC.dir/VerifyC.c.o  -o VerifyFortranC   -L/opt/intel/compilers_and_libraries_2020.1.217/linux/tbb/lib/intel64_lin/gcc4.4  libVerifyFortran.a -lifport -lifcoremt -limf -lsvml -lm -lipgo -lirc -lpthread -lsvml -lirc_s -ldl 
    /usr/bin/ld: libVerifyFortran.a(VerifyFortran.f.o): relocation R_X86_64_32 against `.rodata' can not be used when making a PIE object; recompile with -fPIE
    collect2: error: ld returned 1 exit status
    make[3]: *** [CMakeFiles/VerifyFortranC.dir/build.make:103: VerifyFortranC] Error 1
    make[3]: Leaving directory '/home/werner/Public/repo/github.com/opencollab/arpack-ng.git/build/CMakeFiles/FortranCInterface/VerifyC'
    make[2]: *** [CMakeFiles/Makefile2:81: CMakeFiles/VerifyFortranC.dir/all] Error 2
    make[2]: Leaving directory '/home/werner/Public/repo/github.com/opencollab/arpack-ng.git/build/CMakeFiles/FortranCInterface/VerifyC'
    make[1]: *** [CMakeFiles/Makefile2:88: CMakeFiles/VerifyFortranC.dir/rule] Error 2
    make[1]: Leaving directory '/home/werner/Public/repo/github.com/opencollab/arpack-ng.git/build/CMakeFiles/FortranCInterface/VerifyC'
    make: *** [Makefile:121: VerifyFortranC] Error 2
    
    
Call Stack (most recent call first):
  CMakeLists.txt:208 (FortranCInterface_VERIFY)


-- Configuring incomplete, errors occurred!
See also "/home/werner/Public/repo/github.com/opencollab/arpack-ng.git/build/CMakeFiles/CMakeOutput.log".
See also "/home/werner/Public/repo/github.com/opencollab/arpack-ng.git/build/CMakeFiles/CMakeError.log".

[hongyi-zhao]

1. ARPACK only requires external libs to provide BLAS/LAPACK functionality. It does not need the headers in MKL.

Even so, I don't think the compiler options below should trigger any errors, rather, they can be added to avoid errors in case they are needed.

For ILP64 case, i.e., setting INTERFACE64=1 before configure:

FCFLAGS="-i8 -I${MKLROOT}/include"

For LP64 case, i.e., setting INTERFACE64=0, which is the default setting of ARPACK, before configure:

FCFLAGS="-I${MKLROOT}/include"

[RyanBernX]

Again, it depends on how you compile and link your program.

Consider the following function:

double cblas_ddot (const MKL_INT n, const double *x, const MKL_INT incx, const double *y, const MKL_INT incy);

In ILP64 model, MKL_INT stands for 64-bit integers (aka int64), while in LP64 model, MKL_INT stands for 32-bit integers (aka int32). There is a macro that controls what exactly MKL_INT is. To use the ILP64 model in MKL, you must do both of the followings:

1. Add -DMKL_ILP64 to the compiler options.
2. Link with -lmkl_intel_ilp64.

If you by mistake link with -lmkl_intel_lp64, your program will still compile. However, the results may be incorrect or a segmentation fault will occur eventually.

Libraries such as ARPACK require an external BLAS/LAPACK implementation. However, they have no idea what implementation you are providing (e.g. MKL, openblas, atlas, etc). Fortunately, all BLAS/LAPACK implementations have the same interface except the integer model. Therefore, for C programs, your can provide your own declarations of BLAS/LAPACK functions in your header, as long as they are consistent with the actual libraries used for linking. I don't know much about FORTRAN. But I guess there will be similar issues.

I mean the backward compatibility

I think you probably mean this:

// suppose: long is 64-bit, int is 32-bit
double cblas_ddot (const long n, const double *x, const long incx, const double *y, const long incy);

void my_func(){
    int n = 10, inc = 1; // NOTE: the type of n and inc is int, not long
    double *x, *y;

    // allocation of x and y

    cblas_ddot(n, x, inc, y, inc); // implicit conversion
}

The above program can compile and run correctly, as long as you provide an ILP64 model of BLAS.

Hope this will answer your question.

[hongyi-zhao]

Again, thank you very much for clarifying any misunderstanding or conceptual confusion I may have had.

1. Add `-DMKL_ILP64` to the compiler options.

Just an additional/complementary note excerpted from the intel's developer guide mentioned by you in previously post:

Fortran
--
Compiling for ILP64 | ifort                      -i8                      -I<mkl directory>/include                       ...


C or C++
--
Compiling for ILP64 | icc                      -DMKL_ILP64                      -I<mkl directory>/include ...

[hongyi-zhao]

Again, it depends on how you compile and link your program.

In my opinion, selecting and using the tool chain in a consistency manner, e.g., intel® oneAPI Toolkits, can greatly improve development efficiency and reduce debugging complexity.

[fghoussen]

So it's Intel and/or MKL specific problem. Following exactly https://software.intel.com/content/www/us/en/develop/tools/oneapi/components/onemkl/link-line-advisor.html may help. You may try with cmake too.

If you want to use MKL, whatever build system or Makefile you use, you need to make sure that when typing make VERBOSE=1 you get the exact same compile line than the one that the Intel MKL advisor would have advised you https://www.intel.com/content/www/us/en/developer/tools/oneapi/onemkl-link-line-advisor.html

[hongyi-zhao]

Thank you very much for this tip.