[molpro-user] Building MOLPRO 2002.6 with Infiniband/Topspin Support on EM64T

[Shenggang Li]

Hi, I have been trying to build Molpro 2002.6 with the Infiniband
support.  We have the Topspin version 3.1 software.  I managed to
build the GA toolkit version 3.4b with TCGMSG or MPICH and Topspin
libraries using the PGI compilers version 6.1-2.  Next I configured
the Molpro 2002.6 package with TCGMSG or MPICH.  The compilation went
quite smoothly, except that it failed to link.  I think it needs the
Topspin libraries in order to properly link, but I don't know how to
add them (-L/usr/local/topspin/lib64 -lvapi -lmosal -lts_ib_cm_user). 
Can anyone help me on this?  The following are the content of the
CONFIG file and the error message.  Thank you very much!
-----------------------------------------------------------------------------------------
CONFIG:
ARCHITECTURE="unix unix-i8 unix-linux unix-linux-x86_64 mpp"
# Compilers ..
CC="cc -Dpgf90 -DI64 -m64"
FC="/home/sli/apps/pgi/linux86-64/6.1/bin/pgf90 -pc 64 -i8 -Mcache_align"
F90="/home/sli/apps/pgi/linux86-64/6.1/bin/pgf90 -pc 64 -i8 -Mcache_align"
# compiler command to be used only when linking molpro.exe .. eg mpxlf on IBM SP
LINKFC="/home/sli/apps/pgi/linux86-64/6.1/bin/pgf90 -pc 64 -i8 -fastsse"
YACC="bison -b y"
ARFLAGS="-r"
# C defines
# nb    SEEK is disk seek speed in seconds
#       SPEED is disk transfer rate in 8 bytes/second
# compiler optimisation
COPT="-O3"
FOPT="-fastsse"
# compiler explicit no optimisation
CDEF="-DSEEK=.001 -DSPEED=200000   -DLARGEFILES -DZLIB
-DHAVE_INTTYPES_H   -DMA_ALLOC -DMOLPROC_PAR"
CNOPT=" "
COPT1="-O1"
COPT2="-O2"
FNOPT=" "
FOPT1="-O1"
FOPT2="-O2"
FOPT3="-O3"
# compiler debug flag
CDEBUG="-g"
FDEBUG="-g"
# static variables
FSTATIC=" "
# 64-bit integers
FI64=" "
# profiling
FPROFILE="-p"
CPROFILE="-p"
# additional pre-processor flags
FTCFLAGS="mpp eaf blas1 blas2 blas3 lapack"
# additional directories to be compiled in mpp case
MPPDIR="mpp"
# additional libraries and link options
LIBS="  -lz"
LINKOPT="-g77libs"
# non-standard location of system libraries
LIBDIR="/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64"
# GKS X-windows library
GKSLIB="-lgks0"
# BLAS library
BLASLIB="-L/opt/intel/mkl72cluster/lib/em64t -lmkl_lapack -lmkl
-lguide -lpthread"
# LAPACK library
LAPACKLIB="-L/opt/intel/mkl72cluster/lib/em64t -lmkl_lapack -lmkl
-lguide -lpthread"
# MPP?
MPP="1"
MPPX="0"
WRAPPER="/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/tcgmsg/ipcv4.0/parallel"
USE_MPI=""
USE_LAPI=""
USE_MYRINET=""
MPI_LIB=""
MPPNAME="tcgmsg"
ARCHNAME="p4"
MPITYPE=""
MPIBASEDIR=""
MYRINET_LIB=""
MYRINET_LINKPARS="-lpthread"
MPIGM_FORMAT="0"
USE_GIO="0"
# files in src/ to be compiled without optimization
F77VERSION="pgf90 6.1-2 64-bit target on x86-64 Linux "
NO_OPT="util/iow.f util/displace.f dft/dftigrad.f ccsd/uccsd.f"
F_OPT1="casvb/symut_cvb.f casvb/ci_cvb.f seward/qzit.f opt/frequencies.f"
F_OPT2=" "
F_OPT3=" "
C_OPT1=" "
C_OPT2=" "
# ranlib command (if not needed, true)
RANLIB="true"
# ls command to get user and group
LSFULL="ls -l"
# cp -p if it exists, else cp
CP="cp -p"
# tar command
TAR="tar"
# awk command
AWK="awk"
# strip command
STRIP="strip"
# installation directories
INSTBIN="/home/sli/apps/molpro-2002.6/pgi-6.1-i8-tcgmsg-mellanox-mkl/bin"
INSTLIB="/home/sli/apps/molpro-2002.6/pgi-6.1-i8-tcgmsg-mellanox-mkl/lib"
INSTHTML=""
INSTCGI=""
# makedepend configuration
MAKEDEPEND_OPTIONS=""
MODULE_SUFFIX="mod"
# non-default libraries
LIBDIR_tcgmsg=/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64
BLASLIB_p4="-L/opt/intel/mkl72cluster/lib/em64t -lmkl_lapack -lmkl
-lguide -lpthread"
LAPACKLIB_p4="-L/opt/intel/mkl72cluster/lib/em64t -lmkl_lapack -lmkl
-lguide -lpthread"
WRAPPER_tcgmsg=/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/tcgmsg/ipcv4.0/parallel
PARSE=parse-x86_64-unknown-linux-gnu-i8.o.gz
LICENCE_KEY=
----------------------------------------------------------------------------------------------
The error message:
linking /home/sli/source/molpro-2002.6-109/molpro2002.6/bin/molprop_2002_6_i8_p4_tcgmsg.exe
link failed
/home/sli/apps/pgi/linux86-64/6.1/bin/pgf90 -pc 64 -i8 -fastsse
-fastsse -g77libs -o
/home/sli/source/molpro-2002.6-109/molpro2002.6/bin/molprop_2002_6_i8_p4_tcgmsg.exe
molver.o molpro.o
-L/home/sli/source/molpro-2002.6-109/molpro2002.6/lib -lmolpro
-L/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64
-lglobal -ltcgmsg -lma -lpario -larmci -lgks0
-L/opt/intel/mkl72cluster/lib/em64t -lmkl_lapack -lmkl -lguide
-lpthread -L/opt/intel/mkl72cluster/lib/em64t -lmkl_lapack -lmkl
-lguide -lpthread
-L/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64
-lglobal -ltcgmsg -lma -lpario -larmci -lz
/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64/libarmci.a(vapi.o)(.text+0x226):
In function `armci_server_send_complete':
: undefined reference to `VAPI_poll_cq'
/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64/libarmci.a(vapi.o)(.text+0x2d6):
In function `armci_client_recv_complete':
: undefined reference to `VAPI_poll_cq'
/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64/libarmci.a(vapi.o)(.text+0x3d5):
In function `client_rmw_complete':
: undefined reference to `VAPI_poll_cq'
/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64/libarmci.a(vapi.o)(.text+0x466):
In function `armci_client_send_complete':
: undefined reference to `VAPI_poll_cq'
/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64/libarmci.a(vapi.o)(.text+0x89c):
In function `armci_create_qp':
: undefined reference to `VAPI_create_qp'
/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64/libarmci.a(vapi.o)(.text+0x940):
In function `armci_init_nic':
: undefined reference to `VAPI_open_hca'
/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64/libarmci.a(vapi.o)(.text+0x94d):
In function `armci_init_nic':
: undefined reference to `EVAPI_get_hca_hndl'
/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64/libarmci.a(vapi.o)(.text+0x977):
In function `armci_init_nic':
: undefined reference to `VAPI_query_hca_cap'
/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64/libarmci.a(vapi.o)(.text+0x99a):
In function `armci_init_nic':
: undefined reference to `VAPI_query_hca_port_prop'
/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64/libarmci.a(vapi.o)(.text+0x9cf):
In function `armci_init_nic':
: undefined reference to `VAPI_alloc_pd'
/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64/libarmci.a(vapi.o)(.text+0xa13):
In function `armci_init_nic':
: undefined reference to `VAPI_create_cq'
/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64/libarmci.a(vapi.o)(.text+0xa3f):
In function `armci_init_nic':
: undefined reference to `VAPI_create_cq'
/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64/libarmci.a(vapi.o)(.text+0xd19):
In function `armci_server_alloc_bufs':
: undefined reference to `VAPI_register_mr'
/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64/libarmci.a(vapi.o)(.text+0x1312):
In function `vapi_connect_client':
: undefined reference to `VAPI_modify_qp'
/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64/libarmci.a(vapi.o)(.text+0x142c):
In function `vapi_connect_client':
: undefined reference to `VAPI_modify_qp'
/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64/libarmci.a(vapi.o)(.text+0x14da):
In function `vapi_connect_client':
: undefined reference to `VAPI_modify_qp'
/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64/libarmci.a(vapi.o)(.text+0x1a5c):
In function `posts_scatter_desc':
: undefined reference to `VAPI_post_rr'
/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64/libarmci.a(vapi.o)(.text+0x1a8d):
In function `posts_scatter_desc':
: undefined reference to `VAPI_post_rr'
/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64/libarmci.a(vapi.o)(.text+0x28ea):
In function `armci_server_initial_connection':
: undefined reference to `VAPI_modify_qp'
/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64/libarmci.a(vapi.o)(.text+0x2a00):
In function `armci_server_initial_connection':
: undefined reference to `VAPI_modify_qp'
/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64/libarmci.a(vapi.o)(.text+0x2ab2):
In function `armci_server_initial_connection':
: undefined reference to `VAPI_modify_qp'
/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64/libarmci.a(vapi.o)(.text+0x2b3c):
In function `armci_server_initial_connection':
: undefined reference to `VAPI_post_rr'
/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64/libarmci.a(vapi.o)(.text+0x2b82):
In function `armci_server_initial_connection':
: undefined reference to `EVAPI_set_comp_eventh'
/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64/libarmci.a(vapi.o)(.text+0x2c6e):
In function `armci_call_data_server':
: undefined reference to `VAPI_poll_cq'
/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64/libarmci.a(vapi.o)(.text+0x2ca9):
In function `armci_call_data_server':
: undefined reference to `VAPI_poll_cq'
/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64/libarmci.a(vapi.o)(.text+0x2d63):
In function `armci_call_data_server':
: undefined reference to `EVAPI_poll_cq_block'
/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64/libarmci.a(vapi.o)(.text+0x30c5):
In function `armci_call_data_server':
: undefined reference to `VAPI_post_rr'
/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64/libarmci.a(vapi.o)(.text+0x3251):
In function `armci_vapi_client_mem_alloc':
: undefined reference to `VAPI_register_mr'
/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64/libarmci.a(vapi.o)(.text+0x3538):
In function `armci_vapi_post_send':
: undefined reference to `VAPI_post_sr'
/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64/libarmci.a(vapi.o)(.text+0x3546):
In function `armci_vapi_post_send':
: undefined reference to `EVAPI_post_inline_sr'
/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64/libarmci.a(vapi.o)(.text+0x3821):
In function `armci_client_direct_get':
: undefined reference to `VAPI_post_sr'
/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64/libarmci.a(vapi.o)(.text+0x3b38):
In function `armci_send_data_to_client':
: undefined reference to `VAPI_post_sr'
/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64/libarmci.a(vapi.o)(.text+0x3b6e):
In function `armci_send_data_to_client':
: undefined reference to `VAPI_poll_cq'
/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64/libarmci.a(vapi.o)(.text+0x3d6e):
In function `armci_server_register_region':
: undefined reference to `VAPI_register_mr'
/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64/libarmci.a(vapi.o)(.text+0x3e13):
In function `armci_pin_contig_hndl':
: undefined reference to `VAPI_register_mr'
/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64/libarmci.a(vapi.o)(.text+0x3f3d):
In function `posts_gather_desc':
: undefined reference to `VAPI_post_sr'
/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64/libarmci.a(vapi.o)(.text+0x3f73):
In function `posts_gather_desc':
: undefined reference to `VAPI_post_sr'
/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64/libarmci.a(vapi.o)(.text+0x41a5):
In function `armci_server_direct_send':
: undefined reference to `VAPI_post_sr'
/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64/libarmci.a(vapi.o)(.text+0x41e1):
In function `armci_server_direct_send':
: undefined reference to `VAPI_poll_cq'
/home/sli/source/ga-3.4b/g-tcgmsg-pgi-6.1-2-linux64-mellanox/lib/LINUX64/libarmci.a(vapi.o)(.text+0x4574):
In function `armci_direct_rmw':
: undefined reference to `VAPI_post_sr'
make: *** [bin/molprop_2002_6_i8_p4_tcgmsg.exe] Error 1

--
Shenggang Li
Shelby Hall L103A
Department of Chemistry
University of Alabama
Tuscaloosa, AL 35487-0336
Office: 205-348-4906