[molpro-user] Problems with a large eom-ccsd calculation

Tue Apr 5 11:36:06 BST 2011

Dear Manhui,

Thanks for your response.
There is unfortunately no progress with GA. I compiled molpro
but the binary doesn't work at all, although the compilation
went OK. There is somehow again no ABDATA in lib, but it doesn't
complain to that. What it complains looks strange:

Attempting to use an MPI routine before initializing MPICH

Well I used intel MPI and I don't understand why it refers to MPICH.

Just in case my CONFIG looks the following:

# MOLPRO CONFIG generated at Di 5. Apr 10:21:18 CEST 2011, for host frbw3

CONFIGURE_OPTIONS="-i8" "-openmp" "-icc" "-ifort" "-nohdf5" "-nocuda" 
"-mpp" "-mppbase" "/bwfs/scratch/temphomes/hd/f81/ga-5-0-2-install" 
"-blaspath" "/opt/bwgrid/compiler/intel/ct_4.0/mkl/10.2.5.035/lib/em64t"

AR=ar
ARCHNAME=Linux/x86_64
ARFLAGS=-rS
AWK=awk
BIBTEX=/usr/bin/bibtex
BLASLIB=-L/opt/bwgrid/compiler/intel/ct_4.0/mkl/10.2.5.035/lib/em64t 
-lmkl_intel_ilp64 -lmkl_sequential -lmkl_core
BUILD=p
CAT=cat
CC=/opt/bwgrid/compiler/intel/ct_4.0/Compiler/11.1/072/bin/intel64/icc
CCVERSION=11.1
CC_FRONT=
CDEBUG=-g $(addprefix $(CDEFINE),_DEBUG)
CDEFINE=-D
CFLAGS=-ftz -vec-report0 -DINT64 -DZLIB -openmp
CLEARSPEEDLIB=
CMPPINCLUDE=/bwfs/scratch/temphomes/hd/f81/ga-5-0-2-install/include
COPT=-O2
COPT0=-O0
COPT1=-O1
COPT2=-O2
COPT3=-O3
CP=cp -p
CPROFILE=-p
CSCN=cscn
CSFLAGS=-O3 -I. --dynamic
CUDACC=
CUDACCVERSION=
CUDACDEBUG=-g $(addprefix $(CUDACDEFINE),_DEBUG)
CUDACDEFINE=-D
CUDACFLAGS=
CUDACOPT=
CUDACOPT0=-O0
CUDACOPT1=-O1
CUDACOPT2=-O2
CUDACOPT3=-O3
CUDACPROFILE=-p
CXX=/opt/bwgrid/compiler/intel/ct_4.0/Compiler/11.1/072/bin/intel64/icpc
CXXFLAGS=$(CFLAGS)
DOXYGEN=/usr/bin/doxygen
ECHO=echo
EXPORT=export
F90FLAGS=
FC=/opt/bwgrid/compiler/intel/ct_4.0/Compiler/11.1/072/bin/intel64/ifort
FCVERSION=11.1
FDEBUG=-g $(addprefix $(FDEFINE),_DEBUG)
FDEFINE=-D
FFLAGS=-i8 -r8 -pc64 -auto -Vaxlib -vec-report0 -cxxlib -openmp
FOPT=-O3
FOPT0=-O0
FOPT1=-O1
FOPT2=-O2
FOPT3=-O3
FPP=-fpp
FPROFILE=-p
FSTATIC=
FTCFLAGS=molpro unix unix-i8 Linux lapack mpp blas
HDF5INCLDE=
HDF5LIB=
HOSTFILE_FORMAT=
INSTBIN=/bwfs/scratch/temphomes/hd/f81/molpro2010.1-install/bin
INSTHTML=/bwfs/scratch/temphomes/hd/f81/molpro2010.1-install/html
INSTLIB=/bwfs/scratch/temphomes/hd/f81/molpro2010.1-install/lib/molprop_2010_1_Linux_x86_64_i8
INST_PL=0
INTEGER=8
LAPACKLIB=
LATEX2HTML=
LAUNCHER=/opt/bwgrid/mpi/mpiexec/0.84/bin/mpiexec -n %n %x
LD_ENV=/opt/bwgrid/compiler/intel/ct_4.0/Compiler/11.1/072/lib/intel64:/opt/bwgrid/compiler/intel/ct_4.0/mkl/10.2.5.035/lib/em64t
LD_ENVNAME=LD_LIBRARY_PATH
LIBRARY_SUFFIX=a
LIBS=-lz
LIBS_FRONT=
LINKOPT=
LINKOPT_FRONT=
LN=ln -s
MACROS=MOLPRO MOLPRO_ifort MOLPRO_f2003 GA_TOOLS GA_ GA_VERSION_GE_5 
_I8_ BLAS_INT=8 LAPACK_INT=8 MOLPRO_FORCE_VECTOR MOLPRO_NEXTSCALAR 
MOLPRO_NO_RECURRENCE MOLPRO_NOVECTOR MOLPRO_SHORTLOOP _MOLCAS_MPP_
MAKEDEPEND_OPTIONS=
MAKEINDEX=/usr/bin/makeindex
MAPLE=
MKDIR=mkdir -p
MODULE_FLAG=-I
MODULE_SUFFIX=mod
MPILIB=-L/opt/bwgrid/compiler/intel/ct_4.0/impi/4.0.0.028/intel64/lib 
-lmpi -lmpigf -lmpigi -lpthread -L/usr/lib64 -libverbs -libcommon 
-libumad -lpthread
MPPLIB=-L/bwfs/scratch/temphomes/hd/f81/ga-5-0-2-install/lib/LINUX64 
-lga -larmci
OBJECT_SUFFIX=o
OPT0=copyc6.f
OPT1=nevpt2_optrpc.f explicit_util.f artwo2.f drv2el_l3ext_lmp2g.f 
drv2el_l3ext_cen.f rmp2_f12_drv2.f90 ri_lmp2g.f df_llmp2.f readdump.f
OPT2=integrals.f90 RL4gen1.f basis_integral_shells.f
OPT3=
PAPER=a4paper
PARSE=parse-Linux-x86_64-i8.o
PDFLATEX=/usr/bin/pdflatex -halt-on-error
PNAME=molprop_2010_1_Linux_x86_64_i8
PTSIZE=11
RANLIB=ranlib
RM=rm -rf
SHELL=/bin/sh
STRIP=strip
SUFFIXES=f F f90 F90 c cpp
TAR=tar -cf
UNTAR=tar -xf
VERBOSE=@
VERSION=2010.1XSD=/usr/bin/xmllint --noout --schema
XSLT=/usr/bin/xsltproc
YACC=bison -b y

.SUFFIXES:
MAKEFLAGS+=-r
ifneq ($(LD_ENVNAME),)
$(LD_ENVNAME):=$(LD_ENV):$($(LD_ENVNAME))
endif

Best regards,
Evgeniy

Manhui Wang wrote:
> Dear Evgeniy,
> 
> I have not checked your output carefully, which shows 800MWord (6.4GB
> per process):
> 
> Variable memory set to  800000000 words,  buffer space   230000 words
> 
> I run the job  with 32 processes on 4 nodes, and it might crash
> eventually due to lack of memory you mentioned.
> 
> Could you please update your progress with GA? On my side, I will look
> into my code related to helper. It might exist a bug which is exposed
> when very large global data structure is used.
> 
> Best wishes,
> Manhui
> 
> 
> 
> Evgeniy Gromov wrote:
>> Dear Manhui,
>>
>> Many thanks for your help with the compilation problems and
>> with the problem in the large eom-ccsd calculation.
>> Sorry I forgot to tell you about the memory. I usually specify
>> the memory in the command line using -m option. For that large
>> eom-ccsd job I used 800m (-m800m) because 200m was not enough for
>> the CCSD step. As I wrote the job passed the CCSD and crashed
>> in the beginning of the EOM-CCSD. Actually, are you running it
>> on 1 CPU? I haven't tried it to run on 1 CPU as it will likely
>> take ages cause many MOs. Well I am going to rerun it using
>> a new binary compiled with GA.
>>
>> Best wishes,
>> Evgeniy
>>
>>
>> Manhui Wang wrote:
>>> Dear Evgeniy,
>>>
>>> I can't see the error you mentioned with MPI-2 Molpro2010.1 PL20 so far.
>>> I tried your input without any modification first, and it failed
>>> immediately because you have not set memory in input (the default is
>>> only 8MWord). I have added
>>>
>>> memory,200,m
>>>
>>> in input, it appears to work fine so far (still running).  Could you
>>> please try the new input on your side?
>>>
>>> Best wishes,
>>> Manhui
>>>
>>>
>>> Evgeniy Gromov wrote:
>>>> Manhui,
>>>>
>>>> I attached wrong files in the previous email. Here are the correct
>>>> input and output files. In addition I removed all the MO vectors in
>>>> the output to make the size smaller.
>>>>
>>>> Best regards,
>>>> Evgeniy
>>>>
>>>> Manhui Wang wrote:
>>>>> Dear Evgeniy,
>>>>>
>>>>> Could you please provide the inputs/outputs? I will see whether I can
>>>>> reproduce the problem firstly.
>>>>>
>>>>> Thanks,
>>>>> Manhui
>>>>>
>>>>> Evgeniy Gromov wrote:
>>>>>> Dear Molpro Community,
>>>>>>
>>>>>> I faced problems when running a large eom-ccsd calculation (466 MOs,
>>>>>> no symmetry) with molpro2010.1 in parallel (MPI-2). The calculation
>>>>>> crashed after the CCSD (ground state) step had been done. No error
>>>>>> message was present in the output. In the err file there is the
>>>>>> following diagnostic (though unclear to me):
>>>>>>
>>>>>> ERROR in twosided_helpga_locate_server: over range!
>>>>>> ilo=1,ihigh=148114575
>>>>>>
>>>>>> Any idea what can be the problem.
>>>>>>
>>>>>> Well, it seems to relate to helper process. I suspect that in the case
>>>>>> of GA it would work, although there might be some other problems.
>>>>>> Just in case the same run with 296 MO (another basis set) ran fine.
>>>>>>
>>>>>> Also, I would like to thank Manhui for the response to my previous
>>>>>> post:
>>>>>> "Is there always a helper process in parallel calculations with
>>>>>> molpro2010.1 ?"
>>>>>>
>>>>>> Best regards,
>>>>>> Evgeniy
>>>>>>
>>>>>>
>>
> 

-- 
_______________________________________
Dr. Evgeniy Gromov
Theoretische Chemie
Physikalisch-Chemisches Institut
Im Neuenheimer Feld 229
D-69120 Heidelberg
Germany

Telefon: +49/(0)6221/545263
Fax: +49/(0)6221/545221
E-mail: evgeniy at pci.uni-heidelberg.de
_______________________________________