[molpro-user] ARMCI test errors after compiling with GA/MPICH2, gfortran, ATLAS on Ubuntu 11.04

[Andy May]

Greg,

Apologies for the delay, I was away last week.

The problem with Molpro/ATLAS was a compile time problem. Essentially 
Molpro should be able to be built with just the ATLAS routines, and 
should automatically build the missing LAPACK routines, thus avoiding 
the user to do this manually. This is now fixed in our repository, and 
will be pushed out with some other fixes after testing. (Some LAPACK 
routines not present in ATLAS were also not being compiled by Molpro).

I suspect the problem is that for ATLAS we assume it has been built with 
8-byte integers on 64-bit systems, and probably the ubuntu binaries are 
built with 4-byte integers.

Also Molpro configure has a dilemma when it finds libatlas and then 
liblapack because the name 'liblapack' is rather generic. We assume that 
liblapack is probably an ATLAS LAPACK library since libatlas is present, 
but it is perfectly possible liblapack could be a system one (on 
openSUSE for example liblapack is simply the netlib routines). So the 
fact your liblapack is a full LAPACK library has been ignored, and 
Molpro will have compiled the non-ATLAS routines internally (with 8-byte 
integers), which will add to the confusion.

To use ATLAS you could either:

1. Compile ATLAS by hand with 8-byte integers (this will also give you 
optimal parameters for your hardware). Then build GA again using 
--with-blas8 option, and then finally build Molpro. This will then give 
you everything with 8-byte integers.

2. Proceed as before, but after configure edit CONFIG and change the 
values of BLAS_INT and LAPACK_INT in the MACROS variable. I think this 
should work, but haven't tried it.

Best wishes,

Andy

On 23/05/11 18:23, Gregory Magoon wrote:
> I was able to narrow this down to an issue with ATLAS. Andy, could you
> elaborate
> on your previous comments regarding Molpro not working with the
> unmodified ATLAS
> libraries? Is this a compile-time or run-time issue? If it is a run-time
> issue,
> would something like the procedure described below help address the
> issue I'm
> having? (I have taken this from the ATLAS documentation):
>
> "ATLAS does not provide a full lapack library. However, there is a
> simple way
> to get ATLAS to provide its faster LAPACK routines to a full LAPACK
> library.
> ATLAS's internal routines are distinct from LAPACK's, so it is safe to
> compile
> ATLAS's LAPACK routines directly into a netlib-style LAPACK library.
> If you install LAPACK before ATLAS, this can be automated as described in
> ATLAS/INSTALL.txt. The following describes adding netlib LAPACK to an
> already built ATLAS library:
>
> First, obtain the LAPACK src from netlib and build the LAPACK library as
> normal. Then, in the lib/ subdirectory of your ATLAS build directory
> (where you should have a liblapack.a), issue the following commands:
> mkdir tmp
> cd tmp
> ar x ../liblapack.a
> cp <your LAPACK path & lib> ../liblapack.a
> ar d ../liblapack.a lsame.o xerbla.o csrot.o zdrot.o
> ar r ../liblapack.a *.o
> cd ..
> rm -rf tmp
>
> Just linking in ATLAS's liblapack.a first will not get you the best LAPACK
> performance, mainly because LAPACK's untuned ILAENV will be used instead
> of ATLAS's slightly tuned one."
>
> Thanks again,
> Greg
>
> Quoting Gregory Magoon <gmagoon at mit.edu>:
>
>> An update: although the original command you suggested didn't work, as
>> explained
>> in my earlier e-mail, I tried the same command with the -noblas and
>> -nolapack
>> options that you proposed and the tests seem to be working properly:
>>
>> ./configure -batch -gcc -gfortran -mpp -auto-ga-tcgmsg-mpich2 -instroot
>> /usr/local/molpro2010.1 -noblas -nolapack
>>
>> So, it seems our ATLAS, BLAS and/or LAPACK libraries were causing the
>> issue.
>> However, it is still not clear why. I had just installed these
>> libraries as
>> Ubuntu packages:
>> sudo apt-get install libatlas-base-dev
>> sudo apt-get install liblapack-dev
>> Perhaps there is some critical step I omitted or these are not the
>> correct
>> packages?
>>
>> Thanks again,
>> Greg
>>
>> user at kamet:~/Molpro$ ./configure -batch -gcc -gfortran -mpp
>> -auto-ga-tcgmsg-mpich2 -instroot /usr/local/molpro2010.1 -noblas
>> -nolapack
>>
>> machine type recognized as x86_64 (Generic 64-bit)
>> kernel recognized as Linux
>>
>> user request compiler gfortran
>> GNU Fortran Compiler, Version 4.5.2
>> FC=/usr/bin/gfortran
>>
>> user request compiler gcc
>> GNU Compiler Collection, Version 4.5.2
>> CC=/usr/bin/gcc
>>
>> starting auto-build of prerequisites
>>
>> building MPICH2 version 1.3.3rc1, each step could take a few minutes
>>
>> ./configure --prefix=/home/user/Molpro/src/mpich2-install
>> --with-device=ch3:nemesis --with-pm=hydra --enable-f77
>> F77=/usr/bin/gfortran
>> --enable-fc FC=/usr/bin/gfortran --enable-cc CC=/usr/bin/gcc --enable-cxx
>> CXX=/usr/bin/g++
>> make
>> make install
>> MPICH2 built, if you want to test the MPICH2 build, please run:
>>
>> cd /home/user/Molpro/src/mpich2-1.3.3rc1; make testing
>>
>> building Global Arrays version 5-0-2, each step could take a few minutes
>>
>> ./configure --prefix=/home/user/Molpro/src/ga-install --with-scalapack=no
>> --enable-f77 F77=/usr/bin/gfortran CC=/usr/bin/gcc CXX=/usr/bin/g++
>> --with-tcgmsg --with-mpi='/home/user/Molpro/src/mpich2-install/lib
>> -lmpich
>> -lopa -lmpl -lrt -I/home/user/Molpro/src/mpich2-install/include'
>> make
>> make install
>> Global Arrays built, if you want to test the Global Arrays build,
>> please run:
>>
>> cd /home/user/Molpro/src/ga-5-0-2; make checkprogs; make check
>> MPIEXEC="/home/user/Molpro/src/mpich2-install/bin/mpiexec -np 4"
>>
>> ./configure -batch "-gcc" "-gfortran" "-mpp" "-instroot"
>> "/usr/local/molpro2010.1" "-noblas" "-nolapack" -mppbase
>> /home/user/Molpro/src/ga-5-0-2
>>
>> machine type recognized as x86_64 (Generic 64-bit)
>> kernel recognized as Linux
>>
>> user request compiler gfortran
>> GNU Fortran Compiler, Version 4.5.2
>> FC=/usr/bin/gfortran
>>
>> user request compiler gcc
>> GNU Compiler Collection, Version 4.5.2
>> CC=/usr/bin/gcc
>>
>> ga_GA_MP_LIBS = -lmpich -lopa -lmpl -lrt
>> ga_GA_MP_LDFLAGS = -L/home/user/Molpro/src/mpich2-install/lib
>> ga_GA_MP_CPPFLAGS = -I/home/user/Molpro/src/mpich2-install/include
>> ga_TARGET = LINUX64
>> ga_MSG_COMMS = TCGMSGMPI
>> ga_prefix = /home/user/Molpro/src/ga-install
>> ga_ARMCI_NETWORK_LDFLAGS =
>> ga_ARMCI_NETWORK_LIBS =
>> ga_BLAS_SIZE = 4
>>
>> Use MPP library - Global Arrays version 5 or higher
>> MPPLIB=-L/home/user/Molpro/src/ga-install/lib -lga -larmci
>>
>> MPILIB=-L/home/user/Molpro/src/mpich2-install/lib -lmpich -lopa -lmpl
>> -lrt
>>
>> parallel=mpich2
>>
>> parse-Linux-x86_64-i8.o is your object
>>
>> CONFIG file created; proceed to compilation
>>
>> Quoting Andy May <MayAJ1 at cardiff.ac.uk>:
>>
>>> Greg,
>>>
>>> I can't seem to replicate this problem using options as close as
>>> possible to yours.
>>>
>>> One thing I have noticed is that linking only the routines that come
>>> with ATLAS does not work anymore with Molpro. I have fixed this, and
>>> there will be an update sometime in the future, but this leads me to
>>> conclude the -lapack you are using is either a supplemented ATLAS
>>> library, or a system one.
>>>
>>> I don't think the BLAS/LAPACK will be the problem, but perhaps
>>> configure has not correctly determined the integer size in these
>>> libraries, you can always try using Molpro internal ones with -noblas
>>> -nolapack options.
>>>
>>> There may be problems with the MPICH2 version which ships with
>>> Ubuntu, certainly there were problems with hydra around 1.3.1, but if
>>> you say that ifort works then maybe it's okay. To eliminate this I
>>> would suggest to use a newer version (no need to do this by hand, see
>>> below).
>>>
>>> Also, we are now at PL21 (there is a new tarball).
>>>
>>> Can I therefore suggest you first try the following with 2010.1.21:
>>>
>>> ./configure -batch -gcc -gfortran -mpp -auto-ga-tcgmsg-mpich2
>>> -instroot /usr/local/molpro2010.1
>>>
>>> and see if this then gives you a working build.
>>>
>>> Best wishes,
>>>
>>> Andy
>>>
>>
>>
>>
>>
>
>