[molpro-user] Inconsistent parralel behavior

[Hans-Joachim Werner]

The current casscf program is hardly parallelized, and this explains why there is no speedup. We are working on a new CASSCF program which is much better parallelized, but this will only be available in version 2018.1. Sorry for this.

Joachim Werner



> Am 29.10.2017 um 20:43 schrieb Fogarty, Richard M <r.fogarty14 at imperial.ac.uk>:
> 
> Dear Molpro users,
> 
> I'm having trouble running casscf calculations in parralel, though simpler calculations seem to run somewhat ok. Specifically i get some speedup for Hartree-Fock calculations with multiple cores (compared to 1 core), but no significant speedup for casscf calculations (or caspt2). 
> 
> For example here are some timings (in seconds) i get for a test HF and test-casscf job:
> CPUs Real time (HF)
>        Real time (CASSCF)
> 1 6712.52
> 2360.99 
> 4 5870.76
> 1816.12
> 8 1424.63
> 2032.02
> 12 1019.93
> 1809.47
> 16 938.01
> 2318.11
> 20 813.71
> 3847.01
> 
> So the HF job definitely gets at least some speedup, while the casscf times seem essentially random/unaffected by adding more cores. These results come from running the pre-compiled binary version on a hpc with PBS.
> 
> Any help would be greatly appreciated
> 
> Thanks
> 
> Richard
> 
> 
> 
> Relevant files are below:
> 
> The casscf input file looks like this:
> 
> "memory,627,m !627 megawords of memory (1MW~8MB)
> FILE,2,hf_ccpvtz_12c.wfu 
> GDIRECT
> symmetry,nosym
> basis={
> default=vtz
> }
> geometry={ OMITTED}
> n_frozen = 10 
> n_closed = 29 
> n_occ = 39  
> n_trip_states = 3 
> n_sing_states = 2 
> {multi;                                          
> occ,n_occ
> closed,n_closed
> frozen,n_frozen
> wf,spin=0;state,n_sing_states;
> wf,spin=2;state,n_trip_states}"
> 
> The PBS submit script is as follows
> 
> # batch processing commands
> #PBS -l walltime=59:59:00
> #PBS -lselect=1:ncpus=12:mpiprocs=12:ompthreads=1:mem=67000MB:tmpspace=400000MB
> #PBS -j oe
> #PBS -q pqmaterials
> #PBS -m n
> 
> #Get the relevant restart file in correct place
> rs_line=$(grep 'FILE,2' $PBS_O_WORKDIR/${in}) #Line with wfn file
> rs_line=${rs_line%\!*} #Remove the comment
> full_wfu_name=${rs_line##*,} #Get substring containing rs file name
> full_wfu_name=${full_wfu_name:0:32} 
> full_wfu_name="${full_wfu_name,,}" 
> cp $PBS_O_WORKDIR/$full_wfu_name $TMPDIR/.
> cp $PBS_O_WORKDIR/${in} $TMPDIR/.
> 
> #Run molpro
> cust_error_file="${in/.com/.error}"
> pbsexec /home/rf614/MolPro/bin/molpro -t 1 -n ${core} -d $TMPDIR -W $TMPDIR -v ${in} >> $TMPDIR/$cust_error_file
> 
> #cp files back
> cp $TMPDIR/*.out /$PBS_O_WORKDIR/.
> cp $TMPDIR/*.xml /$PBS_O_WORKDIR/.
> cp $TMPDIR/$cust_error_file /$PBS_O_WORKDIR/.
> 
> 
> Molpro outputs the following to $cust_error_file (This is for the 12-core case):
> 
>  # PARALLEL mode, HOST=cx1-130-5-15
>  nodelist=12
>  first   =12
>  second  =
>  third   =
>  HOSTFILE_FORMAT: $hostname
> 
> cx1-130-5-15.cx1.hpc.ic.ac.uk
> cx1-130-5-15.cx1.hpc.ic.ac.uk
> cx1-130-5-15.cx1.hpc.ic.ac.uk
> cx1-130-5-15.cx1.hpc.ic.ac.uk
> cx1-130-5-15.cx1.hpc.ic.ac.uk
> cx1-130-5-15.cx1.hpc.ic.ac.uk
> cx1-130-5-15.cx1.hpc.ic.ac.uk
> cx1-130-5-15.cx1.hpc.ic.ac.uk
> cx1-130-5-15.cx1.hpc.ic.ac.uk
> cx1-130-5-15.cx1.hpc.ic.ac.uk
> cx1-130-5-15.cx1.hpc.ic.ac.uk
> cx1-130-5-15.cx1.hpc.ic.ac.uk
> 
>         LD_LIBRARY_PATH=''
>  export AIXTHREAD_SCOPE='s'
>  export MOLPRO_PREFIX='/export131/home/rf614/MolPro/install'
>  export MP_NODES='0'
>  export MP_PROCS='12'
>         MP_TASKS_PER_NODE=''
>  export MOLPRO_NOARG='1'
>  export MOLPRO_OPTIONS=' -d /var/tmp/pbs.398951.cx1 -W /var/tmp/pbs.398951.cx1 -v /work/rf614/Post_Doc/molpro_tests/omp_tests/casscf_runs/casscf_part_only/test_casscf.com -t 1'
>         MOLPRO_OPTIONS_FILE=''
>         MPI_MAX_CLUSTER_SIZE=''
>         MV2_ENABLE_AFFINITY=''
>  export RT_GRQ='ON'
>  export TMPDIR='/var/tmp/pbs.398951.cx1'
>  export XLSMPOPTS='parthds=1'
> /export131/home/rf614/MolPro/install/bin/mpiexec.hydra -machinefile /var/tmp/pbs.398951.cx1/procgrp.33961 -np 12 /export131/home/rf614/MolPro/install/bin/molpro.exe  -d /var/tmp/pbs.398951.cx1 -W /var/tmp/pbs.398951.cx1 -v /work/rf614/Post_Doc/molpro_tests/omp_tests/casscf_runs/casscf_part_only/test_casscf.com -t 1
> 
> 
> _______________________________________________
> Molpro-user mailing list
> Molpro-user at molpro.net
> http://www.molpro.net/mailman/listinfo/molpro-user