Log in to Ada:
ssh ada.c3se.chalmers.se
Add the necessary MPICH module to your $HOME/.bashrc file at Ada by typing the following at command-line:
module initadd mpich/1.2.7-PIC/gcc/4.1
Add the following at the end of your $HOME/.bashrc file at Ada:
-------------------------------------------------------------------
# User specific aliases, functions and default modules below
export WM_64=on #Use the 64 bit version (the only pre-installed)
export FOAM_SILENT=1 #No echo in bashrc
export FOAM_SIGFPE=1 #Halts on Nan/Floating Point Exceptions
#export WM_NCOMPPROCS=8 #Compile in parallel
. $HOME/OF1.4_Ada_bashrc
#. /c3se/apps/OpenFOAM/OpenFOAM-1.4/.OpenFOAM-1.4/bashrc # This is the correct one, when it works...
-------------------------------------------------------------------
If you want to use your own installation, just modify the final sourcing to the corresponding bashrc file of your own installation.
Log out and log in to Ada again (ssh ada.c3se.chalmers.se), so that the files have been sourced correctly.
Make sure that you have the following directory structure:
$HOME/OpenFOAM/$LOGNAME-1.4/run
Copy the pitzDaily case from the tutorials directory to your run directory:
cp -r /c3se/apps/OpenFOAM/OpenFOAM-1.4/tutorials/simpleFoam/pitzDaily $HOME/OpenFOAM/$LOGNAME-1.4/run
Move to the run directory:
run
Generate the grid:
blockMesh . pitzDaily
Run simpleFoam on the pitzDaily case:
simpleFoam . pitzDaily
Your computation should now run. Otherwise you should go through this set-up again before you consult me.
-----------------------------------------------------------------------
# Arguments to qsub can be submitted via the script as well by starting
# the line with #PBS
#
# Choose the que:
#PBS -q ada
#
# Specify time for job (here 1h 10 min 5 sec)
#PBS -l walltime=1:10:05
#
# Request 1 processor (node)
#PBS -l nodes=1:ppn=1
#
# Set the name of the job
#PBS -N pitzDailySequential
#
# End of arguments to qsub
# Go to work submission directory (Don't remove this line)
# This is where output from standard output will appear.
# Output from OpenFOAM will appear in the OpenFOAM directories.
cd $PBS_O_WORKDIR
# Time the program
time simpleFoam $FOAM_RUN pitzDaily > $FOAM_RUN/pitzDaily/log
#End of script (make sure line before this gets run)
-----------------------------------------------------------------------
If you named the above script as 'submitPitzDailySequential', submit the job as:
qsub submitPitzDailySequential
Check the status of your jobs with:
qstat
You should get one 'pitzDailySequential.o*', and one 'pitzDailySequential.e*' file in the directory you submitted from. 'pitzDailySequential.o*' contains the standard output. 'pitzDailySequential.e*' contains output from the time command, or possible error messages. The results will be in the pitzDaily time directories as usual.
For parallel jobs you need to decompose your case before you run it. Download decomposeParDict to the 'system' directory in your pitzDaily case. This decomposeParDict will decompose your case into 8 parts using the Metis method.
Here is a script that decomposes your case, runs it and reconstructs it:
-----------------------------------------------------------------------
# Arguments to qsub can be submitted via the script as well by starting
# the line with #PBS
#
# Choose the que:
#PBS -q ada
#
# Specify time for job (here 1h 10 min 5 sec)
#PBS -l walltime=1:10:05
#
# Request 8 processors (2 nodes and 4 processors per node)
#PBS -l nodes=2:ppn=4
#
# Set the name of the job
#PBS -N pitzDailyParallel
#
# End of arguments to qsub
#
export P4_GLOBMEMSIZE=10240000 #This seems to be case-dependent, and this
#value works with 100 000 cells per process
#It may be possible to recompile MPICH
#to avoid setting this value
#
# Go to work submission directory (Don't remove this line)
# This is where output from standard output will appear.
# Output from OpenFOAM will appear in the OpenFOAM directories.
cd $PBS_O_WORKDIR
# Decompose the case for parallel computations
decomposePar $FOAM_RUN pitzDaily
# Time the program
time mpiexec simpleFoam $FOAM_RUN pitzDaily -parallel > $FOAM_RUN/pitzDaily/log
# Reconstruct the case after parallel computations
reconstructPar $FOAM_RUN pitzDaily
#End of script (make sure line before this gets run)
-----------------------------------------------------------------------
If you named the above script as 'submitPitzDailyParallel', submit the job as:
qsub submitPitzDailyParallel
Check the status of your jobs with:
qstat
You should get one 'pitzDailyParallel.o*', and one 'pitzDailyParallel.e*' file in the directory you submitted from. 'pitzDailyParallel.o*' contains the standard output. 'pitzDailyParallel.e*' contains output from the time command, or possible error messages. The results will be in the pitzDaily time directories as usual.
When you compile your applications you must make sure that you use $FOAM_USER_APPBIN in your Make/files file, so that you executable will be written to your own directory. If you keep $FOAM_APPBIN wmake will try to write to the root directories, and the compilation will fail.
Note that you can submit compilation jobs by making a submit script as described above, but with the wmake command. You must then submit your job from the directory of your specific directory and make sure that you keep the line with 'cd $PBS_O_WORKDIR' in your submit script.
Note that you should not install the official release yourself at Ada. An updated bug-fixed version is kept up to date for you in the central installation.
It might in fact be a good idea to re-compile all the applications:
cd $WM_PROJECT_DIR/applications
./Allwmake
(Note that there is a bug in wmake in 1.4, so you will get an error message! Find the corrected wmake in the Forum.)
Now you should be able to run the tests above. Good luck!
Note that you should not do this at Ada. The central installation will be kept updated for you. If you find out that it is not updated, please tell Håkan to update it.
Note that external packages are not part of the svn, so you will have to get them from elsewhere:
src/lam-7.1.2
src/mico-2.3.12
src/openmpi-1.2b3
src/zlib-1.2.1
There might still be a problem with an STL file. See how to fix it at:
http://openfoam.cfd-online.com/cgi-bin/forum/show.cgi?tpc=126&post=13043#POST13043
There might still be a problem with paths to Allwmake. See the bugs report.
Make sure that cmake is available in your path:
module add cmake
Download paraview-2.4.4.tar.gz from:
http://powerlab.fsb.hr/ped/kturbo/OpenFOAM/release/
into ~/OpenFOAM/linuxSrc
Install Paraview:
(You don't have to do this, but then you have to compile parts 'by hand')
cd ~/OpenFOAM/linuxSrc
tar xvf paraview-2.4.4.tar.gz
mkdir paraview-2.4.4-obj
cd paraview-2.4.4-obj
cmake -i ../paraview-2.4.4
Change options:
enable shared library: YES
prefix: ~/OpenFOAM/linuxSrc/paraview-2.4.4
make
make install
cd ~/OpenFOAM/linuxSrc
mkdir ~/OpenFOAM/linux64/paraview-2.4.4/include
cp ./paraview-2.4.4/VTK/Common/vtk* ~/OpenFOAM/linux64/paraview-2.4.4/include/
cp ./paraview-2.4.4/VTK/Filtering/vtk* ~/OpenFOAM/linux64/paraview-2.4.4/include/
cp ./paraview-2.4.4-obj/VTK/vtk* ~/OpenFOAM/linux64/paraview-2.4.4/include/
mkdir ~/OpenFOAM/linux64/paraview-2.4.4/lib
mkdir ~/OpenFOAM/linux64/paraview-2.4.4/lib/paraview-2.4
cp ./paraview-2.4.4-obj/ParaViewConfig.cmake ~/OpenFOAM/linux64/paraview-2.4.4/lib/paraview-2.4/
mkdir ~/OpenFOAM/linux64/paraview-2.4.4/bin
cp ./paraview-2.4.4-obj/bin/paraview ../paraview-2.4.4/bin
cp ./paraview-2.4.4-obj/bin/pvbatch ../paraview-2.4.4/bin
cp ./paraview-2.4.4-obj/bin/pvclient ../paraview-2.4.4/bin
cp ./paraview-2.4.4-obj/bin/pvdataserver ../paraview-2.4.4/bin
cp ./paraview-2.4.4-obj/bin/pvrenderserver ../paraview-2.4.4/bin
cp ./paraview-2.4.4-obj/bin/pvserver ../paraview-2.4.4/bin
cp ./paraview-2.4.4/lib/paraview-2.4/paraview-real ../linux64/paraview-2.4.4/lib/paraview-2.4/
or...
cp paraview-2.4.4-obj/bin/paraview-real ../linux64/paraview-2.4.4/lib/paraview-2.4/
cp paraview-2.4.4-obj/bin/lib* ../linux64/paraview-2.4.4/lib/paraview-2.4/
cp -r paraview-2.4.4-obj/lib/tcl8.4 ../linux64/paraview-2.4.4/lib/paraview-2.4/
cp -r paraview-2.4.4-obj/lib/tk8.4 ../linux64/paraview-2.4.4/lib/paraview-2.4/
~/OpenFOAM/OpenFOAM-1.4/bin/paraFoam.pvs is also needed, copy it from the original distribution?
(see also Hrv's build instructions)
Now you have the source code for the patched version, and you have to modify the set-up and compile the whole stuff (approx 4h!).
In ~/OpenFOAM/OpenFOAM-1.4_patch/.OpenFOAM-1.4/bashrc:
Line 52:
export WM_PROJECT_DIR=$WM_PROJECT_INST_DIR/$WM_PROJECT-${WM_PROJECT_VERSION}_patch
Line 127-128:
#export WM_MPLIB=LAM
export WM_MPLIB=MPICH
In ~/OpenFOAM/OpenFOAM-1.4_patch/.bashrc:
Line 107-109: This requires that you have downloaded the gcc binaries from the OpenFOAM homepage. You can change this to a central installation if available.
export MPICH_VERSION=1.2.7
In ~/.bashrc:
Source ~/OpenFOAM/OpenFOAM-1.4_patch/.OpenFOAM-1.4/bashrc
Ada (RedHat) does not have demangling:
In ~/OpenFOAM/OpenFOAM-1.4/src/OpenFOAM/lnInclude/error.C:
Line 37:
#undef HAS_DEMANGLING
Ada (RedHat) has the ssl library in another place than expected by OpenFOAM:
The file is located in /lib/libssl.so.0.9.7a, but FoamX needs it to be
located in /usr/lib.
Creating a symbolic link named /usr/lib/libssl.so.0.9.7 to
/lib/libssl.so.0.9.7a (you have to be root):
ln -s /lib/libssl.so.0.9.7a /usr/lib/libssl.so.0.9.7
Log out and log in to set all the environment variables correctly.
Compile (4h - get lots of coffee!):
foam
./Allwmake >& Allwmake.log &
If your working copy is up to date you will just get something like:
At revision 7.
If there has been some updates you will see which files have been updated. Then you need to re-compile the updated stuff.