

- OpenFOAM source code testing
- OpenFOAM debug switch details
- Google Test and OpenFOAM
Featured image: Test Well by Doran.
Let’s adress the first obvious question that comes to mind when reading “Google Test and OpenFOAM”: why use Google Test or any other testing framework, when OpenFOAM already has an error reporting system? The error/warning system in OpenFOAM is based on macros. The macro definitions can be found in in error.H
for errors:
1 2 3 4 5 6 7 8 9 10 |
|
And the set of macros prepared for warnings is defined in messageStream.H
1 2 3 4 5 6 7 |
|
These can be used with the debug switch and/or conditional compilation covered in the source code testing blog series. The idea is to report errors and warnings in OpenFOAM code when a variable obtains an undesired value. Here is a micro test application that uses the error macros
1 2 3 4 5 6 7 8 9 10 11 12 |
|
The fun
function (pun intended) is implemented in a separate fun.H
file:
1 2 3 4 5 6 7 8 |
|
- Note
- This application is named
testFoamFatalError
and its code is available in theapplications/test
sub-directory in the our blog code repository.
Compiling and running testFoamFatalError
gives the following output
--> FOAM FATAL ERROR:
Not implemented
From function main
in file fun.H at line 30.
FOAM aborting
In this example the error is in fun()
but the error macro reports it to be in main
. This can happen if you copy-paste a macro invocation from somewhere else, and forget to rename the function name argument. That way you end up with a bug in the error macro call. For notImplemented
it is quite clear that just the function needs to be renamed, but using FatalErrorIn
on its own like this
FatalErrorIn("wrongClass::wrongMemberFunction")
<< "Wrong error statement"
<< ::Foam::abort(FatalError);
might lead to some confusion. Sure, file name and line number are shown and this is enough information to find the location of the error. The additional information is still needed for those tests that fail not because there is a runtime bug somewhere in wrongClass::wrongMemberFunction
, but because some runtime condition set on variable values gets breached. Before you think it is silly to provide a function implementation that results in an error, think about having a virtual member function in a class, that is not used by the class, but must be implemented since it has been inherited from an abstract base class. This smells of a situation where the interface class should be split in parts, but often it is useful to go on with programming and make sure that the function at least reports an error. If you search for notImplemented
in the official C++ documentation, you will find it is used quite often. When applied to member functions, the macro call looks like this:
1 2 3 4 5 6 7 8 9 10 |
|
Notice one important thing: the location of the error needs to be written by the programmer for OpenFOAM macros. This is the first benefit of using Google Test – Google Test figures out where it has been called automagically. There are other benefits and you can read more about them in the FAQ.
There is a catch when using Google Test and OpenFOAM together, however. The citation from the Google Test manual in the blog post about hacking Make/options to enable C++11/14 standard tells us that in order to safely build libraries and applications that work with it, Google Test should be compiled with the same compiler options as the project that uses it. Otherwise, nasty things may happen, at runtime. When OpenFOAM third party libraries are compiled, the compilation flags are preset by the Allwmake
script, in the $WM_THIRD_PARTY/etc/wmakeFiles/
directory. Manually presetting compiler options is not the goal here, we want to be able to extract the compiler flags from a defined OpenFOAM environment and pass them over to the compiler when compiling Google Test, or any other third-party library that requires the flags to be exactly the same as those applied for OpenFOAM libraries and apps.
Google Test and OpenFOAM build rules
The wmake
build system relies on a prescribed set of rules that define compiler and linker options when building (OpenFOAM) applications. Those rules are stored in files. The files are sorted in directories whose names have specific structure. The name structure is defined by the name of the compiler, operating system, architecture of the machine, and compilation options set by the OpenFOAM user in $WM_PROJECT_DIR/etc/bashrc
. In order to build Google Test using the same build configuration that is applied to OpenFOAM, we have to find a way to reconstruct the rules of the wmake
build system first. To do that, we have to understand how wmake
rules really work.
How wmake rules work
When compiling any OpenFOAM application with the wmake
build system we can see the full set of compiler flags
?> wmake
g++ -m64 -Dlinux64 -DWM_DP -Wall -Wextra -Wno-unused-parameter -Wold-style-cast -Wnon-virtual-dtor -O3 \
-DNoRepository -ftemplate-depth-100 -I/applications/solvers ...
Defining pre-processor variables using the -D
option have already covered in the blog post that covered conditional compilation for source code testing. Those variables are to be used by OpenFOAM code, so we can ignore their existence when compiling other libraries with OpenFOAM compiler flags. As for the other compiler flags in the above listing we can see that the g++
compiler is used, OpenFOAM is compiled in Opt
mode (-O3
flag), on a 64
bit linux
system. Basically, to compile another library with the same compiler options/flags we need to be able to re-create the string -m64 -Wall -Wextra -Wno-unused-parameter -Wold-style-cast -Wnon-virtual-dtor -O3
.
Note: this string changes depending on the configuration of the OpenFOAM platform defined by $WM_PROJECT_DIR/etc/bashrc
. This makes the task a tad more complicated. We can try to use the OpenFOAM environmental variables to reconstruct the string elements:
?> env | grep WM_
WM_LINK_LANGUAGE=c++
WM_ARCH=linux64
WM_OSTYPE=POSIX
WM_THIRD_PARTY_DIR=/home/tomislav/OpenFOAM/ThirdParty-2.4.x
WM_CXXFLAGS=-m64 -fPIC
WM_CFLAGS=-m64 -fPIC
WM_PROJECT_VERSION=2.4.x
WM_COMPILER_LIB_ARCH=64
WM_PROJECT_INST_DIR=/home/tomislav/OpenFOAM
WM_CXX=g++
WM_PROJECT_DIR=/home/tomislav/OpenFOAM/OpenFOAM-2.4.x
WM_NCOMPPROCS=4
WM_PROJECT=OpenFOAM
WM_LDFLAGS=-m64
WM_COMPILER=Gcc
WM_MPLIB=OPENMPI
WM_CC=gcc
WM_COMPILE_OPTION=Opt
WM_DIR=/home/tomislav/OpenFOAM/OpenFOAM-2.4.x/wmake
WM_PROJECT_USER_DIR=/home/tomislav/OpenFOAM/tomislav-2.4.x
WM_OPTIONS=linux64GccDPOpt
WM_PRECISION_OPTION=DP
WM_ARCH_OPTION=64
And indeed we can see WM_CXXFLAGS=-m64 -fPIC
– two flags bite the dust. However, there are no warning options defined in the environmental variables, and the optimization mode WM_COMPILE_OPTION=Opt
does not give us -O3
that gcc
understands. Another question pops out: what if we are not using gcc
, but some other compiler? Well, to reconstruct the string made by wmake
we have to think like wmake. The wmake
build system wraps itself around different compilers:
?> ls $WM_PROJECT_DIR/wmake/rules
General linux64Gcc++0x linux64Gcc45 linux64Gcc48 linuxARM7Gcc
linux64Clang linux64Gcc43 linux64Gcc46 linux64Gcc49 linuxClang
linux64Gcc linux64Gcc44 linux64Gcc47 linux64Icc linuxGcc
linuxGcc43 linuxGcc46 linuxGcc49 linuxIcc solarisGcc
linuxGcc44 linuxGcc47 linuxIA64Gcc linuxPPC64Gcc SunOS64Gcc
linuxGcc45 linuxGcc48 linuxIA64Icc linuxPPC64leGcc
This means that we can use the environmental variables that define the platform (linux), architecture (64) and compiler (gcc) to read out wmake
rules for c++
?> cat $WM_DIR/rules/$WM_ARCH$WM_COMPILER/c++
.SUFFIXES: .C .cxx .cc .cpp
c++WARN = -Wall -Wextra -Wno-unused-parameter -Wold-style-cast -Wnon-virtual-dtor
CC = g++ -m64
include $(RULES)/c++$(WM_COMPILE_OPTION)
ptFLAGS = -DNoRepository -ftemplate-depth-100
c++FLAGS = $(GFLAGS) $(c++WARN) $(c++OPT) $(c++DBUG) $(ptFLAGS) $(LIB_HEADER_DIRS) -fPIC
Ctoo = $(WM_SCHEDULER) $(CC) $(c++FLAGS) -c $$SOURCE -o $@
cxxtoo = $(Ctoo)
cctoo = $(Ctoo)
cpptoo = $(Ctoo)
LINK_LIBS = $(c++DBUG)
LINKLIBSO = $(CC) $(c++FLAGS) -shared -Xlinker --add-needed -Xlinker --no-as-needed
LINKEXE = $(CC) $(c++FLAGS) -Xlinker --add-needed -Xlinker --no-as-needed
There we go, this file contains all the compiler options we need. In order to reconstruct the complete compiler call, we need to reconstruct the complete c++FLAGS
string. The problem is that all the variables that define c++FLAGS
are internal – wmake
spawns an internal shell and defines a bunch of variables. Since the variables are internal, they are not available in the main SHELL where we are trying to build Google Test. That makes no difference, since we can pick up the variables that build c++FLAGS
from wmake rule files as well. To conclude, we need to do two things:
- Parse files where the variables are defined – to assemble the compiler flag string.
- Select appropriate wmake rules based on the OpenFOAM environmental variables.
- Note
- We can forget about the linker options – linking to the compiled
gtest
library is defined in theMake/options
of OpenFOAM applications and libraries.
Parsing wmake rules
I chose to whip up a small script in python for the tasks 1. and 2.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 |
|
The script uses environmental variables defined by the OpenFOAM platform to find wmake
files and read out the compiler rules into a string that is then printed into stdout
.
- Note
- You can find this script in
sourceflux-blog-code/src/scripts
under the namegetWmakeCompilerOptions
.
To make it available in the terminal where the sourceflux blog code repository has been configured, this line has been added to the blog code repository bashrc
configuration script:
export PATH=$SOURCEFLUX_BLOG_CODE/src/scripts:$PATH
Using this script, we can extract the wmake
compiler options and store them in an environmental variable, or use them in the build configuration script of other libraries/applications, like Google Test.
- Note
- Extracting the compiler options/flags used by
wmake
with the above script may come in handy for building your own third party dependency with its native build system, it doesn’t have to be Google Test.
Google Test and OpenFOAM projects
To use Google Test and OpenFOAM together, Google Test can either be used as a third-party dependency of a self-sustained OpenFOAM project (like sourceflux-blog-code), or it can be built as a third-party dependency the platform itself. Depending of the choice, the applications and libraries of your project will require different environmental variables to point wmake
to locations of Google Test header files and the Google Test library. Citing the FAQ (emphasis added)
…for your sanity, we recommend to avoid installing pre-compiled Google Test libraries. Instead, each project should compile Google Test itself such that it can be sure that the same flags are used for both Google Test and the tests.
This can be misinterpreted: if you are organizing your code in self-sustained OpenFOAM projects (and you should do that, and not simply code in the platform directory structure), you are still leaning on the platform – without OpenFOAM/foam-extend, your code will not compile/run. Moreover, wmake
will use the rules defined for the platform to build your project. In this sense, the word project refers to the entire OpenFOAM platform, and not your specific project directory. In my case, this means not to use Google Test as a dependency of the sourceflux-blog-code
repository..
To build Google Test, install subversion on your system. Then follow these instructions after sourcing bashrc
that configures OpenFOAM:
?> cd $WM_PROJECT_INST_DIR
?> mkdir GoogleTest
?> cd GoogleTest
?> svn checkout http://googletest.googlecode.com/svn/trunk/ gtest
At this point we have to decide which build system to use for Google Test: CMake, or GNU Make.
Building Google Test with CMake
Edit CMakeLists.txt and insert the following lines
1 2 3 4 5 6 7 8 9 10 11 |
|
just above the line cxx_library(gtest "${cxx_strict}" src/gtest-all.cc)
. Then modify the cxx_library
lines for the Google Test libraries so that ${cxx_strict}
is replaced by the string {$WMAKE_COMPILER_OPTIONS}
cxx_library(gtest "${WMAKE_COMPILER_OPTIONS}" src/gtest-all.cc)
cxx_library(gtest_main "${WMAKE_COMPILER_OPTIONS}" src/gtest_main.cc)
This will set the compile options to WMAKE_COMPILER_OPTIONS
for the shared Google Test libraries. The next question is where to build the library, since we will generate different binaries, each time we source a different set of build options for OpenFOAM (like Opt, Prof, etc). We will do what wmake
does:
?> mkdir $WM_ARCH$WM_COMPILER$WM_COMPILE_OPTION
?> cd !$
Create a directory that is a superposition of build options for OpenFOAM and configure the CMake build in this directory
?> cmake -DBUILD_SHARED_LIBS=ON ../gtest/
?> make
We are setting the variable BUILD_SHARED_LIBS
so that CMake builds shared libraries that can be linked against OpenFOAM applications at compile time and loaded at runtime by the linker. You should see this line in the output of the cmake
configuration command if the configuration was executed successfully:
-- Extending the compiler options with following wmake options: -m64 -fPIC -Wall -Wextra -Wno-unused-parameter -Wold-style-cast -Wnon-virtual-dtor -DNoRepository -ftemplate-depth-100 -O3
To build the library and the tests, execute
?> make
If you want to be sure that the appropriate compiler options have been used, you can open the file CMakeFiles/gtest.dir/flags.make
. In my file, for the OpenFOAM Debug
compile option, this is listed
# compile CXX with /usr/bin/c++
CXX_FLAGS = -fPIC -I/home/tomislav/OpenFOAM/GoogleTest/gtest/include -I/home/tomislav/OpenFOAM/GoogleTest/gtest -m64 -fPIC -Wall -Wextra -Wno-unused-parameter -Wold-style-cast -Wnon-virtual-dtor -DNoRepository -ftemplate-depth-100 -O3
As you see, there are some duplicates (fPIC
), but the compiler doesn’t mind. In order to make it easy for the wmake
build system to find the libgtest.so
, we copy the library binary to the appropriate directory:
?> cp libgtest.so $FOAM_USER_LIBBIN
CMake installation script
I’ve summed up the installation process described in the previous section into a script to make things easier for those only interested in building and installing Google Test with their OpenFOAM applications/libraries without getting into the details. The description above should make it easier to understand how wmake
works and how we can re-use its environmental variables in case build parameters need to be adjusted. The installation process is automatized with the installGoogleTest
python script:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 |
|
- Note
- To use this script, you have to clone the blog post repository and source its
bashrc
configuration script. If you wan to use this script without thesourceflux-blog-code
repo, you need to store your ownCMakeLists.txt
file in another directory and rename theSOURCEFLUX*
variables. - Note
- If you have a properly installed OpenFOAM environment and you have sourced the
bashrc
configuration script in the sourceflux blog code repo, run theinstallGoogleTest
script from anywhere on the command line and it will do the job.
The script does what the build description in the section above does, step by step. Creates the GoogleTest
, clones Google Test in there into gtest
, creates the binary directory, steps into it, configures the out of source build, builds Google Test and ships the libgtest.so
to $FOAM_USER_LIBBIN
. Pretty straightforward stuff. Now that the library is compiled and copied to where OpenFOAM applications and libs can find it, we can start using it and prepare the first OpenFOAM Google Test application.
Google Test and OpenFOAM applications
We will be testing the changing cell set to show how Google Test and OpenFOAM work together on the application level. Since the cell set is assembled by an oscillating circle that returns to the initial position, we can test to see if the final circle contains exactly the same number of cell IDs as it did at the beginning of the simulation.
- Note
- The prerequisite for the steps that follow is the blog post on programming new OpenFOAM applications.
The application source code is available in the sourceflux blog code repository as well, so check that directory out if you encounter problems with this tutorial, and leave comments in the comment section below if that doesn’t help. The include statements are the same as in the testChangingCellSet application, with the addition of gtest.h
for Google Test goodies
1 2 3 |
|
The main difference between a Google Test OpenFOAM application and a standard one is in the main
function:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 |
|
We are writing a TEST
function that is the main function in an OpenFOAM application. All the information on the test structure in Google Test is available in the Google Test documentation. Since we are packing the main function into the TEST
we need to parse the argc
an argv
as external variables. The real main function looks like this:
1 2 3 4 5 6 7 8 9 10 11 12 |
|
Here the test framework is initialized, argv
and argc
are externalized and all test are run. This simple test application for the changing cell set checks if the size of the set in the final time step is twice its original size:
ASSERT_EQ(initialSize, 2 * testCells.size())
<< "initialSize = " << initialSize << "\n"
<< "testCells.size() = " << testCells.size() << "\n";
Of course, since the changing cell set was on purpose made oscillate so that it returns to the original set of mesh cell labels, this test will fail. In order to build the application, we need to tell the compiler where to find the declaration files of Google Test. The location of the declaration and implementation files was prescribed by the installation procedure to $WM_PROJECT_INST_DIR/GoogleTest
, but this can be changed by a personal preference:
1 2 3 4 5 6 7 |
|
To make the test application link against the gtest
library, the Make/options
file needs to be modified accordingly
EXE_LIBS = \
-L$(FOAM_USER_LIBBIN) \
-lfiniteVolume \
-lmeshTools \
-lchangingCellSet \
-lgtest
Since the installGoogleTest
script copies the library binary file to FOAM_USER_LIBBIN
, this directory needs to be added to the search path variable of the linker. Of course, the libgtest.so
library is then listed as well, since it provides the definitions for the test interfaces available in gtest.h
.
Building and running the test application results in the following output:
gtestChangingCellSet.C:81: Failure
Value of: 2 * testCells.size()
Actual: 784
Expected: initialSize
Which is: 392
initialSize = 392
testCells.size() = 392
[ FAILED ] one.two (737 ms)
[----------] 1 test from one (737 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (737 ms total)
[ PASSED ] 0 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] one.two
We can see that the output is quite verbose: the test streams work just like the FatalErrorIn
and WarningIn
streams in OpenFOAM – we can stream stuff in them to make it easier for us to find out why the test failed. This was a very simple example, but in general, the framework allows more complex tests to be built – for instance, tests that share data (test fixtures).
Using the Google Test framework within OpenFOAM libraries is no different than what is shown for the application above. Once the library binary is built with the wmake
compiler options and its location, as well as the location of the sources is fixed, the Make/options
and Make/files
are edited to enable the compiler and linker to find the declaration files and the library binary files, respectively. That’s it.
Conclusions
On the level shown here the systems are not much different, but as the tests get more complex, setting up the tests gets easier with a test framework like Google Test. Just the fact that the output is neatly organized and reported helps even if you are dealing with a lot of simple tests that need to be executed often as the modifications on the source code are implemented. In that case, one can quickly see how changing of some arguments immediately breaks a specific sub-set of tests, that helps making the code more robust.