franck.dernoncourt@gmail.com
Skype: franck.dernoncourt
MIT, Cambridge, USA

profile for Franck Dernoncourt on Stack Exchange, a network of free, community-driven Q&A sites

How to compile Sferes2 on Windows

What is Sferes2?
Configuration
Step 1: Compile Boost
Step 2: Configure Code::Blocks IDE
Step 3: Compile Sferes
Step 4: Resolve a segfault
Step 5: Enable multi-threading with TBB
Step 6 (optional): Eclipse

Sferes2 is a framework for evolutionary computation (EC) experiments and especially for evolutionary robotics (ER). Its main goal is to help researchers in EC and ER to efficiently try new ideas. Sferes2 has been inspired by sferes, another (older and no more maintained) framework for ER.

Sferes2 allows to design efficient programs on modern (multicore) computers: experiments typically require more than 24 hours of computation; in this situation it can be profitable to trade some complexity of implementation for some hours of computation. Nevertheless, the framework is designed to be as simple as possible.

More information on the official website.

W4G Rating Bar
This tutorial has been tested with the following configurations:
  1. Compiler: gcc (4.4.1 <TDM-2 mingw32>)

  2. IDE: Code::Blocks (10.05), Eclipse (Helios Service Release 2)

  3. Windows 7 Ultimate 64bits and Windows XP Professional 32bits

1) First we need to compile Boost. We'll use Boost's latest version: 1.45.0 (probably not the latest version at the time you'll read this tutorial.

1.1) Download Boost sources (http://www.boost.org/users/download/ or directly http://sourceforge.net/projects/boost/files/boost/1.45.0/boost_1_45_0.7z/download)

1.2) To compile Boost, we need to use BJam. BJam's sources are included in boost's sources. Yet, we can't use BJam's binaries: we must compile it because of the following problem: http://comments.gmane.org/gmane.comp.lib.boost.user/64146.
To compile BJam, go to Boost's sources, find build.bat (should be located in <path>\boost_1_45_0\tools\build\v2\engine\src), run cmd, cd to build.bat's folder and execute:
build.bat gcc
(source: http://www.phy.bnl.gov/~bviren/dayabay/offline/external-trunk/Boost/boost_1_34_1/doc/html/jam/building.html)
Compilation lasts less than 30 seconds.

Note 1: gcc must be in the $PATH. If you don't have gcc, you can get it at http://www.mingw.org/wiki/InstallationHOWTOforMinGW. Use the automated installer if you don't want to waste time on it. (don't forget to select the C++ compiler during the installation, it will only install the C compiler by default)
Note 2: don't call cygwin's gcc otherwise you'll get an "Access denied" error, see http://omgili.com/mailinglist/cygwin/cygwin/com/m2t961f6df41004191807y75bc02cfiabd7f684a8d5f8camailgmailcom.html for more details. Better use gcc included in the MinGW you will later use with your IDE.

1.3) Once BJam is compiled, we can now compile Boost. Add bjam.exe in $PATH, run cmd, cd \boost_1_45_0 then execute:
bjam --toolset=gcc "--prefix=C:\DestinationFolder" install
where DestinationFolder is whatever folder path you want (source: http://wiki.codeblocks.org/index.php?title=BoostWindowsQuickRef)
Compilation lasts ca. 30 minutes.
2) Now we need to configure the IDE (Code::Blocks). We'll first compile 2 simple source files.

2.1) Create a new empty project, create a new C++ source file called 'test1.cpp' and copy-paste this:
We have to set up a Code::Blocks global variable for Boost. This step only needs to be performed once, after which the global variable you've created will be available for any project.
  1. Open the Settings menu and select "Global variables...".

  2. Click the "New" button next to the Current variable list, specify a name like "boost", and hit OK.

  3. In the "base" field of the Builtin fields section, browse for the root folder of your Boost installation (e.g. "C:\Program Files (x86)\CodeBlocks\include\boost-1_45"). The Boost installation path is the path you specified in the --prefix option of the build command.

  4. In the "include" field, browse for the "include\boost-1_45" subfolder of your Boost installation.

  5. Hit the Close button to save your global variable


We need to add Boost search directories to your project. This step has to be performed for each project and build target (debug/release):
  1. Right-click your project's name in the Projects section of the Management window and select "Build options..." (or else right-click your project's name, select "Properties", then click on the button "Project's build options").

  2. Select the "Search directories" tab.

  3. With the "Compiler" subtab selected, click the Add button, enter "$(#boost.include)" (without the quotes), and hit OK.

  4. If you are using the bundled version of MinGW with Code::Blocks, and you used your C::B installation in the --prefix option as recommended, the following step is unnecessary (but better to do it still!).

  5. With the "Linker" subtab (still under the "Search directories" tab) selected, click the Add button, enter "$(#boost.lib)" (without the quotes), and hit OK.

(source: http://wiki.codeblocks.org/index.php?title=BoostWindowsQuickRef)

Hit Ctrl+9 to compile 'test1.cpp'. It should compile. If not, you've probably done something wrong in the above steps.


2.2) Now we're going to link our program with a Boost binary library. Remove 'test1.cpp' from the project, create a new C++ source file called 'test2.cpp' and copy-paste this: This is a simple program that extracts the subject lines from emails. It uses the Boost.Regex library, which has a separately-compiled binary component.

In your project's build options, select the "Linker settings" tab, and add "libboost_regex-mgw44-mt-d-1_45.a" :
  1. regex -> because we have #include

  2. mgw44 -> because we compiled boost with gcc (4.4.1)

  3. -d -> each .a has a release and a debug version. -d stands for debug. We would have to add "libboost_regex-mgw44-mt-1_45.a" if we targeted a release version.

  4. 1_45 -> this is the Boost version.

Hit Ctrl+9 to compile 'test2.cpp'. It should compile. If not, you've probably done something wrong in the above steps. Here is the kind of error we get if there is a problem with the link to the separately-compiled binary component: "C:\Program Files (x86)\CodeBlocks\include\boost-1_45\boost\regex\v4\cpp_regex_traits.hpp|366|undefined reference to`boost::re_detail::cpp_regex_traits_char_layer::init()'|"
3) Boost is compiled and we successfully called Boost from Code::Blocks. Time to compile Sferes2! First download it either on the official website or here.
  1. Create a new empty project (in this example we named it "'sferes").

  2. Copy-paste the 3 folders examples, exp and sferes in the project folder (e.g. "C:\Program Files (x86)\CodeBlocksProjects\sferes1").

  3. Right-click your project's name in the Projects section of the Management window and select "Add files recursively".

  4. Browse for "'sferes" project location, hit "OK" then hit "OK" once again (no need to change the files selection), then select all targets..

Now all sources files are accessible in the Management window. Add "$(#boost.include)" and "$(#boost.lib)" as we did above for our test project. Also add the project folder (e.g. "C:\Program Files (x86)\CodeBlocksProjects\sferes1") in the same windows where you've just added "$(#boost.include)". In your project's build options, select the "Linker settings" tab, and add all files "libboost_*.a".

Then remove from project:
  1. \examples\* except \examples\ex_ea.cpp (or any exp .cpp you want to compile, but only 1!)

  2. \exp\example\example.cpp

  3. \modules\*

  4. \scripts\*

  5. \sferes\dbg\dbg.cpp

  6. \tests\*



We need to do some modifications in order to solve a few compilation errors.

Error: C:\Program Files (x86)\CodeBlocksProjects\sferes1\sferes\parallel.hpp|41|error: tbb/task_scheduler_init.h: No such file or directory|
Resolution: In \sferes\parallel.hpp, add #define NO_PARALLEL at the begin of the file.

Error: C:\progws\codeblocks\sferes1\sferes\misc\sys.hpp|61|error: 'gethostname' was not declared in this scope|
Resolution: Replace the two lines " char hostname[30]; int res = gethostname(hostname, 30); " by " char hostname[30] = "test" ; int res = 0; " in the file \sferes\misc\sys.hpp

Error: C:\Program Files (x86)\CodeBlocksProjects\sferes1\sferes\ea\ea.hpp|82|error: 'SVN_VERSION' was not declared in this scope|
Resolution: add " #define SVN_VERSION "3" " at the begin of the file "\sferes\ea\ea.hpp"

Error: C:\progws\codeblocks\sferes1\examples\ex_ea.cpp:99:error: reference to 'stat' is ambiguous|
Cause: Conflict between namespaces (wchar.h vs. stat.hpp))
Resolution: Replace all occurences of "stat::" by "sferes::stat::" in the file \examples\ex_ea.cpp


Now you should a few linking errors coming from Boost in that order:

Error: C:\prog\progcpp\sferes01\sferes\run.hpp|59|undefined reference to `boost::program_options::options_description::m_default_line_length'|
Resolution: Add libboost_program_options-mgw45-mt-d-1_45.a in the same windows where you previously added "$(#boost.include)".

Error: C:\prog\boost\include\boost-1_45\boost\system\error_code.hpp|214|undefined reference to `boost::system::generic_category()'|
Resolution: Add libboost_system-mgw45-mt-d-1_45.a in the same windows where you previously added "$(#boost.include)".

Error: C:/prog/boost/include/boost-1_45/boost/archive/xml_oarchive.hpp:114: undefined reference to `boost::archive:: xml_oarchive_impl<boost::archive::xml_oarchive>:: xml_oarchive_impl(std::ostream&, unsigned int)'
Resolution: Add libboost_serialization-mgw45-mt-d-1_45.a in the same windows where you previously added "$(#boost.include)".

Error: C:/prog/boost/include/boost-1_45/boost/filesystem/v2/operations.hpp:423: undefined reference to `boost::filesystem2:: detail:: create_directory_api(std::string const&)'
Resolution: Add libboost_filesystem-mgw45-mt-d-1_45.a in the same windows where you previously added "$(#boost.include)".

4) Now Sferes2 should compile. It does, but the program keep crashing somewhere between 50 and 1000 generations on average. There is a segfault.

Error: Through GDB debugging we see that this->_pop[population's size] has the value equals to the magic number 0xabababab which is used by Microsoft's HeapAlloc() to mark "no man's land" guard bytes after allocated heap memory

Cause: In Windows RAND_MAX = 32767 (16bits) whereas in Linux RAND_MAX = 214748364 (32bits). Because of that, the function template<typename T> inline T rand(T max = 1.0) located in the file \sferes\misc\rand.hpp sometimes returns 0 (ca. 1/10000 in Windows, almost never on Linux hence so far this bug has gone unnoticed). Although this is coherent with the specs (http://pages.isir.upmc.fr/~mouret/sferes2/html/ref.html#sec-2.3.2)which say that r is a random number in [0,1[, therefore allowing r == 0, this special case results in a segfault given the current implementation. Let's exemplify it with \examples\ex_ea.hpp : \examples\ex_ea.hpp uses a rank-based single-objective evolutionary algorithm called rank simple, which is implemented in \sferes\ea\rank_simple.hpp. For each new individual, void epoch() needs to get a random value r1 between 0 et the population's size - 1 so as to get a random individual this->_pop[r1]. r1 and r2 are computed with _random_rank() (unsigned r1 = _random_rank();). _random_rank() itself calls template< typename T>inline T rand(T max = 1.0) which is located in \sferes\misc\rand.hpp. When rand(T max = 1.0) returns 0, then _random_rank() returns the population's size, which causes a call to this->_pop[population's size], which triggers off a segfault since an array index begins at 0 and ends at array_size-1.

Resolution: There is a bug in sferes\ea\rank_simple.hpp. Add these two lines in \sferes\ea\rank_simple.hpp: right after "unsigned r2 = _random_rank();".
5) Sferes2 is now compiled and stable. It would be great to enable multi-threading! To do so, we will use Intel Threading Building Blocks (TBB).

5.1) First we need to compile TBB.

Download TBB source files (http://threadingbuildingblocks.org/ or directly http://threadingbuildingblocks.org/uploads/77/164/3.0%20Update%205/tbb30_20101215oss_src.tgz). MinGW support was added to the 20090511 development release (source: http://origin-software.intel.com/en-us/forums/showthread.php?t=64410), so this is going to be quite easy. Uncompress the code source archive, run cmd, cd to tbb root folder, then execute:
mingw32-make.exe compiler=gcc arch=ia32 runtime=mingw
This will produce the following 6 dll in \tbb\build\windows_ia32_gcc_mingw_debug and \tbb\build\windows_ia32_gcc_mingw_release:
  1. tbbmalloc.dll

  2. tbbmalloc_debug.dll

  3. tbbmalloc_proxy.dll

  4. tbbmalloc_proxy_debug.dll

  5. tbb.dll

  6. tbb_debug.dll



5.2) We have to make a few adjustments:
  1. In \sferes\parallel.hpp, delete the line #define NO_PARALLEL that you previously added at the begin of the file.

  2. As we did for Boost, we need to create a new "Global variable" we'll call TBB in Code::Blocks. The "base" field as well as the "include" field should point to the folder \tbb\include\ (feel free to copy this folder somewhere else, usually in \CodeBlocks\include\tbb like Boost).

  3. We have to add "$(#tbb.include)" in the "Compiler" subtab, "$(#tbb.lib)" in the "Linker" subtab and add the 6 tbb*.dll in the "Linker settings" tab.


Now we can recompile Sferes2 and it will be multithreaded. Note that \examples\ex_ea.cpp only uses 1 core: you can test the multithreading on \examples\ex_nsga2.cpp
Actually, after successfully compiling Sferes2 through Code::Blocks, I switched to Eclipse as it has far more functionalities. Download Eclipse IDE for C/C++ Developers, or any other version (in which case you will have to install the C/C++ development tools afterwards): http://www.eclipse.org/downloads/

Create a new C++ project in which you add and modify Sferes files as explained in Step 3: Compile Sferes.

Now you simply need to configure the compiler and the linker so as they see all the required libraries. To do so, right click on the project's name, select 'Properties', then 'C/C++ Build', and finally 'Setting'. Add the following:

GCC C++ Compiler --> Includes (debug and release): (-I)
  1. "G:\Win7\PROGRAMMING\CodeBlocks\include\boost-1_45" (Boost)

  2. "G:\Win7\PROGRAMMING\EclipseProjects\testcpp2\src" (Sferes core and modules)

  3. "G:\Win7\PROGRAMMING\CodeBlocks\include" (TBB) (only if TBB is enabled)

  4. "G:\Win7\PROGRAMMING\CodeBlocks\include\eigen-2_0_15" (Eigen2 for nn2 module*)

  5. "G:\Win7\PROGRAMMING\CodeBlocks\include\SDL" (only if SDL is enabled)



GCC C++ Linker --> Libraries (debug) (-l) (the order may be important for SDLmain, SDL and mingw32):
  1. boost_filesystem-mgw44-mt-d-1_45

  2. boost_program_options-mgw44-mt-d-1_45

  3. boost_serialization-mgw44-mt-d-1_45

  4. boost_system-mgw44-mt-d-1_45

  5. mingw32 (only if SDL is enabled)

  6. SDLmain (only if SDL is enabled)

  7. SDL (only if SDL is enabled)

  8. tbb_debug (only if TBB is enabled)

  9. tbbmalloc_debug (only if TBB is enabled)

  10. tbbmalloc_proxy_debug (only if TBB is enabled)



GCC C++ Linker --> Libraries (release) (-l) (the order may be important for SDLmain, SDL and mingw32):
  1. boost_filesystem-mgw44-mt-1_45

  2. boost_program_options-mgw44-mt-1_45

  3. boost_serialization-mgw44-mt-1_45

  4. boost_system-mgw44-mt-1_45

  5. mingw32 (only if SDL is enabled)

  6. SDLmain (only if SDL is enabled)

  7. SDL (only if SDL is enabled)

  8. tbb (only if TBB is enabled)

  9. tbbmalloc (only if TBB is enabled)

  10. tbbmalloc_proxy (only if TBB is enabled)



Libraries search path (debug) (-L):
  1. "G:\Win7\PROGRAMMING\CodeBlocks\lib" (Boost)

  2. "G:\Win7\PROGRAMMING\CodeBlocks\lib\windows_ia32_gcc_mingw_debug" (TBB)



Library search path (release)(-L):
  1. "G:\Win7\PROGRAMMING\CodeBlocks\lib" (Boost)

  2. "G:\Win7\PROGRAMMING\CodeBlocks\lib\windows_ia32_gcc_mingw_release" (TBB)



*Note: to use the nn2 module, you'll need libeigen2-dev >= 2.06; see http://eigen.tuxfamily.org/ (no compilation needed, just include the whole source in GCC C++ Compiler --> Includes)