GEOS–Chem v9–01–03 Online User's Guide

Previous | Next | Printable View (no frames)

3. Compilation

3.1 Overview

Once you have downloaded the GEOS–Chem source code directory, you can compile GEOS–Chem. During this process, your Fortran-90 compiler will create a binary executable file from the instructions contained in the various source code modules and files. You can then place the executable file into a GEOS–Chem run directory and start a simulation.

The compilation process involves the following steps:

1. Selecting the proper C–preprocessor switches in the define.h file (see Chapter 3.4)
2. Invoking the GNU Make utility with the proper options (see Chapter 3.5)

GNU Make manages the entire compilation process. A series of scripts called makefiles (see Chapter 3.3) directs GNU Make to compile the GEOS–Chem source code files in the proper order and with the proper options. These makefiles are placed throughout the GEOS–Chem source code directory.

For more information about the GEOS–Chem source code and makefiles, please see:

• Chapter 7: Coding: Practice and Style.
• GEOS–Chem wiki: GEOS–Chem Makefile Structure

NOTE: In general, executables cannot be easily swapped between computers (unless they are identical systems). If you want to move GEOS–Chem to a different computer, you will more than likely have to recompile it on that system.

3.2 Source code directory structure

We (the GEOS–Chem Support Team) have subdivided the GEOS–Chem source code directory into several subdirectories. This was done for the following reasons:

1. To facilitate the installation of 3rd-party software packages such as:

   • KPP chemical solver
   • Aerosol microphysics codes (e.g. TOMAS, APM)
   • ISORROPIA II
   • Terrestrial models (e.g. CASA, Global Terrestrial Mercury Model), etc.

   into GEOS–Chem. Our guiding principle is that all 3rd-party software packages should be cleanly separatable from the mainline GEOS–Chem code. This will allow the 3rd-party software packages to be updated without having an impact on the rest of the GEOS–Chem code.

2. To simplify the maintenace of the GEOS–Chem code files. Without subdirectories, there would have been hundreds of source code files in a single directory, and it woud have been very difficult to keep track of them all.

The current directory structure is:

In the text below, we shall often omit the top-level Code.v9-01-03 directory and refer to files by their subdirectory only, such as:

• Headers/define.h,
• GeosCore/dao_mod.F
• ISOROPIA/isoropiaII_code.F, etc.

3.3 GEOS–Chem Makefiles

The GEOS–Chem compilation process is managed by makefiles located throughout the source code directory. Makefiles direct the GNU Make utility to execute the various compilation commands in the proper sequence.

In general there are 2 types of makefiles:

• "Router" makefiles: Makefiles that tell GNU Make to look at makefiles in other directories.

• "Regular" makefiles: Makefiles that are used by GNU Make to compile source code programs..

The top-level source code directory (Code.v9-01-03) contains a "router" makefile, which calls down to the "regular" makefiles in the various subdirectories. In each subdirectory, the "regular" makefile tells GNU Make how to compile the source code files. When the source code in all subdirectories has been compiled, the top-level Makefile tells GNU Make to create the GEOS–Chem executable file.

Here is a list of makefiles used by GEOS–Chem:

You will usually not have to modify any of these makefiles, unless you are adding new code into GEOS–Chem. If you need to modify makefiles, then we will be happy to assist you.

Each makefile controls several compilation options. We shall present a detailed discussion of these options in Chapter 3.5.

3.4 Setting the C–preprocessor switches

Before compiling GEOS–Chem, you must set the proper C–preprocessor switches to inform the Fortran-90 compiler which scientific options you would like compiled into the GEOS–Chem executable.

3.4.1 What is the C–preprocessor?

The C–preprocesor (or CPP for short) is a feature of the C and C++ programming languages that allows you to compile blocks of code only if certain conditions are met. For example, let's say you have two blocks of code, Block A and Block B. If you set a certain switch in the code, then Block A will get compiled into the executable but Block B will not. When you run the executable, it will behave as if Block B had never existed. If you set a different switch, then Block B will be compiled but Block A will not, and the executable will behave as if Block A had never existed.

All modern Fortran compilers have inherited the C–preprocessor feature. Some Fortran compilers even call it the "Fortran preprocessor" (FPP). Therefore, even though GEOS–Chem is written in Fortran-90, we can still incorporate C–preprocesor commands into the source code.

With the C–preprocessor, you can design software to handle many different conditions, such as multiple compiler/platform combinations, different file formats, and the like. At compilation time, you can direct the C–preprocessor to only choose the options relevant to the particular computer system you are running on. Options that are not relevant to your system will be ignored. This is a standard programming technique.

The C–preprocessor statments take the form of #define and #if - #endif statements. A simple example from GEOS–Chem is:

#if   defined( GEOS_4 )
      WRITE( 6, '(a)' ) 'Using GMAO GEOS-4 met fields'
#elif defined( GEOS_5 )
      WRITE( 6, '(a)' ) 'Using GMAO GEOS-5 met fields'
#elif defined( GEOS_57 )
      WRITE( 6, '(a)' ) 'Using GMAO GEOS-5.7.x met fields'
#elif defined( MERRA )
      WRITE( 6, '(a)' ) 'Using GMAO MERRA met fields'
#elif defined( GCAP  )
      WRITE( 6, '(a)' ) 'Using GCAP/GISS met fields'
#endif

Here the #elif tag stands for "else if". An #else tag is also allowed.

If we have #defined the GEOS_5 C–preprocessor switch (see Chapter 3.4.2), then GEOS–Chem will be compiled for use with the GEOS–5 met fields. The block of code that prints out the message "Using GMAO GEOS-5 met fields" will be built into the exectuable, and all other print statements will be ignored.

C–preprocessor statements must be placed into the first column of a Fortran source code file, or else you may encounter a syntax error.

3.4.2 Selecting the proper C–preprocessor switches for your simulation

GEOS–Chem uses C–preprocessor switches to select various options, including:

1. The meteorological fields that GEOS–Chem will use
2. The horizontal and vertical grids for the GEOS–Chem simulation
3. The type of compiler you will use to create the GEOS–Chem executable
4. Options for special simulations (e.g. mercury)

These switches are located in header file Headers/define.h, and are as follows:

Met Fields
GCAP	Use GISS met fields for GCAP simulation (23 layer, 4° x 5°)
GEOS_4	Use GEOS–4 met fields (55 layer, hybrid, 1985–2006)
GEOS_5	Use GEOS–5 met fields (72 layer, hybrid, 2005–present)
GEOS_57	Use GEOS–5.7.2 met fields (72 layer, hybrid, 2011–present )
MERRA	Use MERRA met fields (72 layer, hybrid, 1979-2010)
Grids
NESTED_CH	Use China/SE Asia nested grid (GEOS–5 only)
NESTED_NA	Use N. American nested-grid (GEOS–5 only)
NESTED_EU	Use European nested grid (GEOS–5 only)
SEAC4RS	Use SEAC4RS nested grid (GEOS–5.7.2 only). Note: This grid was developed for the SEAC4RS campaign.
GRID025x03125	Use 0.25° x 0.3125 ° nested grid (SEAC4RS only)
GRID05x0666	Use 0.5 ° x 0.667 ° nested grids
GRID1x1	Use 1° x 1° grid
GRID1x125	Use 1° x 1.25° grid (GEOS–4 only)
GRID2x25	Use 2° x 2.5° grid
GRID4x5	Use 4° x 5° grid
GRIDREDUCED	Reduce the number of vertical levels on the fly: Use the 30-level GEOS–4 grid Use the 47-level GEOS–5 grid Use the 47-level GEOS–5.7.2 grid Use the 47-level MERRA grid
Compilers
LINUX_PGI	Compile for Linux with PGI compiler
LINUX_IFORT	Compile for Linux with Intel Fortran Compiler (IFORT 9.x or 10.x)
Other options
GTMM_Hg	Compile for a mercury simulation using the online Global Terrestrial Mercury Model (a.k.a. GTMM). GTMM is turned off by default in order to save compilation time. For more information about using GTMM in a GEOS–Chem mercury simulation, please see this PDF.

The C–preprocessor switches in Headers/define.h take the form of:

!#define GEOS_5 'GEOS_5'

etc. (The quotes are necessary to facilitate compilation on the Linux platforms). In order to activate a particular switch, simply remove the comment character (exclamation mark) from the first column of the line. To de-activate a switch, simply place a comment character (exclamation mark) in the first column of the line.

Please take note of the following:

1. When you change any of the C–preprocessor switches, you should issue a make realclean command (see Chapter 3.5.1). This will remove all previously-created compiler output files (e.g. *.o, *.mod) and executables. After doing make realclean, you can compile GEOS–Chem with whichever options you choose.

2. You may only select one particular met field type (GCAP, GEOS_4, GEOS_5, GEOS_57), one particular grid type (GRID025x03125, GRID05x0666, GRID1x125, GRID2x25, GRID4x5), and one particular compiler.

3. If you have selected GRID05x0666, then you must also select NESTED_CH, NESTED_NA, or NESTED_EU. This will select the proper 0.5° x 0.666° nested grid region (China/SE Asia, North America, or Europe).

4. If you have selected GRID025x03125, then you must also select SEAC4RS.

3.5 Compiling the GEOS–Chem source code

Once you have properly set the C–preprocessor switches in the Headers/define.h header file for the type of simulation that you want to perform, you can compile the GEOS–Chem code.

To compile the code, you need to be in the Code.v9-01-03 top-level directory. There are a large choice of compilation options available. The list of options can be printed to the screen by typing:

make help

3.5.1 Makefile options

Here is a description of all available make options for GEOS–Chem:

Targets
Target name	Description
all	Default target (synonym for "lib exe")
lib	Builds all GEOS–Chem source code (synonym for "libkpp libutil libcore")
libcore	Only builds GEOS–Chem objects & libraries in GeosCore subdir
libiso	Only builds GEOS–Chem objects & libraries in ISOROPIA subdir
libkpp	Only builds GEOS–Chem objects & libraries in KPP subdir
libnc	Only builds GEOS–Chem objects & libraries in NcdfUtil subdir
ncdfcheck	Checks that your netCDF library has been installed properly
libutil	Only builds GEOS–Chem objects & libraries in GeosUtil subdir
libheaders	Only builds GEOS–Chem objects & libraries in Headers subdir
exe	Build GEOS–Chem executable
clean	Removes *.o, *.mod, geos* files in source code subdirs only
realclean	Removes all *.o, *.mod, geos*, lib*.a, *.tex, *.ps, *.pdf files everywhere
doc	Builds GEOS–Chem documentation (*.ps, *.pdf) with ProTeX in doc subdir
docclean	Removes *.tex, *.ps, *.pdf from the doc subdir
help	Displays the help screen
Special targets for TOMAS aerosol microphysics
tomas	Build GEOS–Chem + TOMAS code (synonym for "libtomas exetomas")
libtomas	Only builds GEOS–Chem + TOMAS objects & libraries
exetomas	Only builds GEOS–Chem + TOMAS executable
cleantomas	Removes *.o *.mod files in GeosTomas directory only
Special targets for APM aerosol microphysics
apm	Build GEOS–Chem + APM code (synonym for "libapm exeapm")
libapm	Builds GEOS–Chem + APM objects & libraries in GeosApm directory
exeapm	Builds GEOS–Chem + APM executable in GeosApm directory
cleanapm	Removes *.o *.mod files in GeosApm directory only
Special targets for Mercury simulation
allhg	Build GEOS–Chem + GTMM code
Optional flags
Flag name	Description
COMPILER=___	Choice of the compiler. Options: ifort pgi sun xlf (default is ifort)
HDF5=yes	Enables writing diagnostic timeseries output to HDF5 files (default is no)
DEBUG=yes	Compiles GEOS–Chem for use w/ a debugger (default is no)
BOUNDS=yes	Turns on subscript-array checking (for debugging purposes) (default is no)
OMP=[yes|no]	Turns OpenMP parallelization on/off (default is yes)
IPO=yes	Turns on optmization options -ipo -static (default is no) (ifort only)
TRACEBACK=yes	Turns on -traceback option (default is no) (ifort only)
NONUMA=yes	Turns on -mp=nonuma option (pgi only)
CHEM=___	Specifies which simulation is done. Options: standard, SOA, isoprene (default is standard).
KPPSOLVER=___	Specifies the integrator used w/ KPP:          Options: lsodes radau5 rosenbrock runge_kutta (default is rosenbrock)
Make option
Option name	Description
-jN	-j enables you to compile GEOSChem using N processors simultaneously. This will shorten the total compilation time. (RECOMMENDED!)

When starting with a new version of GEOS–Chem, you should always issue this command:

make realclean

This will clear out any precompiled files and executables (if any have been left in the source code directory inadvertently).

The default target is all. This option, which will cause GNU Make to compile the GEOS–Chem source code and create the executable file, will be selected if you type either

make

or

make all

3.5.2 Optional flags and defaults

To build GEOS–Chem, you need to specify one of the makefile targets listed above (see Chapter 3.5.1), plus optional flags.

Several of the optional flags have default settings:

1. The default compiler is the Intel Fortran Compiler (ifort)
2. The default NOx-Ox-HC-aerosol chemistry (aka "full-chemistry) simulation is "standard" (53 advected tracers, no secondary organic aerosols)
3. The KPP chemical solver will will use the Rosenbrock integrator by default
4. HDF5 diagnostic files (for ND50, ND51, ND51b timeseries) will be turned off by default. (NOTE: This feature may be superseded by GEOS–Chem's netCDF I/O capabilities.)
5. OpenMP parallelization is turned on

Finally, any of the optional flags can be specified in one of two ways:

1. As a command-line argument to the GNU Make utility:
   For example, if we want to use the PGI compiler instead of the Intel Fortran Compiler, we could type:

   make COMPILER=pgi ...

2. As an Unix environment variable:
   If you want to use the PGI compiler, but do not wish to keep typing COMPILER=pgi every time you call GNU Make, you could type this instead:

   setenv COMPILER pgi
   make ...

For clarity, we shall use command-line arguments in the examples below.

3.5.3 Examples

To build the GEOS–Chem executable with all defaults (Intel Fortran Compiler, 53-tracer full-chemistry mechanism, OpenMP parallelization turned on) one can simply type:

make -j4

Here we have used the -j4 option to split the compilation among 4 processors (assuming you have this many processors available to you). This reduces the total compilation time drastically!

If you wanted to select a different compiler (e.g. PGI), but keep all of the other defaults, type:

make -j4 COMPILER=pgi

Depending on your PGI compiler setup, you may also have to add the PGI option NONUMA=yes.

If you wish to turn off the parallelization (e.g. for debugging purposes), you can type:

make -j4 OMP=no

To invoke the Intel Fortran Compiler's -traceback debugging option (which prints the list of routines that were being called at the point when the run died), type:

make -j4 TRACEBACK=yes

If you want to test for array-out-of-bounds errors, then type:

make -j4 BOUNDS=yes

If you wish to use a debugger (like Totalview) to debug a multi-processor GEOS–Chem simulation, type:

make -j4 DEBUG=yes

If you wish to use a debugger (like Totalview) to debug a single-processor GEOS–Chem simulation, type:

make -j4 DEBUG=yes OMP=no

If you wish to compile the code and build the executable in two separate steps, you may type:

make lib
make exe

To automatically generate the GEOS–Chem Reference Guide documentation in PostScript and PDF formats, type:

make doc

To remove all compiler-produced files (i.e. *.o, *.mod) and executable files from the source code directories (but not from the mod, lib, bin directories), type:

make clean

And to remove everything and start over from scratch, type:

make realclean

Please note:

1. On some systems, the GNU Make command will be called gmake instead of make.

2. We recommend updating your compiler to Intel Fortran Compiler (IFORT) Version 11.1.058 or higher versions. Version 11 produces similar results to Version 10.

3. We also recommend that PGI compiler users migrate to Intel Fortran Compiler (IFORT) Version 11.1.058 or higher.

4. If you wish to compile GEOS–Chem with the TOMAS aerosol microphysics option, you must be in the GeosTomas directory before invoking the make command. For more information on how to compile GEOS–Chem with the TOMAS microphysics option, see the Building GEOS–Chem with TOMAS wiki page.

5. For more information about debugging array-out-of-bounds errors, please see Chapter 7.2, GEOS–Chem debugging tips.

3.6 Advanced compilation techniques

Here we describe a couple of methods by which you may partially automate the compilation process.

3.6.1 Compiling GEOS–Chem with from Unix shell scripts

It is possible to write a shell script to compile the GEOS–Chem code, such as the following:

#!/bin/tcsh -f    # Script definition line
cd Code.v9-01-03  # your code dir
rm -f log         # clear log file
make -j4 > log    # build the code
exit(0)           # exit normally

You can then run this script interactively, or submit it to the queueing system on your computer cluster.

3.6.2 Compiling GEOS–Chem with a run-directory Makefile

The GNU Make utility is a powerful and flexible software development tool. With a little effort, you can design a Makefile that can be placed into your GEOS–Chem run directory that will create a GEOS–Chem executable and start a simulation. Here is an example:

# This makefile is a "router" makefile.  It calls the Makefile
# in the GEOS-Che code directory (with various user-supplied options) 
# to build and run the GEOS-Chem model.

# Define variables
SHELL   = /bin/sh

# Make the development version "Code" the default directory
ifndef GEOSDIR
GEOSDIR = /home/bmy/Code.v9-01-03
endif

# Date string DDDHHMM a la "trun"
DATE = `date +%j%H%M`

#=============================================================================
# Makefile targets: type "make help" for a complete list!
#=============================================================================

.PHONY: all geos clean realclean doc docclean help logclean

geos:
	@$(MAKE) -C $(GEOSDIR) 
	cp -f $(GEOSDIR)/bin/geos .
	rm -f log.$(DATE)
	./geos > log.$(DATE) &
	tail -f log.$(DATE)

clean:
	@$(MAKE) -C $(GEOSDIR) clean
	rm -f log*

realclean:
	@$(MAKE) -C $(GEOSDIR) realclean
	rm -f log*

doc:
	@$(MAKE) -C $(GEOSDIR) doc

docclean: 
	@$(MAKE) -C $(GEOSDIR) docclean

help:
	@$(MAKE) -C $(GEOSDIR) help

At the heart of this Makefile are two important blocks of code, the first being:

# Make the development version "Code" the default directory
ifndef GEOSDIR
GEOSDIR = /home/bmy/Code.v9-01-03
endif

# Date string DDDHHMM a la "trun"
DATE = `date +%j%H%M`

This block defines the location of the GEOS–Chem source code directory, which is saved in a variable called GEOSDIR. If a value for GEOSDIR is not passed as a command-line argument, then GEOSDIR will be set to /home/bmy/Code.v9-01-03 (you can edit this location for your own setup). Also, a default date/time string is saved in the variable DATE. This date/time string will be appended to the GEOS–Chem log file output.

The second block of code is used to compile and run the GEOS–Chem code:

geos:
	@$(MAKE) -C $(GEOSDIR) 
	cp -f $(GEOSDIR)/bin/geos .
	rm -f log.$(DATE)
	./geos > log.$(DATE) &
	tail -f log.$(DATE)

The lines:

	@$(MAKE) -C $(GEOSDIR) 
	cp -f $(GEOSDIR)/bin/geos .

tell GNU Make to call down to the "router" makefile in the GEOS–Chem source code directory /home/bmy/Code.v9-01-03. GNU Make will compile GEOS–Chem and create the executable file /home/bmy/Code.v9-01-03/bin/geos. This file is then copied to the run directory.

The next 3 lines:

	rm -f log.$(DATE)
	./geos > log.$(DATE) &
	tail -f log.$(DATE)

removes the log file (if it already exists), runs the GEOS–Chem executable (thus starting the simulation), and follows the log file output wit the Unix tail -f command.

For more information, please see:

• GNU Make manual
• Chapter 6: Running GEOS–Chem

Previous | Next | Printable View (no frames)