GEOS–Chem v9–02 Online User's Guide

Previous | Next | Printable View (no frames)

3. Compilation

3.1 Overview

Once you have downloaded the GEOS–Chem source code directory, you can compile GEOS–Chem. During this process, your Fortran-90 compiler will create a binary executable file from the instructions contained in the various source code modules and files. You can then place the executable file into a GEOS–Chem run directory and start a simulation.

When you compile GEOS–Chem, you invoke the GNU Make utility with command-line options that set various C–preprocessor switches. GNU Make manages the entire compilation process. A series of scripts called makefiles directs GNU Make to compile the GEOS–Chem source code files in the proper order and with the proper options. Each GEOS–Chem source code subdirectory contains its own makefile.

For more information about the GEOS–Chem source code and makefiles, please see:

• Chapter 7: Coding: Practice and Style.
• GEOS–Chem wiki: GEOS–Chem Makefile Structure

NOTE: In general, executables cannot be easily swapped between computers (unless they are identical systems). If you want to move GEOS–Chem to a different computer, you will more than likely have to recompile it on that system.

3.2 Source code directory structure

We (the GEOS–Chem Support Team) have subdivided the GEOS–Chem source code directory into several subdirectories. This was done for the following reasons:

1. To facilitate the installation of 3rd-party software packages such as:

   • KPP chemical solver
   • Aerosol microphysics codes (e.g. APM)
   • ISORROPIA II
   • Terrestrial models (e.g. CASA, Global Terrestrial Mercury Model), etc.

   into GEOS–Chem. Our guiding principle is that all 3rd-party software packages should be cleanly separatable from the mainline GEOS–Chem code. This will allow the 3rd-party software packages to be updated without having an impact on the rest of the GEOS–Chem code.

2. To simplify the maintenace of the GEOS–Chem code files. Without subdirectories, there would have been hundreds of source code files in a single directory, and it woud have been very difficult to keep track of them all.

The current directory structure is:

NOTE: In GEOS–Chem v9–02, the Code.v9-02/GeosTomas directory has been removed and code for the TOMAS aerosol microphysics package has been inlined into the GeosCore directory using C-preprocessor statements. See this wiki page for more information.

In the text below, we shall often omit the top-level Code.v9-02 directory and refer to files by their subdirectory only, such as:

• Headers/define.h,
• GeosCore/dao_mod.F
• ISOROPIA/isoropiaII_code.F, etc.

3.3 GEOS–Chem Makefiles

The GEOS–Chem compilation process is managed by makefiles located throughout the source code directory. Makefiles direct the GNU Make utility to execute the various compilation commands in the proper sequence.

In general there are 2 types of makefiles:

• Router makefiles: Makefiles that tell GNU Make to look at makefiles in other directories.

• Regular makefiles: Makefiles that are used by GNU Make to compile source code programs..

The top-level source code directory (Code.v9-02) contains a router makefile, which calls down to the regular makefiles in the various subdirectories. In each subdirectory, the regular makefile tells GNU Make how to compile the source code files. When the source code in all subdirectories has been compiled, the top-level Makefile tells GNU Make to create the GEOS–Chem executable file.

Here is a list of makefiles used by GEOS–Chem:

You will usually not have to modify any of these makefiles, unless you are adding new code into GEOS–Chem. If you need to modify makefiles, then we will be happy to assist you.

Each makefile controls several compilation options, which are discussed in detail below.

3.4 Setting the C–preprocessor switches

GEOS–Chem relies on several different C–preprocessor switches to inform the Fortran-90 compiler which options you want to include into the executable file. In this section we provide an overview of how C–preprocessor switches work.

3.4.1 What is the C–preprocessor?

The C–preprocesor (or CPP for short) is a feature of the C and C++ programming languages that allows you to compile blocks of code only if certain conditions are met. For example, let's say you have two blocks of code, Block A and Block B. If you set a certain switch in the code, then Block A will get compiled into the executable but Block B will not. When you run the executable, it will behave as if Block B had never existed. If you set a different switch, then Block B will be compiled but Block A will not, and the executable will behave as if Block A had never existed.

All modern Fortran compilers have inherited the C–preprocessor feature. Some Fortran compilers even call it the "Fortran preprocessor" (FPP). Therefore, even though GEOS–Chem is written in Fortran-90, we can still incorporate C–preprocesor commands into the source code.

With the C–preprocessor, you can design software to handle many different conditions, such as multiple compiler/platform combinations, different file formats, and the like. At compilation time, you can direct the C–preprocessor to only choose the options relevant to the particular computer system you are running on. Options that are not relevant to your system will be ignored. This is a standard programming technique.

The C–preprocessor statments take the form of #define and #if - #endif statements. A simple example from GEOS–Chem is:

#if   defined( GEOS_4 )
      WRITE( 6, '(a)' ) 'Using GMAO GEOS-4 met fields'
#elif defined( GEOS_5 )
      WRITE( 6, '(a)' ) 'Using GMAO GEOS-5 met fields'
#elif defined( GEOS_FP )
      WRITE( 6, '(a)' ) 'Using GMAO GEOS-FP met fields'
#elif defined( MERRA )
      WRITE( 6, '(a)' ) 'Using GMAO MERRA met fields'
#elif defined( GCAP  )
      WRITE( 6, '(a)' ) 'Using GCAP/GISS met fields'
#endif

Here the #elif tag stands for "else if". An #else tag is also allowed.

If we have #defined the GEOS_5 C–preprocessor switch, then GEOS–Chem will be compiled for use with the GEOS–5 met fields. The block of code that prints out the message "Using GMAO GEOS-5 met fields" will be built into the exectuable, and all other print statements will be ignored.

C–preprocessor statements must be placed into the first column of a Fortran source code file, or else you may encounter a syntax error.

3.4.2 Selecting the proper C–preprocessor switches for your simulation

GEOS–Chem uses C–preprocessor switches to select various options, including:

1. The meteorological fields that GEOS–Chem will use
2. The horizontal and vertical grids for the GEOS–Chem simulation
3. The type of compiler you will use to create the GEOS–Chem executable
4. Options for special simulations (e.g. mercury)
5. Debugging options

These switches are set with Makefile options from the command line.

3.5 Compiling the GEOS–Chem source code

To compile GEOS–Chem, you need to be in the Code.v9-02 top-level directory. There are a large choice of compilation options available. The list of options can be printed to the screen by typing:

make help

3.5.1 Makefile options

To compile GEOS–Chem, you will invoke GNU Make with the following syntax:

make -jN TARGET REQUIRED-FLAGS [ OPTIONAL-FLAGS ]

In the table below, we list all of the targets and flags that you can use to customize your GEOS–Chem simulation.

Make options
Option	Description
-jN	Enables you to compile N files simultaneously. This will shorten the total compilation time. (RECOMMENDED!)
Targets
Target name	Description
all	Default target (synonym for "lib exe")
lib	Builds all GEOS–Chem source code (synonym for "libkpp libutil libcore")
libcore	Only builds GEOS–Chem objects & libraries in GeosCore subdirectory
libiso	Only builds GEOS–Chem objects & libraries in ISOROPIA subdirectory
libkpp	Only builds GEOS–Chem objects & libraries in KPP subdirectory
libnc	Only builds GEOS–Chem objects & libraries in NcdfUtil subdirectory
ncdfcheck	Checks that your netCDF library has been installed properly
libutil	Only builds GEOS–Chem objects & libraries in GeosUtil subdirectory
libheaders	Only builds GEOS–Chem objects & libraries in Headers subdirectory
exe	Build GEOS–Chem executable
clean	Removes *.o, *.mod, geos* files in source code subdirs only
realclean	Removes all *.o, *.mod, geos*, lib*.a, *.tex, *.ps, *.pdf files everywhere
doc	Builds GEOS–Chem documentation (*.ps, *.pdf) with ProTeX in doc subdir
docclean	Removes *.tex, *.ps, *.pdf from the doc subdir
help	Displays the help screen
Optional targets for the APM aerosol microphysics package
apm	Builds GEOS–Chem + APM code (synonym for "libapm exeapm")
libapm	Builds GEOS–Chem + APM objects & libraries in GeosApm directory
exeapm	Builds GEOS–Chem + APM executable in GeosApm directory
cleanapm	Removes *.o *.mod files in GeosApm directory only
Optional targets for the Global Terrestrial Mercury Model
allhg	Builds GEOS–Chem + GTMM code (synonym for "libhg exehg")
libhg	Builds GEOS–Chem + GTMM objects & libraries in GeosCore and GTMM directories
libgtmm	Builds GEOS–Chem + GTMM objects & libraries in GTMM directory
exehg	Builds GEOS–Chem + GTMM executable in GTMM directory
Required flags
Flag name	Description
MET=___	Select the meteorological field type that GEOS–Chem will use. Options are: gcap: Use GISS met fields for GCAP simulation (23 layer, 4° x 5°) geos4: Use GEOS–4 met fields (55 layer, hybrid, 1985–2006) geos5: Use GEOS–5 met fields (72 layer, hybrid, 2005–2012) geos-fp: Use GEOS–FP met fields (72 layer, hybrid, 2012–present) merra: Use MERRA met fields (72 layer, hybrid, 1979–2010)
GRID=___	Select the horizontal grid for the GEOS–Chem simulation. Options are: 025x03125: Use 0.25° x 0.3125° nested grids (GEOS–FP only) 05x0667: Use 0.5° x 0.667 ° nested grids (GEOS–5 only) 1x125: Use 1° x 1.25° grid (GEOS–4 only) 2x25: Use 2° x 2.5° grid 4x5:: Use 4° x 5° grid
Optional flags
Flag name	Description
COMPILER=___	Choice of the compiler. Options: ifort pgi (Default is ifort)
NEST=___	Select the nested-grid domain for the GEOS–Chem simulation. Options are: ch: Use CH (China) nested grid (GEOS–5 and GEOS–FP only) eu: Use EU (Europe) nested grid (GEOS–5 and GEOS–FP only) na: Use NA (North America) nested-grid (GEOS–5 and GEOS–FP only) se: Use extended SE Asia nested grid (GEOS–FP only)
NO_REDUCED=yes	Compiles GEOS–Chem with the full vertical grid (no)or reduced vertical grid. (Default is no)
Parallelization and optimization options
OMP=yes	Turns OpenMP parallelization on/off (Default is yes)
NONUMA=yes	Turns on -mp=nonuma option (pgi only)
IPO=yes	Turns on optmization options -ipo -static for the ifort compiler. (Default is no)
OPT=___	Specifies a different optimization level than the default setting. (Default is -O2)
KPP chemistry solver options
CHEM=___	Specifies which simulation is done. Options: standard, SOA (Default is standard).
KPPSOLVER=___	Specifies the integrator used w/ KPP:          Options: lsodes radau5 rosenbrock runge_kutta (Default is rosenbrock)
Debugging options
TRACEBACK=yes	Turns on -traceback, which prints the error trace stack (Default is no)
DEBUG=yes	Compiles GEOS–Chem for use w/ a debugger (Default is no)
BOUNDS=yes	Turns on subscript-array checking (Default is no)
FPE=yes	Turns on checking for floating-point exceptions (Default is no)
TOMAS aerosol microphysics package options
TOMAS=yes	Build GEOS–Chem for 30-bin TOMAS aerosol microphysics
TOMAS12=yes	Build GEOS–Chem for 12-bin TOMAS aerosol microphysics
TOMAS15=yes	Build GEOS–Chem for 15-bin TOMAS aerosol microphysics
TOMAS40=yes	Build GEOS–Chem for 40-bin TOMAS aerosol microphysics

Please note the following:

1. You must specify a value for MET. Allowable values are: MET=gcap MET=geos4 MET=geos5 MET=geos-fp MET=merra.

2. You must specify a value for GRID. Allowable values are: GRID=025x03125 GRID=05x0666 GRID=2x25 GRID=4x5.

3. If you have selected GRID=025x03125 or GRID=05x0666, then you must also select NEST=ch, NEST=na, NEST=eu, or NEST=se. This will select the proper 0.25° x 0.3125° or 0.5° x 0.666° nested grid region (CH, NA, EU, SE)

4. Makefile target names are case-sensitive. For example, you should type make all but not make ALL.

5. Makefile flags will accept case-insensitive input. You may also omit dashes from met field names. The following permutations are acceptable: MET=geos5 MET=Geos5 MET=GEOS-5 MET=geosfp MET=Geos_Fp, etc.

6. Makefile flags that require a simple yes/no answer will accept case-insensitive input (e.g. DEBUG=yes DEBUG=Yes DEBUG=y, DEBUG=No, DEBUG=n).

7. Whenevery you want to change the met field type or horizontal grid settings, you should issue a make realclean command. This will remove all previously-created compiler output files (e.g. *.a, *.o, *.mod) and executables. After doing make realclean, you can recompile GEOS–Chem again with your new options.

When starting with a new version of GEOS–Chem, you should always issue this command:

make realclean

This will clear out any precompiled files and executables (if any have been left in the source code directory inadvertently).

The default target is all. This option, which will cause GNU Make to compile the GEOS–Chem source code and create the executable file, will be selected if you type either

make

or

make all

3.5.2 Optional flags and defaults

To build GEOS–Chem, you need to specify one of the makefile targets listed above (see Chapter 3.5.1), plus optional flags.

Several of the optional flags have default settings:

1. The default compiler is the Intel Fortran Compiler (ifort)
2. The default NOx-Ox-HC-aerosol chemistry (aka "full-chemistry) simulation is "standard" (66 advected tracers, no secondary organic aerosols)
3. The KPP chemical solver will use the Rosenbrock integrator by default
4. OpenMP parallelization is turned on

Finally, any of the optional flags can be specified in one of two ways:

1. As a command-line argument to the GNU Make utility:
   For example, if we want to use the PGI compiler instead of the Intel Fortran Compiler, we could type:

   make -j4 MET=geos5 GRID=4x5 COMPILER=pgi ...

2. As an Unix environment variable:
   If you want to use the PGI compiler, but do not wish to keep typing COMPILER=pgi every time you call GNU Make, you could type this instead:

   setenv MET geos5
   setenv GRID 4x5
   setenv COMPILER pgi
   make -j4 ... 

For clarity, we shall use command-line arguments in the examples below.

3.5.3 Examples

To build the GEOS–Chem executable with all defaults (Intel Fortran Compiler, 66-tracer full-chemistry mechanism, OpenMP parallelization turned on) one can simply type:

make -j4 MET=geos5 GRID=4x5

Here we have used the -j4 option to split the compilation among 4 processors (assuming you have this many processors available to you). This reduces the total compilation time drastically! Here we've used use MET=geos5 and GRID=4x5 but you can specify different values for these flags.

If you wanted to select a different compiler (e.g. PGI), but keep all of the other defaults, type:

make -j4 MET=geos5 GRID=4x5 COMPILER=pgi

Depending on your PGI compiler setup, you may also have to add the PGI option NONUMA=yes.

If you wish to turn off the parallelization (e.g. for debugging purposes), you can type:

make -j4 MET=geos5 GRID=4x5 OMP=no

To invoke the Intel Fortran Compiler's -traceback debugging option (which prints the list of routines that were being called at the point when the run died), type:

make -j4 MET=geos5 GRID=4x5 TRACEBACK=yes

If you want to test for array-out-of-bounds errors, then type:

make -j4 MET=geos5 GRID=4x5 BOUNDS=yes

If you wish to use a debugger (like Totalview) to debug a multi-processor GEOS–Chem simulation, type:

make -j4 MET=geos5 GRID=4x5 DEBUG=yes

If you wish to use a debugger (like Totalview) to debug a single-processor GEOS–Chem simulation, type:

make -j4 MET=geos5 GRID=4x5 DEBUG=yes OMP=no

If you wish to compile the code and build the executable in two separate steps, you may type:

make -j4 MET=geos5 GRID=4x5 lib
make -j4 MET=geos5 GRID=4x5 exe

To automatically generate the GEOS–Chem Reference Guide documentation in PostScript and PDF formats, type:

make doc

To remove all compiler-produced files (i.e. *.o, *.mod) and executable files from the source code directories (but not from the mod, lib, bin directories), type:

make clean

And to remove everything and start over from scratch, type:

make realclean

Please note:

1. On some systems, the GNU Make command will be called gmake instead of make.

2. We recommend updating your compiler to Intel Fortran Compiler (IFORT) Version 11.1.058 or higher versions. Version 11 produces similar results to Version 10.

3. We also recommend that PGI compiler users migrate to Intel Fortran Compiler (IFORT) Version 11.1.058 or higher.

4. For more information about debugging array-out-of-bounds errors, please see Chapter 7.2, GEOS–Chem debugging tips.

3.6 Advanced compilation techniques

Here we describe a couple of methods by which you may partially automate the compilation process.

3.6.1 Compiling GEOS–Chem with from Unix shell scripts

It is possible to write a shell script to compile the GEOS–Chem code, such as the following:

#!/bin/tcsh -f                       # Script definition line
cd Code.v9-02                        # your code dir
rm -f log                            # clear log file
make -j4 MET=geos5 GRID=4x5 > log    # build the code
exit(0)                              # exit normally

You can then run this script interactively, or submit it to the queueing system on your computer cluster.

3.6.2 Compiling GEOS–Chem with a run-directory Makefile

The GNU Make utility is a powerful and flexible software development tool. With a little effort, you can design a Makefile that can be placed into your GEOS–Chem run directory that will create a GEOS–Chem executable and start a simulation. Here is an example:

# This makefile is a "router" makefile.  It calls the Makefile
# in the GEOS-Chem code directory (with various user-supplied options) 
# to build and run the GEOS-Chem model.

# Define variables
SHELL   = /bin/bash

# Make the development version "Code" the default directory
ifndef GEOSDIR
GEOSDIR = /home/bmy/Code.v9-02
endif

# Date string DDDHHMM a la "trun"
DATE = `date +%j%H%M`

#=============================================================================
# Makefile targets: type "make help" for a complete list!
#=============================================================================

.PHONY: all geos clean realclean doc docclean help logclean

geos:
	@$(MAKE) -C $(GEOSDIR) 
	cp -f $(GEOSDIR)/bin/geos .
	rm -f log.$(DATE)
	./geos > log.$(DATE) &
	tail -f log.$(DATE)

clean:
	@$(MAKE) -C $(GEOSDIR) clean
	rm -f log*

realclean:
	@$(MAKE) -C $(GEOSDIR) realclean
	rm -f log*

doc:
	@$(MAKE) -C $(GEOSDIR) doc

docclean: 
	@$(MAKE) -C $(GEOSDIR) docclean

help:
	@$(MAKE) -C $(GEOSDIR) help

At the heart of this Makefile are two important blocks of code, the first being:

# Make the development version "Code" the default directory
ifndef GEOSDIR
GEOSDIR = /home/bmy/Code.v9-02
endif

# Date string DDDHHMM a la "trun"
DATE = `date +%j%H%M`

This block defines the location of the GEOS–Chem source code directory, which is saved in a variable called GEOSDIR. If a value for GEOSDIR is not passed as a command-line argument, then GEOSDIR will be set to /home/bmy/Code.v9-02 (you can edit this location for your own setup). Also, a default date/time string is saved in the variable DATE. This date/time string will be appended to the GEOS–Chem log file output.

The second block of code is used to compile and run the GEOS–Chem code:

geos:
	@$(MAKE) -C $(GEOSDIR) 
	cp -f $(GEOSDIR)/bin/geos .
	rm -f log.$(DATE)
	./geos > log.$(DATE) &
	tail -f log.$(DATE)

The lines:

	@$(MAKE) -C $(GEOSDIR) 
	cp -f $(GEOSDIR)/bin/geos .

tell GNU Make to call down to the router makefile in the GEOS–Chem source code directory /home/bmy/Code.v9-02. GNU Make will compile GEOS–Chem and create the executable file /home/bmy/Code.v9-02/bin/geos. This file is then copied to the run directory.

The next 3 lines:

	rm -f log.$(DATE)
	./geos > log.$(DATE) &
	tail -f log.$(DATE)

removes the log file (if it already exists), runs the GEOS–Chem executable (thus starting the simulation), and follows the log file output wit the Unix tail -f command.

For more information, please see:

• GNU Make manual
• Chapter 6: Running GEOS–Chem

Previous | Next | Printable View (no frames)