List routines by category:
Atmospheric Sciences |
Benchmarking |
Color |
Date/Time |
Doc |
File & I/O |
BPCH Format |
Scientific Data Formats |
GAMAP Examples |
GAMAP Internals |
GAMAP Utilities |
GAMAP Data Manipulation |
GAMAP Models &
Grids |
GAMAP Plotting |
General |
Graphics |
Math & Units |
Plotting |
Regridding |
Strings |
Structures |
Time Series
List routines by alphabetical order:
A |
B |
C |
D |
E |
F |
G |
H |
I |
J |
K |
L |
M |
N |
O |
P |
Q |
R |
S |
T |
U |
V |
W |
X |
Y |
Z
NAME:
BPCH2COARDS
PURPOSE:
Reads data from a binary punch file and saves it in a
COARDS-compliant netCDF (network Common Data Format) file.
NOTE: COARDS is a formatting standard for netCDF files which
is widely used in both the atmospheric & climate communities.
COARDS-compliant netCDF files can be read by GAMAP, GrADS and
other plotting packages.
See http://ferret.wrc.noaa.gov/noaa_coop/coop_cdf_profile.html
for more information about the COARDS standard.
CATEGORY:
File & I/O, BPCH Format, Scientific Data Formats
CALLING SEQUENCE:
BPCH2COARDS, INFILE, OUTFILE [, Keywords ]
INPUTS:
INFILE -> Name of the binary punch file to read. If INFILE
is not passed, the user will be prompted to supply a
file name via a dialog box.
OUTFILE -> Name of the netCDF file to be written. It is
recommended to insert the tokens %DATE% (or %date%)
into OUTFILE, since BPCH2COARDS will write a separate
netCDF file for each unique YYYYMMDD value contained
within the *.bpch file. If OUTFILE is not specified,
BPCH2COARDS will use the default file name
"coards.%DATE%.nc".
KEYWORD PARAMETERS:
DIAGN -> Array of diagnostic categories from the bpch file
to save to netCDF format. If omitted, BPCH2COARDS will
save all diagnostic categories.
/VERBOSE -> If set, will print the name of each tracer
as it is being written to the netCDF file. Useful
for debugging purposes.
/NC4 -> Write a netCDF4 file instead of netCDF3. Default is
NetCDF3. NetCDF4 support requires IDL 8.0 or later.
COMPRESS -> Integer 0-9 specifies amount of compression in
netCDF4 files. Default is 2, with very little benefit
for higher compression.
/PCOORD -> Use mean pressure as the vertical coordinate rather
sigma or eta
/ALTCOORD -> Use mean altitude as the vertical coordinate
rather than sigma or eta
/TROPONLY -> Write only tropospheric layers
/ONEFILE -> Write all data to one netCDF output file.
Default is one file per calendar day.
_EXTRA=e -> Picks up additional keywords for NCDF_SET
OUTPUTS:
None
SUBROUTINES:
External Subroutines Required:
============================================
CTM_GET_DATA TAU2YYMMDD (function)
UNDEFINE REPLACE_TOKEN (function)
STRREPL (function) GETMODELANDGRIDINFO
REQUIREMENTS:
(1) References routines from GAMAP and TOOLS packages.
(2) You must use a version of IDL containing the NCDF routines.
NOTES:
(1) BPCH2COARDS assumes that each data block in the *.bpch
file is either 2-D (lon-lat) or 3-D (lon-lat-alt).
(2) BPCH2COARDS assumes that the number type of each data
block in the *.bpch file is REAL*4 (a.k.a. FLOAT).
(3) BPCH2COARDS assumes that all data blocks in the *.bpch
file adhere to same horizontal grid. This will always
be true for output files from the GEOS-CHEM model.
(4) BPCH2COARDS will write a separate COARDS-compliant netCDF
file corresponding to each unique YYYYMMDD date. This
prevents the files from becoming too large to be read
with IDL.
(5) BPCH2COARDS will replace the %DATE% (or %date%) token with
the current YYYYMMDD value. Therefore, it is recommended
to insert this token into the string passed via OUTFILE.
(6) BPCH2COARDS will write arrays containing the latitudes,
longitudes to the netCDF file. For 3-D data blocks,
the eta or sigma centers will also be written to the
file. Time will be written as TAU values (i.e. hours
since 00:00 GMT on 01-Jan-1985.
(7) The netCDF library has apparently been updated in
IDL 6.0+. The result is that variable names containing
characters such as '$', '=', and ':' may now cause an
error in NCDF_VARDEF. Therefore, we now pre-screen
tracer names with function NCDF_VALID_NAME.
EXAMPLE:
BPCH2COARDS, 'myfile.bpch', 'myfile.%DATE%.nc'
; Will write the contents of "myfile.bpch" to one
; or more COARDS-compliant netCDF files adhering
; to the filename convention "myfile.YYYYMMDD.nc"
MODIFICATION HISTORY:
rjp & bmy, 17 Mar 2005: GAMAP VERSION 2.03
- Based on bpch2nc.pro
bmy, 21 Jul 2005: GAMAP VERSION 2.04
- Bug fix:
bmy, 13 Jul 2006: GAMAP VERSION 2.05
- Remove call to PTR_FREE
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
- Now count GCAP among the GEOS family
for the purpose of converting the TAU
date to a YYYY/MM/DD date.
phs, 29 Oct 2009: GAMAP VERSION 2.14
- Can process files with 3D data on both
centers and edges of the grid boxes.
bmy, 19 Dec 2011: GAMAP VERSION 2.16
- Now handles multiple vertical dimensions
in the bpch file properly.
- Bug fix: now write vertical levels edges to
the file.
bmy, 27 Sep 2012: - Bug fix: Now handle data blocks that straddle
the date line.
bmy, 05 Nov 2013: GAMAP VERSION 2.17
- Change attributes for better COARDS compliance
bmy, 12 Feb 2014: GAMAP VERSION 2.18
- Add more modifications for 4-D data blocks
from bpch files created w/ GC_COMBINE_ND49
bmy, 03 Mar 2015: - Now define dims in order: time, lev, lon, lat,
which is more COARDS compliant.
cdholmes, 29 Mar 2017 - Add support for NetCDF4 and
compression
- Add support for pressure and altitude
as vertical coordinate
- ONEFILE puts all data into a single
output file
- TROPONLY limits the output to
troposphere layers
- Singleton "altXXX" dimensions are avoided
(See /n/home09/ryantosca/IDL/gamap2/file_io/bpch2coards.pro)
NAME:
BPCH2GMI
PURPOSE:
Reads data from a binary punch file and saves it in
netCDF (network Common Data Format) format. The data
will be shifted so that the first longitude is 0 degrees
(i.e. the prime meridian) in order to conform with the
GMI (Global Model Initiative) model grid definition.
CATEGORY:
File & I/O, BPCH Format, Scientific Data Formats
CALLING SEQUENCE:
BPCH2GMI, INFILE, OUTFILE [, Keywords ]
INPUTS:
INFILE -> Name of the binary punch file to read. If
INFILE is not passed, the user will be prompted
to supply a file name via a dialog box.
OUTFILE -> Name of the netCDF file to be written. It is
recommended to insert the tokens %DATE% and %TIME%
into OUTFILE, since BPCH2NC will write a separate
netCDF file for each time index in the *.bpch file.
The tokens %DATE% and %TIME% will be overwritten
with the current values of YYYYMMDD and HHMMSS.
Default is "bpch2nc_output.%DATE%.%TIME%.nc".
KEYWORD PARAMETERS:
DIAGN -> A diagnostic category name (e.g. "IJ-AVG-$") or
array of names which will restrict the data block
selection. If DIAGN is omitted, then all data blocks
within INFILE will be saved in netCDF format to OUTFILE.
/VERBOSE -> If set, will print the names of each tracer
as it is being written to the netCDF file.
_EXTRA=e -> Picks up additional keywords for netCDF routines
OUTPUTS:
SUBROUTINES:
Internal Subroutines:
============================================
B2G_Valid_VarName (function)
B2G_SetNcDim (function)
B2G_GetNcDim (function)
External Subroutines Required:
============================================
CTM_GET_DATA TAU2YYMMDD (function)
UNDEFINE REPLACE_TOKEN (function)
STRREPL (function) GETMODELANDGRIDINFO
REQUIREMENTS:
(1) References routines from GAMAP and TOOLS packages.
(2) You must use a version of IDL containing the NCDF routines.
NOTES:
(1) BPCH2GMI assumes that each data block in the *.bpch file
is either 2-D (lon-lat) or 3-D (lon-lat-alt).
(2) BPCH2GMI assumes that the number type of each data block
in the *.bpch file is REAL*4 (a.k.a. FLOAT).
(3) BPCH2GMI assumes that all data blocks in the *.bpch file
file adhere to same horizontal grid. This will always
be true for output files from the GEOS-CHEM model.
(4) BPCH2GMI will write a separate NC file corresponding
to each time index in the *.bpch file. This prevents
file sizes from getting large, especially if there is
a lot of diagnostic output in the *.bpch file.
(5) BPCH2GMI will replace the %DATE% token with the
current YYYYMMDD value, and will replace the %TIME%
token with the current HHMMSS value. Therefore, it
is recommended to insert these tokens into the string
passed via OUTFILE. The tokens %DATE% and %TIME% tokens
may also be passed in lowercase (e.g, %date%, %time% ).
(6) BPCH2GMI will write arrays containing the latitudes,
longitudes to the netCDF file. For 3-D data blocks,
the sigma centers will also be written to the file.
Date and time are stored as global attributes.
(7) The netCDF library has apparently been updated in
IDL 6.0+. The result is that variable names containing
characters such as '$', '=', and ':' may now cause an
error in NCDF_VARDEF. Therefore, we now pre-screen
tracer names with function NCDF_VALID_NAME.
EXAMPLE:
BPCH2GMI, 'myfile.bpch', 'myfile.%DATE%.%TIME%.nc'
; Will write the contents of "myfile.bpch" to one
; or more netCDF files "myfile.YYYYMMDD.HHMMSS.nc"
MODIFICATION HISTORY:
bmy & phs, 20 Aug 2007: GAMAP VERSION 2.10
- Based on BPCH2NC
bmy, 19 Dec 2007: GAMAP VERSION 2.12
- Now save sigma edges & centers or
eta edges & centers to the file.
- Extra error trap, if there is only one
level in the file then set IS_3D=0.
bmy, 20 Dec 2011: GAMAP VERSION 2.16
- Changed default filename to "bpch2
(See /n/home09/ryantosca/IDL/gamap2/file_io/bpch2gmi.pro)
NAME:
BPCH2HDF
PURPOSE:
Reads data from a binary punch file and saves it in HDF-SD
(Hierarchical Data Format, Scientific Dataset) format.
CATEGORY:
File & I/O, BPCH Format, Scientific Data Formats
CALLING SEQUENCE:
BPCH2HDF, INFILE, OUTFILE [, Keywords ]
INPUTS:
INFILE -> Name of the binary punch file to read. If
INFILE is not passed, the user will be prompted
to supply a file name via a dialog box.
OUTFILE -> Name of the HDF file to be written. It is
recommended to insert the tokens %DATE% and %TIME%
into OUTFILE, since BPCH2HDF will write a separate
HDF file for each time index in the *.bpch file.
The tokens %DATE% and %TIME% will be overwritten
with the current values of YYYYMMDD and HHMMSS.
Default is "bpch2hdf_output.%DATE%.%TIME%.hdf".
KEYWORD PARAMETERS:
DIAGN -> A diagnostic category name (e.g. "IJ-AVG-$") or
array of names which will restrict the data block
selection. If DIAGN is omitted, then all data blocks
within INFILE will be saved in HDF format to OUTFILE.
_EXTRA=e -> Picks up additional keywords for HDF_SETSD
OUTPUTS:
SUBROUTINES:
External Subroutines Required:
=========================================
CTM_GET_DATA TAU2YYMMDD (function)
UNDEFINE REPLACE_TOKEN (function)
HDF_SETSD GETMODELANDGRIDINFO
REQUIREMENTS:
(1) References routines from GAMAP and TOOLS packages.
(2) You must use a version of IDL containing the HDF-SD routines.
NOTES:
(1) BPCH2HDF assumes that each data block in the *.bpch file
is either 2-D (lon-lat) or 3-D (lon-lat-alt).
(2) BPCH2HDF assumes that all data blocks in the *.bpch file
file adhere to same horizontal grid. This will be true
for output files from the GEOS-CHEM model.
(3) BPCH2HDF will write a separate HDF file corresponding
to each time index in the *.bpch file. This prevents
file sizes from getting large, especially if there is
a lot of diagnostic output in the *.bpch file.
(4) BPCH2HDF will replace the %DATE% token with the
current YYYYMMDD value, and will replace the %TIME%
token with the current HHMMSS value. Therefore, it
is recommended to insert these tokens into the string
passed via OUTFILE. These tokens may be in either
uppercase or lowercase.
(4) BPCH2HDF will also write arrays containing the latitudes,
longitudes, sigma coordinates (for 3-D data blocks only!)
to the HDF file.
(5) BPCH2HDF will write arrays containing the latitudes,
longitudes to the netCDF file. For 3-D data blocks,
the sigma centers and sigma edges will also be written
to the file.
EXAMPLE:
BPCH2HDF, 'myfile.bpch', 'myfile.%DATE%.%TIME%.hdf'
; Will write the contents of "myfile.bpch" to
; one or more HDF files "myfile.YYYYMMDD.HHMMSS.hdf"
MODIFICATION HISTORY:
bmy, 22 May 2002: GAMAP VERSION 1.50
bmy, 22 Oct 2002: GAMAP VERSION 1.52
- bug fix: now do not write vertical layer
dim info to HDF file for 2-D grids
bmy, 22 May 2003: GAMAP VERSION 1.53
- Make sure LONGNAME is not a null string
bmy, 18 Sep 2003: - Call PTR_FREE to free the pointer memory
bmy, 03 Jun 2004: GAMAP VERSION 2.02
- now pass extra keywords to CTM_GET_DATA
via _EXTRA=e keyword
bmy, 03 Sep 2004: GAMAP VERSION 2.03
- now defines ETAC and ETAE variables
for hybrid grids
bmy, 03 Dec 2004: GAMAP VERSION 2.03
- add DIAGN keyword (passed to CTM_GET_DATA)
in order to refine data block search
bmy, 19 May 2006: GAMAP VERSION 2.05
- Now do not free the pointer memory
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
- Now count GCAP among the GEOS family
for the purpose of converting the TAU
date to a YYYY/MM/DD date.
bmy, 18 Feb 2009: GAMAP VERSION 2.13
- Bug fix: should be N_ELEMENTS( DiagN )
instead of N_ELEMENTS( Category ) in the
IF statement for CTM_GET_DATA.
(See /n/home09/ryantosca/IDL/gamap2/file_io/bpch2hdf.pro)
NAME:
BPCH2NC
PURPOSE:
Reads data from a binary punch file and saves it in
netCDF (network Common Data Format) format.
CATEGORY:
File & I/O, BPCH Format, Scientific Data Formats
CALLING SEQUENCE:
BPCH2NC, INFILE, OUTFILE [, Keywords ]
INPUTS:
INFILE -> Name of the binary punch file to read. If
INFILE is not passed, the user will be prompted
to supply a file name via a dialog box.
OUTFILE -> Name of the netCDF file to be written. It is
recommended to insert the tokens %DATE% and %TIME%
into OUTFILE, since BPCH2NC will write a separate
netCDF file for each time index in the *.bpch file.
The tokens %DATE% and %TIME% will be overwritten
with the current values of YYYYMMDD and HHMMSS.
Default is "bpch2nc_output.%DATE%.%TIME%.nc".
KEYWORD PARAMETERS:
DIAGN -> A diagnostic category name (e.g. "IJ-AVG-$") or
array of names which will restrict the data block
selection. If DIAGN is omitted, then all data blocks
within INFILE will be saved in netCDF format to OUTFILE.
/VERBOSE -> If set, will print the names of each tracer
as it is being written to the netCDF file.
_EXTRA=e -> Picks up additional keywords for NCDF_SET
OUTPUTS:
SUBROUTINES:
Internal Subroutines:
============================================
B2N_Valid_VarName (function)
B2N_SetNcDim (function)
B2N_GetNcDim (function)
External Subroutines Required:
============================================
CTM_GET_DATA TAU2YYMMDD (function)
UNDEFINE REPLACE_TOKEN (function)
STRREPL (function) GETMODELANDGRIDINFO
REQUIREMENTS:
(1) References routines from GAMAP and TOOLS packages.
(2) You must use a version of IDL containing the NCDF routines.
NOTES:
(1) BPCH2NC assumes that each data block in the *.bpch file
is either 2-D (lon-lat) or 3-D (lon-lat-alt).
(2) BPCH2NC assumes that the number type of each data block
in the *.bpch file is REAL*4 (a.k.a. FLOAT).
(3) BPCH2NC assumes that all data blocks in the *.bpch file
file adhere to same horizontal grid. This will always
be true for output files from the GEOS-CHEM model.
(4) BPCH2NC will write a separate NC file corresponding
to each time index in the *.bpch file. This prevents
file sizes from getting large, especially if there is
a lot of diagnostic output in the *.bpch file.
(5) BPCH2NC will replace the %DATE% token with the
current YYYYMMDD value, and will replace the %TIME%
token with the current HHMMSS value. Therefore, it
is recommended to insert these tokens into the string
passed via OUTFILE. The tokens %DATE% and %TIME% tokens
may also be passed in lowercase (e.g, %date%, %time% ).
(6) BPCH2NC will write arrays containing the latitudes,
longitudes to the netCDF file.
(7) For pure-sigma grids (e.g. GEOS-1, GEOS-STRAT, GEOS-3),
BPCH2NC will write the following additional variables to
the netCDF file:
(1) SIGE : Sigma coordinate at level edges
(2) SIGC : Sigma coordinate at level centers
The pressure at the bottom edge of level L is given by:
Pe(L) = Ptop + [ SIGE(L) * ( Psurface - Ptop ) ]
and the pressure at the vertical center of level L is:
Pc(L) = Ptop + [ SIGC(L) * ( Psurface - Ptop ) ]
%%%%% NOTE: This is mostly obsolete, as most met fields
%%%%% that are used to drive CTM's are now placed onto
%%%%% hybrid grids.
(8) For hybrid grids (e.g. GEOS-4, GEOS-5, MERRA), the following
dditional variables will be written to the netCDF file:
(1) ETAE : Eta coordinate on level edges
(2) ETAC : Eta coordinate on level centers
(3) Ap : Hybrid grid "A" parameter
(4) Bp : Hybrid grid "B" parameter
The pressure at the bottom edge of level L is given by:
Pe(L) = Ap(L) + ( Bp(L) * Psurface )
and the pressure at the vertical center of level L is:
Pc(L) = ( Pe(L) + Pe(L+1) ) * 0.5
The hybrid ETA coordinates (similar to sigma) at the
level edges and centers are, correspondingly:
ETAE(L) = ( Pe(L) - Ptop ) / ( Psurface - Ptop )
ETAC(L) = ( Pc(L) - Ptop ) / ( Psurface - Ptop )
%%%%% NOTE: The ETAe and ETAc values are only approximate
%%%%% and are computed with a surface pressure of 1013.25
%%%%% hPa. For your scientific analysis, you should compute
%%%%% the pressures at level edges from Ap, Bp, and a
%%%%% spatially-varying surface pressure field (e.g. saved
%%%%% out from GEOS-Chem or another model).
(7) The date and time of the data are stored in the netCDF
file as global attributes.
(8) The netCDF library has apparently been updated in
IDL 6.0+. The result is that variable names containing
characters such as '$', '=', and ':' may now cause an
error in NCDF_VARDEF. Therefore, we now pre-screen
tracer names with function NCDF_VALID_NAME.
EXAMPLE:
BPCH2NC, 'myfile.bpch', 'myfile.%DATE%.%TIME%.nc'
; Will write the contents of "myfile.bpch" to one
; or more netCDF files "myfile.YYYYMMDD.HHMMSS.nc"
MODIFICATION HISTORY:
bmy, 22 May 2002: GAMAP VERSION 1.50
bmy, 22 Oct 2002: GAMAP VERSION 1.52
- bug fix: now do not write vertical layer
dim info to netCDF file for 2-D grids
bmy, 22 May 2003: GAMAP VERSION 1.53
- Bug fix: LONGNAME cannot be a null string
bmy, 22 Sep 2003: - Now declare all variables first, then save
data into them. This is much more efficient!
- Remove reference to NCDF_SET routine
- Call PTR_FREE to free the pointer memory
- Bug fix: now sort TAU0 values for call to UNIQ
- added /VERBOSE keyword
bmy, 09 Oct 2003: - for IDL 6.0+, use '__' to separate category
name from the tracer name
bmy, 21 Oct 2003: - Now uses function NCDF_Valid_Name to screen
out and replace invalid characters for netCDF
variable names
bmy, 06 Nov 2003: GAMAP VERSION 2.01
- added extra global attributes to facilitate
reading netCDF files created by BPCH2NC
into GAMAP
bmy, 29 Mar 2004: GAMAP VERSION 2.02
- Now saves ETA centers for hybrid grid
instead of sigma centers
bmy, 17 Mar 2005: GAMAP VERSION 2.03
- Bug fix: now prints ETAC properly
when the /VERBOSE keyword is set
bmy, 03 Oct 2006: GAMAP VERSION 2.05
- Bug fix: now do not call PTR_FREE
to free the pointer memory
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
- Now count GCAP among the GEOS family
for the purpose of converting the TAU
date to a YYYY/MM/DD date.
bmy, 21 Jan 2008: GAMAP VERSION 2.12
- Now save sigma edges & centers or
eta edges & centers to the file.
- Extra error trap, if there is only one
level in the file then set IS_3D=0.
- Now error check for duplicate variable names
bmy, 28 Nov 2008: GAMAP VERSION 2.15
- Now save out hybrid-grid Ap and Bp
parameters so that users can accurately
compute the pressure at level edges and
centers.
- Updated comments
bmy, 07 Mar 2012: GAMAP VERSION 2.16
- Correct typos in /VERBOSE output
(See /n/home09/ryantosca/IDL/gamap2/file_io/bpch2nc.pro)
NAME:
CTM_READ_COARDS
PURPOSE:
Reads data blocks from a COARDS-compliant netCDF file (such
as created by routine BPCH2COARDS) into GAMAP. CTM_READ_COARDS
is is an internal routine which is called by CTM_OPEN_FILE.
NOTE: COARDS is a formatting standard for netCDF files which
is widely used in both the atmospheric & climate communities.
COARDS-compliant netCDF files can be read by GAMAP, GrADS and
other plotting packages.
See http://ferret.wrc.noaa.gov/noaa_coop/coop_cdf_profile.html
for more information about the COARDS standard.
CATEGORY:
GAMAP Internals, Scientific Data Formats
CALLING SEQUENCE:
CTM_READ_COARDS, [, Keywords ]
INPUTS:
ILUN -> GAMAP file unit which will denote the netCDF file.
FILENAME -> Name of the netCDF grid file to be read.
FILEINFO -> Array of FILEINFO structures which will be
returned to CTM_OPEN_FILE. CTM_OPEN_FILE will
append FILEINFO to the GAMAP global common block.
DATAINFO -> Array of DATAINFO structures (each element
specifies a GAMAP data block) which will be returned
to CTM_OPEN_FILE. CTM_OPEN_FILE will append FILEINFO
to the GAMAP global common block.
KEYWORD PARAMETERS:
_EXTRA=e -> Picks up extra keywords
OUTPUTS:
None
SUBROUTINES:
Internal Subroutines:
====================================================
CRC_Get_DimInfo CRC_Get_IndexVars
CRC_Read_Global_Atts CRC_Get_Tracer
CRC_Get_Data CRC_Save_Data
External Subroutines Required:
====================================================
CTM_GRID (function) CTM_TYPE (function)
CTM_MAKE_DATAINFO (function) STRRIGHT (function)
STRREPL (function)
REQUIREMENTS:
Requires routines from both GAMAP and TOOLS packages.
NOTES:
(1) Assumes that data blocks have the following dimensions:
(a) longitude, latitude, time
(b) longitude, latitude, levels, time
(2) Assumes that times are given in GMT.
(3) If information about each tracer in the COARDS-compliant
netCDF file is stored in the GAMAP "tracerinfo.dat" file,
then CTM_READ_COARDS will be able to read the file without
having to ask the user to supply a GAMAP category and
tracer name.
EXAMPLE:
ILUN = 21
FILENAME = 'coards.20010101.nc'
CTM_READ_COARDS, ILUN, FILENAME, FILEINFO, DATAINFO
; Reads data from the COARDS-compliant netCDF file
; coards.20010101.nc and stores it the FILEINFO and
; DATAINFO arrays of structures. If you are calling
; CTM_READ_COARDS from CTM_OPEN_FILE, then CTM_OPEN_FILE
; will append FILEINFO and DATAINFO to the GAMAP global
; common block.
MODIFICATION HISTORY:
bmy, 21 Mar 2005: GAMAP VERSION 2.03
bmy, 21 Jul 2005: GAMAP VERSION 2.04
- bug fix in CRC_SAVE_DATA
bmy, 06 Mar 2006: GAMAP VERSION 2.05
- minor bug fix in CRC_READ_GLOBAL_ATTS
- bug fix in CRC_SAVE_DATA: add a fake 4th
dim to DATA array if needed
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
bmy, 16 Sep 2008: GAMAP VERSION 2.13
- Convert some global attributes to number
types in case attributes were initially
saved to the netCDF file as strings
phs, 15 Sep 2009: - Added check on reading tracerinfo.dat
- Convert some tracer names created by
BPCH2COARDS to their original value.
bmy, 14 Dec 2011: GAMAP VERSION 2.16
- Updated to read the GAMAP category value
from a netCDF variable attribute (if present)
bmy, 19 Dec 2011: - Generalized to handle several different
vertical levels
bmy, 21 Dec 2011: - Now will interpret netCDF attributes
"begin_date" and "begin_time" in the same
way as "start_date" and "start_time"
bmy, 22 Dec 2011: - Now compute FIRST (nested datablock offsets)
properly for nested grids. For now assume
that the data will always start on the first
vertical level.
- Bug fix: test for Latrev gt 0 to avoid
INADVERTENTLY reversing latitudes
bmy, 03 Jan 2012: - Skip over Ap and Bp index arrays
- Now use better error checks for
the time and vertical level dimensions
for each tracer in the netCDF file.
bmy, 05 Jan 2012: - Now interpret DELTA_TIME attribute
correctly when specified in hhmmss format.
bmy, 10 Jan 2012: - Fix to interpret data blocks with multiple
vertical dimensions in the same file
bmy, 13 Jan 2012: - When the time dimension in the netCDF
file is 1 (esp. when time is an UNLIMITED
variable, we need to add a fake dimension
of 1 back onto the data block to avoid
crashes. This is a quirk in how the IDL
NCDF_VARGET function works.
bmy, 23 Jan 2017: GAMAP VERSION 2.19
- Add modifications to read netCDF restart files
(See /n/home09/ryantosca/IDL/gamap2/internals/ctm_read_coards.pro)
NAME:
CTM_READ_EOSGR
PURPOSE:
Reads data blocks from a HDF-EOS Grid file into GAMAP.
(This is an internal routine which is called by CTM_OPEN_FILE.)
CATEGORY:
GAMAP Internals, Scientific Data Formats
CALLING SEQUENCE:
CTM_READ_EOSGR, ILUN, FILENAME, FILEINFO, DATAINFO, [, Keywords ]
INPUTS:
ILUN -> GAMAP file unit which will denote the HDF-EOS file.
FILENAME -> Name of the HDF-EOS grid file to be read.
FILEINFO -> Array of FILEINFO structures which will be
returned to CTM_OPEN_FILE. CTM_OPEN_FILE will
append FILEINFO to the GAMAP global common block.
DATAINFO -> Array of DATAINFO structures (each element
specifies a GAMAP data block) which will be returned
to CTM_OPEN_FILE. CTM_OPEN_FILE will append FILEINFO
to the GAMAP global common block.
KEYWORD PARAMETERS:
_EXTRA=e -> Picks up any extra keywords
OUTPUTS:
None
SUBROUTINES:
Internal Subroutines:
=====================================================
CRE_Get_DimInfo CRE_Get_TracerInfo CRE_Save_Data
External Subroutines Required:
=====================================================
CTM_GRID (function) CTM_TYPE (function)
CTM_MAKE_DATAINFO (function) STRRIGHT (function)
REQUIREMENTS:
Requires routines from both GAMAP and TOOLS packages.
NOTES:
(1) Currently is set up to read HDF-EOS files containing
GMAO met data files. You must have all possible met
field names listed in your "tracerinfo.dat" file or
else you will get an "Invalid Selection" error.
EXAMPLE:
ILUN = 21
FILENAME = 'a_llk_03.tsyn2d_mis_x.t20030801'
CTM_READ_EOSGR, ILUN, FILENAME, FILEINFO, DATAINFO
; Reads data from HDF-EOS file a_llk_03.tsyn2d_mis_x.t20030801
; and stores it the FILEINFO and DATAINFO arrays of
; structures. If calling CTM_READ_GMI from CTM_OPEN_FILE,
; then CTM_OPEN_FILE will append FILEINFO and DATAINFO
; to the GAMAP common block.
MODIFICATION HISTORY:
bmy, 12 Nov 2003: GAMAP VERSION 2.01
- initial version
bmy, 19 Feb 2004: GAMAP VERSION 2.01a
- added c402_rp_02 to the assim list
- bug fix: use DEFAULT keyword for SELECT_MODEL
bmy, 09 Mar 2004: GAMAP VERSION 2.02
- now test for "GEOS3", "GEOS4" strings in
the file name to determine model type
- now undefine variables after use
- now make sure that data block begins at the
date line and has longitude values in the
range [-180,180] degrees.
- always ensure that L=1 is the surface level
bmy, 25 Aug 2004: GAMAP VERSION 2.03
- Added c402_cer to the assim list
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
- Remove reference to unused SORT_STRU
bmy, 05 Dec 2007: GAMAP VERSION 2.12
- Modified for DAS.llk.asm files
bmy, 01 Oct 2008: GAMAP VERSION 2.13
- Increase DI, DJ in CRE_GET_DIMINFO for
the 0.5 x 0.667 grid
- Also now test for GEOS-5 in the filename
in CRE_GET_DIMINFO
- Bug fix in CRE_SAVE_DATA: If there is
only one data time in the HDF-EOS file,
then add an extra dimension to THISDATA
so that the CASE statement will be
interpreted properly.
(See /n/home09/ryantosca/IDL/gamap2/internals/ctm_read_eosgr.pro)
NAME:
CTM_READ_NCDF
PURPOSE:
Reads data blocks from a netCDF file created by routine
BPCH2NC or BPCH2GMI into GAMAP. (This is an internal routine
which is called by CTM_OPEN_FILE.)
CATEGORY:
GAMAP Internals, Scientific Data Formats
CALLING SEQUENCE:
CTM_READ_NCDF, [, Keywords ]
INPUTS:
ILUN -> GAMAP file unit which will denote the netCDF file.
FILENAME -> Name of the netCDF grid file to be read.
FILEINFO -> Array of FILEINFO structures which will be
returned to CTM_OPEN_FILE. CTM_OPEN_FILE will
append FILEINFO to the GAMAP global common block.
DATAINFO -> Array of DATAINFO structures (each element
specifies a GAMAP data block) which will be returned
to CTM_OPEN_FILE. CTM_OPEN_FILE will append FILEINFO
to the GAMAP global common block.
KEYWORD PARAMETERS:
_EXTRA=e -> Picks up extra keywords
OUTPUTS:
None
SUBROUTINES:
Internal Subroutines:
====================================================
CRN_Get_DimInfo CRN_Get_Time CRN_Get_Tracer
CRN_Read_Global_Atts CRN_Get_Data CRN_Save_Data
External Subroutines Required:
====================================================
CTM_GRID (function) CTM_TYPE (function)
CTM_MAKE_DATAINFO (function) STRRIGHT (function)
REQUIREMENTS:
Requires routines from both GAMAP and TOOLS packages.
NOTES:
(1) Currently assumes that the netCDF file was written
by GAMAP routine BPCH2NC.
EXAMPLE:
ILUN = 21
FILENAME = 'geos.20010101.nc'
CTM_READ_NCDF, ILUN, FILENAME, FILEINFO, DATAINFO
; Reads data from the netCDF file geos.20010101.nc
; and stores it the FILEINFO and DATAINFO arrays of
; structures. If calling CTM_READ_GMI from CTM_OPEN_FILE,
; then CTM_OPEN_FILE will append FILEINFO and DATAINFO
; to the GAMAP common block.
MODIFICATION HISTORY:
bmy, 05 Nov 2003: GAMAP VERSION 2.01
- initial version
bmy, 26 Mar 2004: GAMAP VERSION 2.02
- bug fix: now correctly separates "__"
in netCDF tracer names with STRPOS
bmy, 28 Feb 2005: GAMAP VERSION 2.03
- bug fix: now also exclude ETAC from
being passed to CRN_GET_TRACER
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
- Modified to read files from BPCH2GMI
bmy, 19 Dec 2007: GAMAP VERSION 2.12
- Now also skip ETAE, SIGE arrays
- Also don't add any vertical info to
the GRIDINFO structure if NLAYERS=1
bmy, 24 Jan 2011: GAMAP VERSION 2.15
- Now skip over Ap and Bp index fields
in the netCDF file
bmy, 14 Dec 2011: GAMAP VERSION 2.16
- Now also check to see if the GAMAP
category string is passed via the netCDF
"gamap_category" attribute
(See /n/home09/ryantosca/IDL/gamap2/internals/ctm_read_ncdf.pro)
NAME:
CTM_WRITENC
PURPOSE:
Save GAMAP datainfo records to disk (in netCDF format)
CATEGORY:
GAMAP Utilities, Scientific Data Formats
CALLING SEQUENCE:
CTM_WRITENC, DATAINFO, FILENAME=FILENAME
INPUTS:
DATAINFO -> a datainfo record or an array of datainfo records
KEYWORD PARAMETERS:
FILENAME -> Filename of output file. Should end in '.bpch'.
SCALE -> An optional scaling factor. This factor will be applied
to _all_ data record upon saving. The globally stored records
are not affected.
NEWUNIT -> With this keyword you can change the unit _name_ for
the saved data. This will _not_ perform a unit conversion!
For a true unit conversion you must also use the SCALE
keyword. NEWUNIT will be applied to _all_ records!
OUTPUTS:
A binary punch file with the selected data records will be
created.
SUBROUTINES:
External Subroutines Required:
====================================
CTM_GET_DATA
OPEN_FILE
DATATYPE
REQUIREMENTS:
Must have a version of IDL w/ the netCDF library installed.
NOTES:
This routine forces reading of all selected data records via
ctm_get_data. This may take a while for huge ASCII punch files.
EXAMPLE:
gamap [,options] ; call gamap to read records from a file
@gamap_cmn ; get global datainfo structure
d = *pglobalDataInfo
ind = where(d.tracer eq 2) ; select all Ox records
ctm_writebpch,d[ind],filename='oxconc.bpch'
MODIFICATION HISTORY:
mgs, 20 May 1999: GAMAP VERSION 1.47
- adapted from "ctm_writebpch.pro"
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
(See /n/home09/ryantosca/IDL/gamap2/gamap_util/ctm_writenc.pro)
NAME:
EOS_GETGR
PURPOSE:
Convenience routine to read variables from an HDF-EOS
grid data structure.
CATEGORY:
File & I/O, Scientific Data Formats
CALLING SEQUENCE:
DATA = EOS_GETGR( FID, NAME [, Keywords, _EXTRA=e ] )
INPUTS:
FID -> HDF-EOS File ID, as returned by routine EOS_GD_START.
NAME -> Name of the HDF-EOS grid dataset variable that
you want to extract from the HDF-EOS file.
KEYWORD PARAMETERS:
GRIDNAME -> Name of the HDF-EOS grid under which the data
is stored in the file. You can use the IDL HDF_BROWSER
routine to easily find the grid name.
_EXTRA=e -> Passes extra keywords to routine EOS_SW_READFIELD.
OUTPUTS:
DATA -> Array containing extracted data from the HDF-EOS file.
SUBROUTINES:
None
REQUIREMENTS:
Need to use a version of IDL w/ HDF-EOS routines installed.
NOTES:
None
EXAMPLE:
; Make sure HDF is supported on this platform
IF ( EOS_EXISTS() eq 0 ) then MESSAGE, 'HDF not supported!'
; Open the HDF file and get the file ID # (FID)
FID = EOS_GD_OPEN( 'gridfile.hdf', /READ )
IF ( FID lt 0 ) THEN MESSAGE, 'Error opening file!'
; Read a variable from a grid file
DATA = EOS_GETGR( fId, 'Latitude', GRIDNAME='GRID1' )
; Close the file
STATUS = EOS_GD_CLOSE( FID )
IF ( STATUS lt 0 ) THEN MESSAGE, 'Error closing file!'
MODIFICATION HISTORY:
bmy, 18 Sep 2002: TOOLS VERSION 1.51
bmy, 19 Dec 2002: TOOLS VERSION 1.52
- fixed typos
bmy, 04 Jun 2003: TOOLS VERSION 1.53
- fixed more typos
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
(See /n/home09/ryantosca/IDL/gamap2/file_io/eos_getgr.pro)
NAME:
EOS_GETSW
PURPOSE:
Convenience routine to read variables from an HDF-EOS
satellite swath data structure.
CATEGORY:
File & I/O, Scientific Data Formats
CALLING SEQUENCE:
DATA = EOS_GETSW( FID, NAME [, Keywords, _EXTRA=e ] )
INPUTS:
FID -> HDF File ID, as returned by routine EOS_SW_START.
NAME -> Name of the satellite swath dataset variable that
you want to extract from the HDF-EOS file.
KEYWORD PARAMETERS:
SWATHNAME -> Name of the HDF-EOS swath under which the data
is stored in the file. You can use the IDL HDF_BROWSER
routine to easily find the swath name.
_EXTRA=e -> Passes extra keywords to routine EOS_SW_READFIELD.
OUTPUTS:
DATA -> Array containing extracted data from the HDF-EOS file.
SUBROUTINES:
None
REQUIREMENTS:
Need to use a version of IDL w/ HDF-EOS routines installed.
NOTES:
None
EXAMPLE:
; Make sure HDF is supported on this platform
IF ( EOS_EXISTS() eq 0 ) then MESSAGE, 'HDF not supported!'
; Open the HDF file and get the file ID # (FID)
FID = EOS_SW_OPEN( 'swathfile.hdf', /READ )
IF ( FID lt 0 ) THEN MESSAGE, 'Error opening file!'
; Read a variable from a swath file
DATA = EOS_GETSW( fId, 'Latitude', SWATHNAME='swath1' )
; Close the file
STATUS = EOS_SW_CLOSE( FID )
IF ( STATUS lt 0 ) THEN MESSAGE, 'Error closing file!'
MODIFICATION HISTORY:
bmy, 18 Sep 2002: TOOLS VERSION 1.51
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
(See /n/home09/ryantosca/IDL/gamap2/file_io/eos_getsw.pro)
NAME:
HDF_GETSD
PURPOSE:
Convenience routine to read scientific dataset variables
from Hierarchical Data Format (HDF) files
CATEGORY:
File & I/O, Scientific Data Formats
CALLING SEQUENCE:
DATA = HDF_GETSD( FID, NAME [, _EXTRA=e ] )
INPUTS:
FID -> HDF File ID, as returned by routine HDF_SD_START.
NAME -> Name of the scientific dataset variable that
you want to extract from the file.
KEYWORD PARAMETERS:
_EXTRA=e -> Passes extra keywords to routine HDF_SD_GETDATA.
OUTPUTS:
DATA -> Array containing extracted data from the HDF file.
SUBROUTINES:
None
REQUIREMENTS:
Need to use a version of IDL w/ HDF routines installed.
NOTES:
Taken from MOP02Viewer by Yottana Khunatorn (bmy, 7/17/01)
EXAMPLE:
; Make sure HDF is supported on this platform
IF ( HDF_EXISTS() eq 0 ) then MESSAGE, 'HDF not supported!'
; Open the HDF file and get the file ID # (FID)
FID = HDF_SD_START( 'fvdas_flk_01.ana.eta.20011027.hdf', /Read )
IF ( FID lt 0 ) then MESSAGE, 'Error opening file!'
; Read the UWND field from disk
DATA = HDF_GETSD( fId, 'UWND' )
; Close the file
HDF_SD_END, FID
MODIFICATION HISTORY:
bmy, 05 Nov 2001: VERSION 1.00
bmy, 23 Apr 2002: TOOLS VERSION 1.50
- updated documentation
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
(See /n/home09/ryantosca/IDL/gamap2/file_io/hdf_getsd.pro)
NAME:
HDF_GETSDATTR
PURPOSE:
Convenience routine to read attributes (global or variable-
associated) from Hierarchical Data Format (HDF) files.
CATEGORY:
File & I/O, Scientific Data Formats
CALLING SEQUENCE:
DATA = HDF_GETSDATTR( ID, NAME [ , Keywords ] )
INPUTS:
ID -> HDF File ID as returned by routine HDF_SD_START,
or scientific dataset ID, as returned by routine
HDF_SD_SELECT.
NAME -> Name of the attribute to be read from the HDF file.
KEYWORD PARAMETERS:
COUNT -> Returns the total number of values in the
specified attribute to the calling program.
HDF_TYPE -> Returns the HDF type of the attribute to the
calling program. HDF types are returned as a scalar
string. Possible returned values are DFNT_NONE,
DFNT_CHAR, DFNT_FLOAT32, DFNT_FLOAT64, DFNT_INT8,
DFNT_INT16, DFNT_INT32, DFNT_UINT8, DFNT_UINT16, and
DFNT_UINT32.
TYPE -> Returns the IDL type pf the attribute to the calling
program. The type of the attribute is returned as a
scalar string. Possible returned values are BYTE, INT,
LONG, FLOAT, DOUBLE, STRING, or UNKNOWN.
OUTPUTS:
DATA -> Array containing attribute data from the HDF file.
SUBROUTINES:
IDL HDF routines used:
==========================
HDF_SD_AttrInfo
HDF_SD_AttrFind (function)
REQUIREMENTS:
Need to use a version of IDL w/ HDF routines installed.
NOTES:
None
EXAMPLE:
; Make sure HDF is supported on this platform
IF ( NCDF_EXISTS() eq 0 ) then MESSAGE, 'HDF not supported!'
; Open the HDF file and get the file ID # (FID)
FID = HDF_SD_START( 'fvdas_flk_01.ana.eta.20011027.hdf', /READ )
IF ( FID lt 0 ) then MESSAGE, 'Error opening file!'
; Read the Ak, Bk, and PTOP attributes from the HDF file
; These are GLOBAL attributes associated w/ the file
AK = HDF_GETSDATTR( FID, 'ak' )
BK = HDF_GETSDATTR( FID, 'bk' )
PTOP = HDF_GETSDATTR( FID, 'ptop' )
; Close the HDF file
HDF_SD_END, FID
MODIFICATION HISTORY:
bmy, 30 Apr 2002: TOOLS VERSION 1.50
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
(See /n/home09/ryantosca/IDL/gamap2/file_io/hdf_getsdattr.pro)
NAME:
HDF_GETVD
PURPOSE:
Convenience routine to read VDATA variables
from Hierarchical Data Format (HDF) files
CATEGORY:
File & I/O, Scientific Data Formats
CALLING SEQUENCE:
VDATA = HDF_GETVD( FID, NAME [, _EXTRA=e ] )
INPUTS:
FID -> HDF File ID, as returned by routine HDF_OPEN.
NAME -> Name of the VDATA variable that you
want to extract from the file.
KEYWORD PARAMETERS:
_EXTRA=e -> Passes extra keywords to routine HDF_VD_READ.
OUTPUTS:
VDATA -> Array containing extracted data from the HDF file.
SUBROUTINES:
None
REQUIREMENTS:
Need to use a version of IDL w/ HDF routines installed.
NOTES:
Taken from MOP02Viewer by Yottana Khunatorn (bmy, 7/17/01)
EXAMPLE:
FID = HDF_OPEN( 'fvdas_flk_01.ana.eta.20011027.hdf', /Read )
IF ( FID lt 0 ) then Message, 'Error opening file!'
PTOP = HDF_GETVD( fId, 'PTOP' )
HDF_CLOSE, FID
; Opens an HDF-format file and gets the file ID. Then
; call HDF_GETSD to return the PTOP variable from the
; file. Then close the file and quit.
MODIFICATION HISTORY:
bmy, 05 Nov 2001: VERSION 1.00
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
(See /n/home09/ryantosca/IDL/gamap2/file_io/hdf_getvd.pro)
NAME:
HDF_SETSD
PURPOSE:
Convenience routine to write data into the Hierarchical Data
Format Scientific Dataset (HDF-SD) structure
CATEGORY:
File & I/O, Scientific Data Formats
CALLING SEQUENCE:
HDF_SETSD, FID, DATA, NAME [, Keywords ]
INPUTS:
FID -> HDF File ID, as returned by routine HDF_SD_START.
DATA -> Data (array or scalar) to be written to HDF-SD format.
NAME -> Name under which the data array will be saved
to the HDF file.
KEYWORD PARAMETERS:
LONGNAME -> Longer descriptive name for the data. This will
be saved as the "long_name" attribute. Default is ''.
RANGE -> A 2-element vector containing the [min,max] of
the data array. If not passed, RANGE will be computed
(but only for numeric data types). RANGE will be saved
to the HDF file as the "valid_range" attribute.
_EXTRA=e -> picks up extra keywords for HDF_SD_SETINFO, such
as FILL, UNIT, COORDSYS, etc...
OUTPUTS:
None
SUBROUTINES:
Uses the following IDL HDF routines:
===========================================
HDF_SD_Create (function) HDF_SD_SetInfo
HDF_SD_AddData HDF_SD_EndAccess
DATATYPE (function)
REQUIREMENTS:
Need to use a version of IDL w/ HDF routines installed.
NOTES:
(1) Since HDF supports the STRING type, we do not have to
treat BYTE data like ASCII characters (cf ncdf_set.pro)
EXAMPLE:
; Find out if HDF is supported on this platform
IF ( HDF_EXISTS() eq 0 ) then MESSAGE, 'HDF not supported!'
; Open the HDF file
FID = HDF_SD_START( 'myhdf.hdf', /Create )
IF ( FID lt 0 ) then Message, 'Error opening file!'
; Write data to disk
HDF_SETSD, FID, DATA, 'NOx', $
LONGNAME='Nitrogen Oxide',$
UNIT='v/v', $
FILL=0.0,
; Close HDF File
HDF_SD_END, FID
; Writes NOx data to an HDF file.
MODIFICATION HISTORY:
bmy, 17 Apr 2002: TOOLS VERSION 1.50
bmy, 11 Sep 2002: TOOLS VERSION 1.51
- Now call routine DATATYPE to determine
the type of the data so that we can
write all data types to the HDF file.
- Don't add the RANGE attribute to
the HDF file for a string type value.
- Updated comments
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
(See /n/home09/ryantosca/IDL/gamap2/file_io/hdf_setsd.pro)
NAME:
MAKE_MULTI_NC
PURPOSE:
Convert bpch files (matching a mask) to netCDF format.
The routine can work recursively : files in subdirectories
are also searched for.
This is a convenient generic wrapper for GAMAP routine
Bpch2coards.pro, to work on several files and directories at
once.
When working recursively, the directory tree in the input
directory is recreated as needed in the output directory.
CATEGORY:
File & I/O, BPCH Format, Scientific Data Formats
CALLING SEQUENCE:
MAKE_MULTI_NC
INPUTS:
none
KEYWORD PARAMETERS:
MASK -> string, default is '*', that is all files
INPUT_PARENT_DIR -> top directory of files to convert, if
not specified, a dialog window pops up
OUTPUT_PARENT_DIR -> top directory destination for netCDF
files, if not specified, a dialog window pops up.
RECURSION -> to search subdirectories. Default is 0
(OFF). Set to 1 to turn it on.
TOKEN -> set if you want to replace 'bpch' with 'nc' in all
part of the full name. Default is to have extension
".nc" added to file name only.
OUTPUTS:
None
REQUIREMENTS:
References routines from the GAMAP package.
NOTES:
To work recursively on the directories, I use FILE_SEARCH
with 2 arguments. This is not possible with MFINDFILE, and
works only with IDL 5.5 and above.
EXAMPLES:
indir = '/as/data/geos/GEOS_1x1/EDGAR_200607/'
outdir = '/home/phs/data/EDGAR_200607/nc/'
mask = '*1x1'
make_multi_nc, input_parent_dir=indir, out=outdir, mask=mask, /r
MODIFICATION HISTORY:
Thu Oct 29 16:40:28 2009, Philippe Le Sager
now pass _extra keyword to BPCH2COARDS (eg /NAMER
needed for nested data)
Thu Feb 19 11:12:16 2009, Philippe Le Sager
renamed MAKE_MULTI_NC. Added recursion keyword
(default is OFF). Now **AUTOMATICALLY** creates missing
subdirectory in output directory tree, when in
recursive mode.
Tue Jan 27 10:40:49 2009, Philippe Le Sager
v1, based on make_c_nc.pro to work on several
directories at once.
bmy, 30 Nov 2010: GAMAP VERSION 2.15
- Updated comments and category in header
bmy, 01 Feb 2012: GAMAP VERSION 2.16
- Skip processing ASCII text files
mps, 04 Mar 2015: - Bug fix: Now assign InFileName before checking
file type
(See /n/home09/ryantosca/IDL/gamap2/file_io/make_multi_nc.pro)
NAME:
MAKE_NC_RECURSIVE
PURPOSE:
Converts files matching a mask to netCDF format.
The routine works recursively. This is a convenience
wrapper for GAMAP routine bpch2coards.pro, to work on
several directories at once.
You just need to (1) create the same directory tree in the
output location, then (2) run the routine once.
CATEGORY:
File & I/O, BPCH Format, Scientific Data Formats
CALLING SEQUENCE:
MAKE_NC_RECURSIVE
INPUTS:
none
KEYWORD PARAMETERS:
MASK -> string, default is '*', that is all files
INPUT_PARENT_DIR -> top directory of files to convert, if
not specified, a dialog window pops up
NEW_PARENT_DIR -> top directory destination for netCDF
files, if not specified, a dialog window pops up
TOKEN -> set if you want to replace 'bpch' with 'nc' in all
part of the full name
OUTPUTS:
None
REQUIREMENTS:
References routines from the GAMAP package.
NOTES:
To work recursively on the directories, I use FILE_SEARCH
with 2 arguments. This is not possible with MFINDFILE, and
works only with IDL 5.5 and above.
EXAMPLE:
; after creating a subdirectories tree in outdir similar to the
; one in indir:
indir = '/as/data/geos/GEOS_1x1/EDGAR_200607/'
outdir = '/home/phs/data/EDGAR_200607/nc/'
mask = '*1x1'
make_nc_recursive, input_parent_dir=indir, new=outdir, mask=mask
MODIFICATION HISTORY:
Tue Jan 27 10:40:49 2009, Philippe Le Sager
v1, based on make_c_nc.pro to work on several
directories at once.
bmy, 30 Nov 2010: GAMAP VERSION 2.15
- Updated comments and categories in header
(See /n/home09/ryantosca/IDL/gamap2/file_io/make_nc_recursive.pro)
NAME:
NCDF_GET
PURPOSE:
Convenience routine to read data into netCDF format.
CATEGORY:
File & I/O, Scientific Data Formats
CALLING SEQUENCE:
DATA = NCDF_GET( FID, NAME [, Keywords ] )
INPUTS:
FID -> netCDF File ID, as returned by routine NCDF_OPEN.
NAME -> Name under which the data array will be saved
to the netCDF file.
KEYWORD PARAMETERS:
VARINFO -> Returns a structure containing information
about the variable read in from disk. The structure
has the following form:
{ NAME : "", $
DATATYPE : "", $
NDIMS : 0L, $
NATTS : 0L, $
DIM : LONARR(NDIMS) }
LONGNAME -> Returns the value saved under the "long_name"
attribute in the netCDF file.
UNIT -> Returns the value of the "unit" attribute
saved in the netCDF file.
RANGE -> Returns the value of the "valid_range"
saved in the netCDF file
_EXTRA=e -> Picks up extra keywords got NCDF_VarGet.
OUTPUTS:
DATA -> Array containing extracted data from the netCDF file.
SUBROUTINES:
Uses the following IDL netCDF routines:
========================================
NCDF_VarId (function) NCDF_VarGet
NCDF_VarInfo (function) NCDF_AttGet
NCDF_AttName (function)
REQUIREMENTS:
Need to use a version of IDL w/ netCDF routines installed.
NOTES:
(1) Only looks for the "long_name", "unit", and "valid_range"
attributes. The user can extend this as he/she desires.
For a more general program, see ~/IDL/tools/ncdf_read.pro
by Martin Schultz.
EXAMPLE:
; Define array to write to file
ARRAY = DIST( 100, 50 )
; Find out if netCDF is supported on this platform
IF ( NCDF_EXISTS() eq 0 ) then MESSAGE, 'netCDF not supported!'
; Open netCDF file and get the file ID # (FID)
FID = NCDF_OPEN( 'myfile.nc' )
IF ( FID lt 0 ) then Message, 'Error opening file!'
; Read data from the netCDF file
; Return data attributes in the VARINFO array
; Also returns the text from the UNIT string
DATA = NCDF_GET( FID, 'BIOBSRCE::NOx', $
VARINFO=VARINFO, UNIT=UNIT )
; Close the netCDF file
NCDF_CLOSE, FID
MODIFICATION HISTORY:
bmy, 22 May 2002: TOOLS VERSION 1.50
bmy, 21 Oct 2003: TOOLS VERSION 1.53
- If we can't find a netCDF variable name,
then try again using a "sanitized" name
w/ all bad characters stripped out
bmy, 28 Jun 2007: TOOLS VERSION 2.06
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
- Added /VERBOSE keyword to reduce
warning messages
(See /n/home09/ryantosca/IDL/gamap2/file_io/ncdf_get.pro)
NAME:
NCDF_READ
PURPOSE:
Open a netCDF file and read data from it. The data is
returned as a structure whose tag names are the names of
the variables with blanks etc. replaced. If no variables
are specified with the VARIABLES keyword, only dimensional
information is returned. You can load all variables using
the ALL keyword. Other keyword options include OFFSET, COUNT, STRIDE,
NO_DIMENSIONS, NO_STRUCT, DIMNAMES, VARNAMES, VARDIMS, ATTRIBUTES.
Thus, this program includes ncdump functionality.
If no filename is given, a file selection dialog is
opened with the default mask '*.nc'. The name of the selected
file is returned in the TRUENAME keyword. A file selection
dialog also appears when the file cannot be found (see
OPEN_FILE.PRO). This can be turned off with the NO_DIALOG
keyword. The VERBOSE keyword provides information while
analyzing and reading the file.
AUTHOR:
Dr. Martin Schultz
Max-Planck-Institut fuer Meteorologie
Bundesstr. 55, D-20146 Hamburg
email: martin.schultz@dkrz.de
CATEGORY:
File & I/O, Scientific Data Formats
CALLING SEQUENCE:
NCDF_READ, result, filename=, truename=,
variables=, all=, varnames=,
vardimid=, vardims=, attributes=,
count=, offset=, stride=,
dimnames=, dims=, no_dimensions=,
no_struct=, no_dialog=, verbose=, title=
ARGUMENTS:
RESULT(out) -> a structure containing all the variable data
from the netCDF file. If only one variable is specified
and the NO_STRUCT keyword set, then RESULT will be an
array instead of a structure. Use the ALL keyword
to load all variables at once. Note, that the COUNT, OFFSET,
and STRIDE keywords can affect the size of RESULT.
RESULT is set to -1L if an error occurs before the structure
has been built. You can use CHKSTRU.PRO to test for this.
KEYWORD PARAMETERS:
FILENAME(in) -> the name of the netCDF file to be opened.
NCDF_READ uses OPEN_FILE to check the validity of
the file first. You can specify a search mask
instead of a filename in which case a file selection
dialog is displayed (unless you set the NO_DIALOG
keyword). The TRUENAME keyword contains the name
of the selected file or an empty string if the
file selection was canceled.
TRUENAME(out) -> the (fully qualified) name of the file selected
with the file selection dialog or an unaltered copy
of FILENAME if FILENAME is a valid filename.
VARIABLES(in) -> a string array containing the names of variables
for which data shall be read. Default is to read
only the dimensional information from the file.
(Currently) no warning is issued if a variable is not in the file.
ALL(in) -> set this keyword to load all variables stored in the
netCDF file. Generally, you cannot usethis keyword together with
COUNT, OFFSET, and STRIDE.
VARNAMES(out) -> a string array containing the names of all variables
as stored in the file. Note, that the tag names of e.g. the
result structure are filtered with the Valid_TagName function.
VARDIMID(out) -> a structure with integer arrays containing the
dimension ID's for all variables. See also VARDIMS which returns
the dimensions themselves.
VARDIMS(out) -> a structure with integer arrays containing the
dimensions for all variables in the netCDF file. These are not
kept in sync with potential COUNT, OFFSET, and STRIDE values,
but reflect the data sizes as stored in the file.
ATTRIBUTES(out) -> a structure holding the variable and global
attributes stored in the netCDF file (global attributes
are stored in tag name GLOBAL).
COUNT(in) -> an integer array containing the number of values to
be read for each dimension of the variables. Mapping of the
COUNT dimensions to the variable dimensions is achieved via
the first entry in the VARIABLES list and the COUNT parameter
will be applied to all variables that have that dimension.
Example: The first variable has dimensions LON, LAT, LEV,
the second variable has dimensions LON, LAT, and the third
variable has LAT, LEV. A COUNT of [40,20,10] would lead to
result dimensions of [40,20,10], [40,20], and [20,10].
OFFSET(in) -> an integer array containing the offsets for each
dimension of the variables to be read. Dimension mapping
is the same as for COUNT.
STRIDE(in) -> an integer array containing the stride for each
dimension of the variables to be read. Dimension mapping
is the same as for COUNT.
DIMNAMES(out) -> a string array containing the names of the
dimensions stored in the netCDF file.
DIMS(out) -> a long array containing the dimension values. Purely
for convenience. Use VARDIMS to retrieve the dimensions of
the variables.
TITLE(in) -> A title for the file selection dialog if an
incomplete or incorrect filename is specified. This
keyword is ignored if the no_dialog keyword is set.
NO_DIMENSIONS(in) -> set this keyword if you do not want to store
the dimensional variables from the file in the result structure.
DIMNAMES and DIMS will still be retrieved.
NO_STRUCT(in) -> if only one variable is selected with the
VARIABLE keyword, you can set this keyword to return only
the data for this variable as an array. This keyword implies
the functionality of NO_DIMENSIONS.
NO_DIALOG(in) -> set this keyword if you do not want interactive
behaviour when a file mask is given instead of a filename or
when the specified file does not exist.
VERBOSE(in) -> set this keyword to get detailed information while
reading the netCDF file.
SUBROUTINES:
Valid_TagName : replaces invalid characters in variable names so that
a structure can be built.
ncdf_attributes : retrieves global and variable attributes from netcdf
file and stores them as structure.
ncdf_dimensions : retrieves size and name for all dimensions in netcdf file.
ncdf_varnames : retrieves names and dimension information for all variables
in the netCDF file.
ncdf_mapdims : map dimension indices for COUNT, OFFSET, and STRIDE with
dimensions of first selected variable.
ncdf_TestDimensions : compute the COUNT, OFFSET, and STRIDE vectors that
must be applied for retrieving the data of one variable.
REQUIREMENTS:
uses OPEN_FILE and STRREPL.
NOTES:
Correct handling of dimensional information requires that the variables
storing the dimension values have the same name as the dimensions
themselves - a common feature in most netCDF standards.
I am still working on a netcdf file object which will be even
more powerful. At some point ncdf_read will only be a
procedure interface to this objec!
EXAMPLE:
ncdf_read,result,/All
; plot ozone vs. temperature
plot,result.temperature,result.ozone
MODIFICATION HISTORY:
mgs, 18 Sep 1999: VERSION 1.00
mgs, 29 Feb 2000: - added variables keyword
- added CATCH error handler
mgs, 21 Mar 2000: - bug fix for tag names
mgs, 09 May 2000: VERSION 2.00
- now only reads dimensions as default
- added ALL keyword to compensate
- returns dimnames and attributes
(makes ncdf_detail obsolete)
- added COUNT, OFFSET and STRIDE keywords
- added NO_DIMENSIONS and NO_DIALOG
keywords and more
mgs, 22 Aug 2000: - added title keyword
bmy, 22 May 2002: GAMAP VERSION 1.50
- Now replace ":", "=", "#" with "_" in
structure tag names, so IDL won't choke
bmy, 21 Oct 2003: GAMAP VERSION 1.53
- also replace "(" and ")" in tag names
- replace "$" with "S" (close to dollar sign)
- replace "*" with "A" (for Asterisk)
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
(See /n/home09/ryantosca/IDL/gamap2/file_io/ncdf_read.pro)
NAME:
NCDF_SET
PURPOSE:
Convenience routine to write data into netCDF format.
CATEGORY:
File & I/O, Scientific Data Formats
CALLING SEQUENCE:
NCDF_SET, FID, DATA, NAME, DIMS [, Keywords ]
INPUTS:
FID -> netCDF File ID, as returned by routine NCDF_CREATE.
DATA -> Data (array or scalar) to be written to netCDF file.
NAME -> Name under which the data array will be saved
to the netCDF file.
KEYWORD PARAMETERS:
LONGNAME -> Longer descriptive name for the data.
This will be saved as the "long_name" attribute.
RANGE -> A 2-element vector containing the [min,max] of
the data array. If not passed, RANGE will be computed
(but only if DATA is a numeric type). RANGE is saved
to the netCDF file as the "valid_range" attribute.
UNIT -> String containing the units of the data.
This will be saved as the "unit" attribute.
_EXTRA=e -> Picks up extra keywords
OUTPUTS:
None
SUBROUTINES:
Uses the following routines:
=====================================================
NCDF_Control NCDF_VarDef (function)
NCDF_AttPut DATATYPE (function from TOOLS)
NCDF_VarPut NCDF_VALID_NAME (function from TOOLS)
REQUIREMENTS:
Need to use a version of IDL w/ netCDF routines installed.
NOTES:
(1) For now, treat BYTE data like CHAR data. This is
most likely since netCDF does not support STRING types,
strings have to be stored as arrays of bytes.
EXAMPLE:
; Define array to write to file
ARRAY = DIST( 100, 50 )
; Find out if netCDF is supported on this platform
IF ( NCDF_EXISTS() eq 0 ) then MESSAGE, 'netCDF not supported!'
; Open netCDF file and get the file ID # (FID)
FID = NCDF_CREATE( 'myfile.nc', /CLOBBER )
IF ( FID lt 0 ) then Message, 'Error opening file!'
; Set dimensions for netCDF file
S = SIZE( ARRAY, /DIM )
DIM1 = NCDF_DIMDEF( FID, 'Length', S[0] )
DIM2 = NCDF_DIMDEF( FID, 'Width', S[1] )
; Go into netCDF DATA mode
NCDF_CONTROL, FID, /ENDEF
; Call NCDF_SET to write the array to the file
NCDF_SET, FID, ARRAY, 'My Data', [ DIM1, DIM2 ], $
LONGNAME='Data array created by me!', $
UNIT='unitless'
; Close the netCDF file
NCDF_CLOSE, FID
MODIFICATION HISTORY:
bmy, 19 Apr 2002: TOOLS VERSION 1.50
bmy, 10 Sep 2002: TOOLS VERSION 1.51
- Now call routine DATATYPE to determine
the type of the data so that we can
write to the netCDF file appropriately
- Don't add the RANGE attribute to
the netCDF file for a string type value.
- Updated comments
bmy, 21 Oct 2003: TOOLS VERSION 1.53
- now "sanitize" the netCDF variable name
w/ routine NCDF_VALID_NAME. The new netCDF
library in IDL 6.0+ chokes on bad characters.
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
(See /n/home09/ryantosca/IDL/gamap2/file_io/ncdf_set.pro)
NAME:
NCDF_VALID_NAME
PURPOSE:
Strips invalid characters from a string which is to be
used as a netCDF variable name. Based on original code
by Martin Schultz.
CATEGORY:
File & I/O, Scientific Data Formats
CALLING SEQUENCE:
RESULT = NCDF_VALID_NAME( ARG )
INPUTS:
ARG -> netCDF variable name string to be examined.
KEYWORD PARAMETERS:
None
OUTPUTS:
RESULT -> New netCDF name string with "bad" characters
replaced by "good" characters.
SUBROUTINES:
External Subroutines Used:
==========================
STRREPL (function)
REQUIREMENTS:
None
NOTES:
In IDL 6.0+, the netCDF library has been updated. Some
characters which used to be allowed in netCDF variable names
are no longer allowed. Therefore, use this function to
replace "bad" characters with "good" characters when
reading or writing to/from netCDF files.
EXAMPLE:
RESULT = NCDF_VALID_NAME( 'IJ-AVG-$::CO' )
PRINT, RESULT
; Prints "IJ-AVG-S__CO"
MODIFICATION HISTORY:
bmy, 21 Oct 2003: VERSION 1.00
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
(See /n/home09/ryantosca/IDL/gamap2/file_io/ncdf_valid_name.pro)
NAME:
READ_H5DATASET
PURPOSE:
Convenience routine to read dataset variables
from Hierarchical Data Format version 5 (HDF5) files.
Also works for HDF-EOS files!
CATEGORY:
File & I/O, Scientific Data Formats
CALLING SEQUENCE:
DATAFIELD = READ_H5DATASET( FID, DATASET_NAME )
INPUTS:
FID -> HDF5 File ID, as returned by routine H5F_OPEN
DATASET_NAME -> Name of the scientific dataset variable that
you want to extract from the file.
KEYWORD PARAMETERS:
None
OUTPUTS:
DATAFIELD -> Array containing extracted data from the HDF file.
SUBROUTINES:
None
REQUIREMENTS:
Need to use a version of IDL w/ HDF5 library installed.
NOTES:
From Trevor Beck (trevor.beck@noaa.gov) for GOME-2.
EXAMPLE:
; Specify the file name
FILE = 'GOME_xxx_1B_M02_20070105012056Z_20070105030556Z_R_O_20080613081807Z.337p4_356p1.brs.hcho.he5'
; Make sure the file is a HDF5 file
IF ( H5F_IS_HDF5( FILE ) eq 0 ) then MESSAGE, 'Not an HDF-5 file!'
; Open the HDF file and get the file ID # (FID)
FID = H5F_OPEN( FILE )
IF ( FID lt 0 ) then MESSAGE, 'Error opening file!'
; Read the AMF field from disk
; NOTE: the swath name is "Column"
AMF = READ_H5DATASET( FID, "/Column/Amf" )
; Close the file
H5_CLOSE, FID
MODIFICATION HISTORY:
bmy, 28 May 2009: VERSION 1.00
(See /n/home09/ryantosca/IDL/gamap2/file_io/read_h5dataset.pro)