List routines by category:
Atmospheric Sciences |
Benchmarking |
Color |
Date/Time |
Doc |
File & I/O |
BPCH Format |
Scientific Data Formats |
GAMAP Examples |
GAMAP Internals |
GAMAP Utilities |
GAMAP Data Manipulation |
GAMAP Models &
Grids |
GAMAP Plotting |
General |
Graphics |
Math & Units |
Plotting |
Regridding |
Strings |
Structures |
Time Series
List routines by alphabetical order:
A |
B |
C |
D |
E |
F |
G |
H |
I |
J |
K |
L |
M |
N |
O |
P |
Q |
R |
S |
T |
U |
V |
W |
X |
Y |
Z
NAME:
BPCH2ASCII
PURPOSE:
Translates data from GAMAP-readable binary punch file v. 2.0
format to a simple ASCII file format
CATEGORY:
File & I/O, BPCH Format
CALLING SEQUENCE:
BPCH2ASCII, INFILE, OUTFILE [ , Keywords ]
INPUTS:
INFILE -> Name of the binary punch file to read. If
INFILE is not passed, the user will be prompted
to supply a file name via a dialog box.
OUTFILE -> Name of the ASCII file to be written. It is
recommended to insert the tokens %DATE% and %TIME%
into OUTFILE, since BPCH2ASCII will write a separate
netCDF file for each time index in the *.bpch file.
The tokens %DATE% and %TIME% will be overwritten
with the current values of YYYYMMDD and HHMMSS.
Default is "bpch2nc_output.%DATE%.%TIME%.ascii".
KEYWORD PARAMETERS:
DIAGN -> A diagnostic category name (e.g. "IJ-AVG-$") or
array of names which will restrict the data block
selection. If DIAGN is omitted, then all data blocks
within INFILE will be saved in ASCII format to OUTFILE.
/VERBOSE -> If set, then BPCH2ASCII will also echo the
header lines for each data block to the screen.
FORMAT -> String containing the numeric format for
for the data values. Default is '(7(e13.6,1x))'
OUTPUTS:
None
SUBROUTINES:
External Subroutines Required:
=============================================
CTM_GET_DATA TAU2YYMMDD (function)
GETMODELANDGRIDINFO REPLACE_TOKEN (function)
UNDEFINE
REQUIREMENTS:
References routines from both GAMAP and TOOLS packages.
NOTES:
(1) BPCH2ASCII assumes that all data blocks in the *.bpch file
file adhere to same grid. This will be true for output
files from the GEOS-CHEM model.
(2) BPCH2ASCII will write a separate ASCII file corresponding
to each time index in the *.bpch file. This prevents
file sizes from getting large, especially if there is
a lot of diagnostic output in the *.bpch file.
(3) BPCH2NC will replace the %DATE% token with the
current YYYYMMDD value, and will replace the %TIME%
token with the current HHMMSS value. Therefore, it
is recommended to insert these tokens into the string
passed via OUTFILE. The tokens %DATE% and %TIME% tokens
may be either in uppercase or lowercase.
(4) The format of the ASCII file is:
Data block #1 header line
Data block #1 values (format specified by FORMAT keyword)
Data block #2 header line
Data block #2 values (format specified by FORMAT keyword)
...
The header line will contain the units and size of
each data block.
(5) The data is written to the ASCII file in column-major
order (i.e. the same way as in FORTRAN), so you can read
the data into FORTRAN w/ the following code:
READ( IUNIT, '(a)' ) HEADER
READ( IUNIT, '(1p,7(e13.6,1x))' )
& ((DATA(I,J), I=1,IMX), J=1,JMX)
where IMX and JMX are the dimensions of the data block.
EXAMPLE:
BPCH2ASCII, 'myfile.bpch', 'myfile.%DATE%.%TIME%.ascii'
; Read data from binary punch file 'myfile.bpch'
; and writes it to ASCII file 'myfile.bpch.ascii'.
MODIFICATION HISTORY:
bmy, 22 May 2002: GAMAP VERSION 1.50
bmy, 28 May 2002: GAMAP VERSION 1.51
- Added FORMAT keyword
bmy, 03 Jun 2004: GAMAP VERSION 2.02
- now pass extra keywords to CTM_GET_DATA
via _EXTRA=e keyword
bmy, 03 Dec 2004: GAMAP VERSION 2.03
- add CATEGORY keyword (passed to CTM_GET_DATA)
in order to refine data block search
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
(See /n/home09/ryantosca/IDL/gamap2/file_io/bpch2ascii.pro)
NAME:
BPCH2COARDS
PURPOSE:
Reads data from a binary punch file and saves it in a
COARDS-compliant netCDF (network Common Data Format) file.
NOTE: COARDS is a formatting standard for netCDF files which
is widely used in both the atmospheric & climate communities.
COARDS-compliant netCDF files can be read by GAMAP, GrADS and
other plotting packages.
See http://ferret.wrc.noaa.gov/noaa_coop/coop_cdf_profile.html
for more information about the COARDS standard.
CATEGORY:
File & I/O, BPCH Format, Scientific Data Formats
CALLING SEQUENCE:
BPCH2COARDS, INFILE, OUTFILE [, Keywords ]
INPUTS:
INFILE -> Name of the binary punch file to read. If INFILE
is not passed, the user will be prompted to supply a
file name via a dialog box.
OUTFILE -> Name of the netCDF file to be written. It is
recommended to insert the tokens %DATE% (or %date%)
into OUTFILE, since BPCH2COARDS will write a separate
netCDF file for each unique YYYYMMDD value contained
within the *.bpch file. If OUTFILE is not specified,
BPCH2COARDS will use the default file name
"coards.%DATE%.nc".
KEYWORD PARAMETERS:
DIAGN -> Array of diagnostic categories from the bpch file
to save to netCDF format. If omitted, BPCH2COARDS will
save all diagnostic categories.
/VERBOSE -> If set, will print the name of each tracer
as it is being written to the netCDF file. Useful
for debugging purposes.
/NC4 -> Write a netCDF4 file instead of netCDF3. Default is
NetCDF3. NetCDF4 support requires IDL 8.0 or later.
COMPRESS -> Integer 0-9 specifies amount of compression in
netCDF4 files. Default is 2, with very little benefit
for higher compression.
/PCOORD -> Use mean pressure as the vertical coordinate rather
sigma or eta
/ALTCOORD -> Use mean altitude as the vertical coordinate
rather than sigma or eta
/TROPONLY -> Write only tropospheric layers
/ONEFILE -> Write all data to one netCDF output file.
Default is one file per calendar day.
_EXTRA=e -> Picks up additional keywords for NCDF_SET
OUTPUTS:
None
SUBROUTINES:
External Subroutines Required:
============================================
CTM_GET_DATA TAU2YYMMDD (function)
UNDEFINE REPLACE_TOKEN (function)
STRREPL (function) GETMODELANDGRIDINFO
REQUIREMENTS:
(1) References routines from GAMAP and TOOLS packages.
(2) You must use a version of IDL containing the NCDF routines.
NOTES:
(1) BPCH2COARDS assumes that each data block in the *.bpch
file is either 2-D (lon-lat) or 3-D (lon-lat-alt).
(2) BPCH2COARDS assumes that the number type of each data
block in the *.bpch file is REAL*4 (a.k.a. FLOAT).
(3) BPCH2COARDS assumes that all data blocks in the *.bpch
file adhere to same horizontal grid. This will always
be true for output files from the GEOS-CHEM model.
(4) BPCH2COARDS will write a separate COARDS-compliant netCDF
file corresponding to each unique YYYYMMDD date. This
prevents the files from becoming too large to be read
with IDL.
(5) BPCH2COARDS will replace the %DATE% (or %date%) token with
the current YYYYMMDD value. Therefore, it is recommended
to insert this token into the string passed via OUTFILE.
(6) BPCH2COARDS will write arrays containing the latitudes,
longitudes to the netCDF file. For 3-D data blocks,
the eta or sigma centers will also be written to the
file. Time will be written as TAU values (i.e. hours
since 00:00 GMT on 01-Jan-1985.
(7) The netCDF library has apparently been updated in
IDL 6.0+. The result is that variable names containing
characters such as '$', '=', and ':' may now cause an
error in NCDF_VARDEF. Therefore, we now pre-screen
tracer names with function NCDF_VALID_NAME.
EXAMPLE:
BPCH2COARDS, 'myfile.bpch', 'myfile.%DATE%.nc'
; Will write the contents of "myfile.bpch" to one
; or more COARDS-compliant netCDF files adhering
; to the filename convention "myfile.YYYYMMDD.nc"
MODIFICATION HISTORY:
rjp & bmy, 17 Mar 2005: GAMAP VERSION 2.03
- Based on bpch2nc.pro
bmy, 21 Jul 2005: GAMAP VERSION 2.04
- Bug fix:
bmy, 13 Jul 2006: GAMAP VERSION 2.05
- Remove call to PTR_FREE
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
- Now count GCAP among the GEOS family
for the purpose of converting the TAU
date to a YYYY/MM/DD date.
phs, 29 Oct 2009: GAMAP VERSION 2.14
- Can process files with 3D data on both
centers and edges of the grid boxes.
bmy, 19 Dec 2011: GAMAP VERSION 2.16
- Now handles multiple vertical dimensions
in the bpch file properly.
- Bug fix: now write vertical levels edges to
the file.
bmy, 27 Sep 2012: - Bug fix: Now handle data blocks that straddle
the date line.
bmy, 05 Nov 2013: GAMAP VERSION 2.17
- Change attributes for better COARDS compliance
bmy, 12 Feb 2014: GAMAP VERSION 2.18
- Add more modifications for 4-D data blocks
from bpch files created w/ GC_COMBINE_ND49
bmy, 03 Mar 2015: - Now define dims in order: time, lev, lon, lat,
which is more COARDS compliant.
cdholmes, 29 Mar 2017 - Add support for NetCDF4 and
compression
- Add support for pressure and altitude
as vertical coordinate
- ONEFILE puts all data into a single
output file
- TROPONLY limits the output to
troposphere layers
- Singleton "altXXX" dimensions are avoided
(See /n/home09/ryantosca/IDL/gamap2/file_io/bpch2coards.pro)
NAME:
BPCH2GMI
PURPOSE:
Reads data from a binary punch file and saves it in
netCDF (network Common Data Format) format. The data
will be shifted so that the first longitude is 0 degrees
(i.e. the prime meridian) in order to conform with the
GMI (Global Model Initiative) model grid definition.
CATEGORY:
File & I/O, BPCH Format, Scientific Data Formats
CALLING SEQUENCE:
BPCH2GMI, INFILE, OUTFILE [, Keywords ]
INPUTS:
INFILE -> Name of the binary punch file to read. If
INFILE is not passed, the user will be prompted
to supply a file name via a dialog box.
OUTFILE -> Name of the netCDF file to be written. It is
recommended to insert the tokens %DATE% and %TIME%
into OUTFILE, since BPCH2NC will write a separate
netCDF file for each time index in the *.bpch file.
The tokens %DATE% and %TIME% will be overwritten
with the current values of YYYYMMDD and HHMMSS.
Default is "bpch2nc_output.%DATE%.%TIME%.nc".
KEYWORD PARAMETERS:
DIAGN -> A diagnostic category name (e.g. "IJ-AVG-$") or
array of names which will restrict the data block
selection. If DIAGN is omitted, then all data blocks
within INFILE will be saved in netCDF format to OUTFILE.
/VERBOSE -> If set, will print the names of each tracer
as it is being written to the netCDF file.
_EXTRA=e -> Picks up additional keywords for netCDF routines
OUTPUTS:
SUBROUTINES:
Internal Subroutines:
============================================
B2G_Valid_VarName (function)
B2G_SetNcDim (function)
B2G_GetNcDim (function)
External Subroutines Required:
============================================
CTM_GET_DATA TAU2YYMMDD (function)
UNDEFINE REPLACE_TOKEN (function)
STRREPL (function) GETMODELANDGRIDINFO
REQUIREMENTS:
(1) References routines from GAMAP and TOOLS packages.
(2) You must use a version of IDL containing the NCDF routines.
NOTES:
(1) BPCH2GMI assumes that each data block in the *.bpch file
is either 2-D (lon-lat) or 3-D (lon-lat-alt).
(2) BPCH2GMI assumes that the number type of each data block
in the *.bpch file is REAL*4 (a.k.a. FLOAT).
(3) BPCH2GMI assumes that all data blocks in the *.bpch file
file adhere to same horizontal grid. This will always
be true for output files from the GEOS-CHEM model.
(4) BPCH2GMI will write a separate NC file corresponding
to each time index in the *.bpch file. This prevents
file sizes from getting large, especially if there is
a lot of diagnostic output in the *.bpch file.
(5) BPCH2GMI will replace the %DATE% token with the
current YYYYMMDD value, and will replace the %TIME%
token with the current HHMMSS value. Therefore, it
is recommended to insert these tokens into the string
passed via OUTFILE. The tokens %DATE% and %TIME% tokens
may also be passed in lowercase (e.g, %date%, %time% ).
(6) BPCH2GMI will write arrays containing the latitudes,
longitudes to the netCDF file. For 3-D data blocks,
the sigma centers will also be written to the file.
Date and time are stored as global attributes.
(7) The netCDF library has apparently been updated in
IDL 6.0+. The result is that variable names containing
characters such as '$', '=', and ':' may now cause an
error in NCDF_VARDEF. Therefore, we now pre-screen
tracer names with function NCDF_VALID_NAME.
EXAMPLE:
BPCH2GMI, 'myfile.bpch', 'myfile.%DATE%.%TIME%.nc'
; Will write the contents of "myfile.bpch" to one
; or more netCDF files "myfile.YYYYMMDD.HHMMSS.nc"
MODIFICATION HISTORY:
bmy & phs, 20 Aug 2007: GAMAP VERSION 2.10
- Based on BPCH2NC
bmy, 19 Dec 2007: GAMAP VERSION 2.12
- Now save sigma edges & centers or
eta edges & centers to the file.
- Extra error trap, if there is only one
level in the file then set IS_3D=0.
bmy, 20 Dec 2011: GAMAP VERSION 2.16
- Changed default filename to "bpch2
(See /n/home09/ryantosca/IDL/gamap2/file_io/bpch2gmi.pro)
NAME:
BPCH2HDF
PURPOSE:
Reads data from a binary punch file and saves it in HDF-SD
(Hierarchical Data Format, Scientific Dataset) format.
CATEGORY:
File & I/O, BPCH Format, Scientific Data Formats
CALLING SEQUENCE:
BPCH2HDF, INFILE, OUTFILE [, Keywords ]
INPUTS:
INFILE -> Name of the binary punch file to read. If
INFILE is not passed, the user will be prompted
to supply a file name via a dialog box.
OUTFILE -> Name of the HDF file to be written. It is
recommended to insert the tokens %DATE% and %TIME%
into OUTFILE, since BPCH2HDF will write a separate
HDF file for each time index in the *.bpch file.
The tokens %DATE% and %TIME% will be overwritten
with the current values of YYYYMMDD and HHMMSS.
Default is "bpch2hdf_output.%DATE%.%TIME%.hdf".
KEYWORD PARAMETERS:
DIAGN -> A diagnostic category name (e.g. "IJ-AVG-$") or
array of names which will restrict the data block
selection. If DIAGN is omitted, then all data blocks
within INFILE will be saved in HDF format to OUTFILE.
_EXTRA=e -> Picks up additional keywords for HDF_SETSD
OUTPUTS:
SUBROUTINES:
External Subroutines Required:
=========================================
CTM_GET_DATA TAU2YYMMDD (function)
UNDEFINE REPLACE_TOKEN (function)
HDF_SETSD GETMODELANDGRIDINFO
REQUIREMENTS:
(1) References routines from GAMAP and TOOLS packages.
(2) You must use a version of IDL containing the HDF-SD routines.
NOTES:
(1) BPCH2HDF assumes that each data block in the *.bpch file
is either 2-D (lon-lat) or 3-D (lon-lat-alt).
(2) BPCH2HDF assumes that all data blocks in the *.bpch file
file adhere to same horizontal grid. This will be true
for output files from the GEOS-CHEM model.
(3) BPCH2HDF will write a separate HDF file corresponding
to each time index in the *.bpch file. This prevents
file sizes from getting large, especially if there is
a lot of diagnostic output in the *.bpch file.
(4) BPCH2HDF will replace the %DATE% token with the
current YYYYMMDD value, and will replace the %TIME%
token with the current HHMMSS value. Therefore, it
is recommended to insert these tokens into the string
passed via OUTFILE. These tokens may be in either
uppercase or lowercase.
(4) BPCH2HDF will also write arrays containing the latitudes,
longitudes, sigma coordinates (for 3-D data blocks only!)
to the HDF file.
(5) BPCH2HDF will write arrays containing the latitudes,
longitudes to the netCDF file. For 3-D data blocks,
the sigma centers and sigma edges will also be written
to the file.
EXAMPLE:
BPCH2HDF, 'myfile.bpch', 'myfile.%DATE%.%TIME%.hdf'
; Will write the contents of "myfile.bpch" to
; one or more HDF files "myfile.YYYYMMDD.HHMMSS.hdf"
MODIFICATION HISTORY:
bmy, 22 May 2002: GAMAP VERSION 1.50
bmy, 22 Oct 2002: GAMAP VERSION 1.52
- bug fix: now do not write vertical layer
dim info to HDF file for 2-D grids
bmy, 22 May 2003: GAMAP VERSION 1.53
- Make sure LONGNAME is not a null string
bmy, 18 Sep 2003: - Call PTR_FREE to free the pointer memory
bmy, 03 Jun 2004: GAMAP VERSION 2.02
- now pass extra keywords to CTM_GET_DATA
via _EXTRA=e keyword
bmy, 03 Sep 2004: GAMAP VERSION 2.03
- now defines ETAC and ETAE variables
for hybrid grids
bmy, 03 Dec 2004: GAMAP VERSION 2.03
- add DIAGN keyword (passed to CTM_GET_DATA)
in order to refine data block search
bmy, 19 May 2006: GAMAP VERSION 2.05
- Now do not free the pointer memory
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
- Now count GCAP among the GEOS family
for the purpose of converting the TAU
date to a YYYY/MM/DD date.
bmy, 18 Feb 2009: GAMAP VERSION 2.13
- Bug fix: should be N_ELEMENTS( DiagN )
instead of N_ELEMENTS( Category ) in the
IF statement for CTM_GET_DATA.
(See /n/home09/ryantosca/IDL/gamap2/file_io/bpch2hdf.pro)
NAME:
BPCH2NC
PURPOSE:
Reads data from a binary punch file and saves it in
netCDF (network Common Data Format) format.
CATEGORY:
File & I/O, BPCH Format, Scientific Data Formats
CALLING SEQUENCE:
BPCH2NC, INFILE, OUTFILE [, Keywords ]
INPUTS:
INFILE -> Name of the binary punch file to read. If
INFILE is not passed, the user will be prompted
to supply a file name via a dialog box.
OUTFILE -> Name of the netCDF file to be written. It is
recommended to insert the tokens %DATE% and %TIME%
into OUTFILE, since BPCH2NC will write a separate
netCDF file for each time index in the *.bpch file.
The tokens %DATE% and %TIME% will be overwritten
with the current values of YYYYMMDD and HHMMSS.
Default is "bpch2nc_output.%DATE%.%TIME%.nc".
KEYWORD PARAMETERS:
DIAGN -> A diagnostic category name (e.g. "IJ-AVG-$") or
array of names which will restrict the data block
selection. If DIAGN is omitted, then all data blocks
within INFILE will be saved in netCDF format to OUTFILE.
/VERBOSE -> If set, will print the names of each tracer
as it is being written to the netCDF file.
_EXTRA=e -> Picks up additional keywords for NCDF_SET
OUTPUTS:
SUBROUTINES:
Internal Subroutines:
============================================
B2N_Valid_VarName (function)
B2N_SetNcDim (function)
B2N_GetNcDim (function)
External Subroutines Required:
============================================
CTM_GET_DATA TAU2YYMMDD (function)
UNDEFINE REPLACE_TOKEN (function)
STRREPL (function) GETMODELANDGRIDINFO
REQUIREMENTS:
(1) References routines from GAMAP and TOOLS packages.
(2) You must use a version of IDL containing the NCDF routines.
NOTES:
(1) BPCH2NC assumes that each data block in the *.bpch file
is either 2-D (lon-lat) or 3-D (lon-lat-alt).
(2) BPCH2NC assumes that the number type of each data block
in the *.bpch file is REAL*4 (a.k.a. FLOAT).
(3) BPCH2NC assumes that all data blocks in the *.bpch file
file adhere to same horizontal grid. This will always
be true for output files from the GEOS-CHEM model.
(4) BPCH2NC will write a separate NC file corresponding
to each time index in the *.bpch file. This prevents
file sizes from getting large, especially if there is
a lot of diagnostic output in the *.bpch file.
(5) BPCH2NC will replace the %DATE% token with the
current YYYYMMDD value, and will replace the %TIME%
token with the current HHMMSS value. Therefore, it
is recommended to insert these tokens into the string
passed via OUTFILE. The tokens %DATE% and %TIME% tokens
may also be passed in lowercase (e.g, %date%, %time% ).
(6) BPCH2NC will write arrays containing the latitudes,
longitudes to the netCDF file.
(7) For pure-sigma grids (e.g. GEOS-1, GEOS-STRAT, GEOS-3),
BPCH2NC will write the following additional variables to
the netCDF file:
(1) SIGE : Sigma coordinate at level edges
(2) SIGC : Sigma coordinate at level centers
The pressure at the bottom edge of level L is given by:
Pe(L) = Ptop + [ SIGE(L) * ( Psurface - Ptop ) ]
and the pressure at the vertical center of level L is:
Pc(L) = Ptop + [ SIGC(L) * ( Psurface - Ptop ) ]
%%%%% NOTE: This is mostly obsolete, as most met fields
%%%%% that are used to drive CTM's are now placed onto
%%%%% hybrid grids.
(8) For hybrid grids (e.g. GEOS-4, GEOS-5, MERRA), the following
dditional variables will be written to the netCDF file:
(1) ETAE : Eta coordinate on level edges
(2) ETAC : Eta coordinate on level centers
(3) Ap : Hybrid grid "A" parameter
(4) Bp : Hybrid grid "B" parameter
The pressure at the bottom edge of level L is given by:
Pe(L) = Ap(L) + ( Bp(L) * Psurface )
and the pressure at the vertical center of level L is:
Pc(L) = ( Pe(L) + Pe(L+1) ) * 0.5
The hybrid ETA coordinates (similar to sigma) at the
level edges and centers are, correspondingly:
ETAE(L) = ( Pe(L) - Ptop ) / ( Psurface - Ptop )
ETAC(L) = ( Pc(L) - Ptop ) / ( Psurface - Ptop )
%%%%% NOTE: The ETAe and ETAc values are only approximate
%%%%% and are computed with a surface pressure of 1013.25
%%%%% hPa. For your scientific analysis, you should compute
%%%%% the pressures at level edges from Ap, Bp, and a
%%%%% spatially-varying surface pressure field (e.g. saved
%%%%% out from GEOS-Chem or another model).
(7) The date and time of the data are stored in the netCDF
file as global attributes.
(8) The netCDF library has apparently been updated in
IDL 6.0+. The result is that variable names containing
characters such as '$', '=', and ':' may now cause an
error in NCDF_VARDEF. Therefore, we now pre-screen
tracer names with function NCDF_VALID_NAME.
EXAMPLE:
BPCH2NC, 'myfile.bpch', 'myfile.%DATE%.%TIME%.nc'
; Will write the contents of "myfile.bpch" to one
; or more netCDF files "myfile.YYYYMMDD.HHMMSS.nc"
MODIFICATION HISTORY:
bmy, 22 May 2002: GAMAP VERSION 1.50
bmy, 22 Oct 2002: GAMAP VERSION 1.52
- bug fix: now do not write vertical layer
dim info to netCDF file for 2-D grids
bmy, 22 May 2003: GAMAP VERSION 1.53
- Bug fix: LONGNAME cannot be a null string
bmy, 22 Sep 2003: - Now declare all variables first, then save
data into them. This is much more efficient!
- Remove reference to NCDF_SET routine
- Call PTR_FREE to free the pointer memory
- Bug fix: now sort TAU0 values for call to UNIQ
- added /VERBOSE keyword
bmy, 09 Oct 2003: - for IDL 6.0+, use '__' to separate category
name from the tracer name
bmy, 21 Oct 2003: - Now uses function NCDF_Valid_Name to screen
out and replace invalid characters for netCDF
variable names
bmy, 06 Nov 2003: GAMAP VERSION 2.01
- added extra global attributes to facilitate
reading netCDF files created by BPCH2NC
into GAMAP
bmy, 29 Mar 2004: GAMAP VERSION 2.02
- Now saves ETA centers for hybrid grid
instead of sigma centers
bmy, 17 Mar 2005: GAMAP VERSION 2.03
- Bug fix: now prints ETAC properly
when the /VERBOSE keyword is set
bmy, 03 Oct 2006: GAMAP VERSION 2.05
- Bug fix: now do not call PTR_FREE
to free the pointer memory
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
- Now count GCAP among the GEOS family
for the purpose of converting the TAU
date to a YYYY/MM/DD date.
bmy, 21 Jan 2008: GAMAP VERSION 2.12
- Now save sigma edges & centers or
eta edges & centers to the file.
- Extra error trap, if there is only one
level in the file then set IS_3D=0.
- Now error check for duplicate variable names
bmy, 28 Nov 2008: GAMAP VERSION 2.15
- Now save out hybrid-grid Ap and Bp
parameters so that users can accurately
compute the pressure at level edges and
centers.
- Updated comments
bmy, 07 Mar 2012: GAMAP VERSION 2.16
- Correct typos in /VERBOSE output
(See /n/home09/ryantosca/IDL/gamap2/file_io/bpch2nc.pro)
NAME:
BPCH2PCH
PURPOSE:
Translates data from GAMAP-readable binary punch
file v. 2.0 format to the ancient ASCII-punch
file standard.
CATEGORY:
File & I/O, BPCH Format
CALLING SEQUENCE:
BPCH2PCH, FILENAME [, OUTFILENAME [, Keywords ] ]
INPUTS:
FILENAME -> Name of the binary punch file from which
to read data. FILENAME may be a file mask, and may
contain wild card characters (e.g. ~/ctm.bpch.*). If
FILENAME is omitted or contains a wild card character,
the user will be prompted to pick a file via a dialog box.
OUTFILENAME (optional) -> Name of the output ASCII punch
file. Default is 'ASCIIfile.pch'
KEYWORD PARAMETERS:
DIAGN -> A diagnostic category name (e.g. "IJ-AVG-$") or
array of names which will restrict the data block
selection. If DIAGN is omitted, then all data blocks
within INFILE will be saved in ASCII punch format
to OUTFILE.
/EXTRA_SPACE -> If set, will put an extra space between
the numbers in the ASCII punch file. This might
be necessary when using MATLAB or S-PLUS to read
in the ASCII punch file.
OUTPUTS:
Writes data to ASCII punch file format
SUBROUTINES:
CTM_GET_DATA
REQUIREMENTS:
References routines from both GAMAP and TOOLS packages.
NOTES:
Some limitations:
(1) Works only for global lon-lat diagnostics.
(2) The top header line might be inaccurate (but nobody
really reads that anyway, so forget it for now...)
EXAMPLE:
BPCH2PCH, '~/bmy/ctm.bpch', '~/bmy/ctm.pch'
; Reads data from binary punch file '~/bmy/ctm.bpch'
; and writes it to ASCII punch file '~/bmy/ctm.pch'.
MODIFICATION HISTORY:
bmy, 08 Nov 1999: VERSION 1.00
bmy, 03 Jun 2004: GAMAP VERSION 2.02
- now pass extra keywords to CTM_GET_DATA
via _EXTRA=e keyword;
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
(See /n/home09/ryantosca/IDL/gamap2/file_io/bpch2pch.pro)
NAME:
BPCH_LINK
PURPOSE:
Copies data from several binary punch files into a single
binary punch file. Also can trim data down to nested-grid
resolution if necessary
CATEGORY:
File & I/O, BPCH Format
CALLING SEQUENCE:
BPCH_LINK, INFILES, OUTFILE [, Keywords ]
INPUTS:
INFILES -> A path name or file mask (with wildcards)
which indicates the names of the individual files
to be linked together in a single bpch file.
OUTFILE -> Name of the bpch file that will contain data
from the individual bpch files specified by INFILES.
KEYWORD PARAMETERS:
/CREATE_NESTED --> If set, then BPCH_LINK will trim data
to the nested grid resolution as specified by the
XRANGE and YRANGE keywords.
XRANGE -> A 2-element vector containing the minimum and
maximum box center longitudes which define the nested
model grid. Default is [-180,180].
YRANGE -> A 2-element vector containing the minimum and
maximum box center latitudes which define the nested
model grid. Default is [-90,90].
OUTPUTS:
None
SUBROUTINES:
External Subroutines Required:
==============================
OPEN_FILE UNDEFINE
CTM_DIAGINFO
REQUIREMENTS:
Requires routines from the GAMAP and TOOLS packages.
NOTES:
None
EXAMPLE:
BPCH_LINK, 'ctm.bpch.*', 'new.ctm.bpch'
; Consolidates data from the 'ctm.bpch.*' files
; into a single file named 'new.ctm.bpch'
MODIFICATION HISTORY:
bmy, 31 Jan 2003: VERSION 1.00
bmy, 09 Apr 2003: VERSION 1.01
- now can save to nested grid
bmy, 15 May 2003: VERSION 1.02
- now can pass a list of files via INFILES
bmy, 20 Nov 2003: GAMAP VERSION 2.01
- now gets the spacing between diagnostic
offsets from CTM_DIAGINFO
bmy, 28 May 2004: GAMAP VERSION 2.02
- Now use MFINDFILE to get INLIST regardless
of the # of elements of INFILES
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
bmy, 02 Apr 2008: GAMAP VERSION 2.12
- Now read/write bpch as big-endian
(See /n/home09/ryantosca/IDL/gamap2/file_io/bpch_link.pro)
NAME:
BPCH_MONTHLY_MEAN
PURPOSE:
Creates monthly means from GEOS-Chem output saved at less
than monthly intervals. Ideal for working with output of
high-resolution model output, especially if your queuing
system time limits do not permit a 1-month simulation to be
completed in a single run stage.
CATEGORY:
File & I/O, BPCH Format
CALLING SEQUENCE:
BPCH_MONTHLY_MEAN, FILES [, Keywords ]
INPUTS:
FILES -> A vector containing the pathnames of the files from
which you would like to create monthly mean output.
KEYWORD PARAMETERS:
OUTFILENAME -> Name of the file (bpch format) that will
contain the monthly mean output. The default is
"bpch_monthly_mean_output.bpch".
OUTPUTS:
None
SUBROUTINES:
External Subroutines Required
================================
REQUIREMENTS:
Requires other routines from the GAMAP package.
NOTES:
Assumes that each of the files passed via the FILES argument
contains an identical sequence of diagnostic data blocks.
This will normally be the case for GEOS-Chem simulations that
have to be separated into several run stages for the queue
system.
Error checking is minimal, we will fix this later. This
routine in intended for use with files that are created from
individual stages of a long GEOS-Chem simulation. As such,
we can usually assume that all files will have the same
sequence of data blocks, and that all data blocks will be on
the same grid.
EXAMPLE:
FILES = [ 'ctm.bpch.2011010100', 'ctm.bpch.2011011500' ]
BPCH_MONTHLY_MEAN, FILES, OUTFILENAME = 'monthly_mean.bpch'
; Creates monthly-mean output from two GEOS-Chem bpch
; files that contain 15-day averaged data.
MODIFICATION HISTORY:
bmy, 21 Dec 2011: GAMAP VERSION 2.16
- Initial version
(See /n/home09/ryantosca/IDL/gamap2/file_io/bpch_monthly_mean.pro)
NAME:
BPCH_SEP
PURPOSE:
Separates data from one binary punch file into another binary
punch file by time (TAU0), tracer, or location indices. Useful
for making smaller bpch files so that we don't run out of IDL
memory when reading/processing them.
CATEGORY:
File & I/O, BPCH Format
CALLING SEQUENCE:p
BPCH_SEP, INFILE, OUTFILE [, Keywords ]
INPUTS:
INFILE -> A path name or file mask (with wildcards)
which indicates the names of the individual files
to be linked together in a single bpch file.
OUTFILE -> Name of the bpch file that will contain data
from the individual bpch files specified by INFILES.
KEYWORD PARAMETERS:
DIAGN -> Array of diagnostic categories for which to
save out to OUTFILE Default is to save all diagnostic
categories to OUTFILE.
TAU0 -> Time index (hours from 1 Jan 1985) denoting the
data blocks to be saved from INFILE to OUTFILE. You
can use NYMD2TAU to compute this from a YYYYMMDD date.
TRACER -> Tracer number(s) for which to save to OUTFILE.
Default is to save all tracers.
II, JJ, LL -> Longitude, latitude, altitude index arrays used
to cut down the data block to less than global size. Use
IDL notation, meaning first element is 0.
GLOBALCOORDS -> When the data entries in the bpch file
contain only part of the global CTM grid (e.g. ND48,
ND49, ND50, ND51) this indicates
that II, JJ, LL indices should be
interpreted as global CTM indices, rather than relative
to the regional subset grid.
EXCLUDE -> Reverses the selection criteria, so that the
designated DIAGN, TAU0, and TRACER are EXCLUDED from the
output file. Everything else is included. In order to be
excluded, a datablock must match all specified keywords:
DIAGN, TAU0, and TRACER.
OUTPUTS:
None
SUBROUTINES:
External Subroutines Required:
===============================================
CTM_DIAGINFO LITTLE_ENDIAN (function)
OPEN_FILE UNDEFINE
REQUIREMENTS:
None
NOTES:
(1) Assumes that II, JJ, LL contain consecutive indices in
longitude, latitude, and altitude, respectively.
(2) Also assumes that II, JJ, LL are in IDL notation
(i.e. starting from zero). This is so that you can
pass the output from the WHERE command to BPCH_SEP.
EXAMPLES:
(1)
BPCH_SEP, 'ctm.bpch.big', 'ctm.bpch.small', tau0=140256D
; Pulls out data blocks for TAU0=140256 (1/1/2001) from
; "ctm.bpch.big" and saves them in "ctm.bpch.small"
(2)
INTYPE = CTM_TYPE( 'GEOS4', RES=4 )
INGRID = CTM_GRID( INTYPE )
INDX = WHERE( INGRID.XMID ge -60 AND INGRID.XMID le 60 )
INDY = WHERE( INGRID.YMID ge 0 AND INGRID.YMID le 60 )
BPCH_SEP, 'ctm.bpch.big', 'ctm.bpch.small', II=INDX, JJ=INDY
; Pulls out all data blocks for the geographical area
; from 60W - 60E longitude and 0-60N latitude.
MODIFICATION HISTORY:
bmy, 18 Sep 2003: GAMAP VERSION 1.53
bmy, 20 Nov 2003: GAMAP VERSION 2.01
- now gets the spacing between diagnostic
offsets from CTM_DIAGINFO
bmy, 07 Jul 2005: GAMAP VERSION 2.04
- minor bug fix; now can save out data
blocks for more than one matching TAU0
phs, 24 Oct 2006: GAMAP VERSION 2.05
- Added the II, JJ, LL keywords for
selecting a smaller geographical area.
These must be index arrays.
- Added the TRACERN keyword
- Added SWAP_ENDIAN=LITTLE_ENDIAN() in
the call to OPEN_FILE
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
bmy, 02 Apr 2008: GAMAP VERSION 2.12
- Cosmetic changes
bmy, 18 Jul 2016: GAMAP VERSION 2.19
- Now use the spacing obtained from
the diaginfo.dat file
cdh, 04 Apr 2017: -Add GlobalCoord keyword so that II,JJ,LL keywords
work with ND48, ND49, ND50 and ND51 output
-Prevent writing data when no matching
II,JJ,LL coordinates
(See /n/home09/ryantosca/IDL/gamap2/file_io/bpch_sep.pro)
NAME:
BPCH_TEST
PURPOSE:
Reads header and data block information from binary punch
(BPCH for short) files and prints the file pointer locations.
CATEGORY:
File & I/O, BPCH Format
CALLING SEQUENCE:
BPCH_TEST [, FILENAME, [ Keywords ] ]
INPUTS:
FILENAME (optional) -> Name of the binary punch file to read.
If omitted, a dialog box will prompt the user to make
a selection.
KEYWORD PARAMETERS:
/NOPAUSE -> If set, will not pause after displaying information
about each data block. Default is to pause to allow the
user to examine each data blocks header information.
OUTPUTS:
None
SUBROUTINES:
External Subroutines Required:
------------------------------
LITTLE_ENDIAN (function)
REQUIREMENTS:
References routines from the TOOLS package.
NOTES:
BPCH_TEST does not return any data values from the binary
punch file. It is meant to be used for debugging purposes.
EXAMPLES:
BPCH_TEST, 'my.bpch'
BPCH_TEST, FILENAME = 'my.bpch'
; will print info about each data block in 'my.bpch'
MODIFICATION HISTORY:
bmy, 10 Dec 1999: VERSION 1.00
bmy, 25 May 2000: GAMAP VERSION 1.45
- allow user to quit after viewing
each data block header
- add FILENAME keyword, so that the filename
can be passed as a parameter or a keyword
bmy, 21 Jul 2000: GAMAP VERSION 1.46
- now print locations of min, max data values
- changed FILETYPE to reflect new definitions
from CTM_OPEN_FILE
bmy, 24 Aug 2004: GAMAP VERSION 2.03
- Now recognizes bpch file containing
GEOS-CHEM station timeseries data
- Updated comments, cosmetic change
bmy, 05 Feb 2013: GAMAP VERSION 2.17
- Now recognize FTI "CTM bin 4D", which is
output from gc_combine_nd49.pro etc.
(See /n/home09/ryantosca/IDL/gamap2/file_io/bpch_test.pro)
NAME:
CONVERT_O3PL
PURPOSE:
Converts single-tracer Ox rate files from their native
binary format to binary punch format
CATEGORY:
File & I/O, BPCH Format
CALLING SEQUENCE:
CONVERT_O3PL [, Keywords ]
INPUTS:
None
KEYWORD PARAMETERS:
INMODELNAME -> A string containing the name of the model
grid on which the input data resides. Default is GEOS_STRAT.
INRESOLUTION -> Specifies the resolution of the model grid
on which the input data resides. RESOLUTION can be
either a 2 element vector with [ DI, DJ ] or
a scalar (DJxDI: 8=8x10, 4=4x5, 2=2x2.5, 1=1x1,
0.5=0.5x0.5). Default is 2.
OUTPUTS:
None
SUBROUTINES:
External Subroutines Required:
==============================
CTM_TYPE (function)
CTM_GRID (function)
CTM_MAKE_DATAINFO (function)
REQUIREMENTS:
None
NOTES:
Input file names are hardwired for now.
EXAMPLE:
CONVERT_O3PL, INMODELNAME ='GEOS1', $
INRESOLUTION = 4, $
; Regrids P(O3) and L(O3) data from
; GEOS-1 4 x 5 grid to GISS-II-PRIME 4 x 5 grid.
MODIFICATION HISTORY:
bmy, 16 Jul 2002: VERSION 1.00
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
bmy, 02 Apr 2008: GAMAP VERSION 2.12
- Now read input file as big-endian
(See /n/home09/ryantosca/IDL/gamap2/file_io/convert_o3pl.pro)
NAME:
CREATE_TAGOX_RESTART
PURPOSE:
Creates an initial tagged-Ox restart file w/ 13 tracers
(i.e. corresponding to Arlene Fiore's original runs)
CATEGORY:
File & I/O
CALLING SEQUENCE:
CREATE_TAGOX_RESTART
INPUTS:
None
KEYWORD PARAMETERS:
FILENAME -> Name of full-chemistry restart file containing Ox
data (stored under tracer #2) to be used in creating a
Tagged Ox restart file.
OUTFILENAME -> Name of the Tagged Ox restart file that will
be created. Default is "restart.Ox".
/ZERO_STRAT -> Set this
OUTPUTS:
None
SUBROUTINES:
External subroutines required:
==============================
CTM_GRID (function)
CTM_MAKE_DATAINFO (function)
CTM_WRITEBPCH
REQUIREMENTS:
None
NOTES:
Assumes Ox tracers have an offset of 40.
EXAMPLE:
CREATE_TAGOX_RESTART, FILENAME='gctm.trc.20010701', $
OUTFILENAME="gctm.trc.20010701.Ox'
; Reads Ox from a full chemistry restart file and
; creates a tagged-Ox restart file for initial spinup.
MODIFICATION HISTORY:
bmy, 18 Aug 2003: VERSION 1.01
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
(See /n/home09/ryantosca/IDL/gamap2/file_io/create_tagox_restart.pro)
NAME:
EOS_GETGR
PURPOSE:
Convenience routine to read variables from an HDF-EOS
grid data structure.
CATEGORY:
File & I/O, Scientific Data Formats
CALLING SEQUENCE:
DATA = EOS_GETGR( FID, NAME [, Keywords, _EXTRA=e ] )
INPUTS:
FID -> HDF-EOS File ID, as returned by routine EOS_GD_START.
NAME -> Name of the HDF-EOS grid dataset variable that
you want to extract from the HDF-EOS file.
KEYWORD PARAMETERS:
GRIDNAME -> Name of the HDF-EOS grid under which the data
is stored in the file. You can use the IDL HDF_BROWSER
routine to easily find the grid name.
_EXTRA=e -> Passes extra keywords to routine EOS_SW_READFIELD.
OUTPUTS:
DATA -> Array containing extracted data from the HDF-EOS file.
SUBROUTINES:
None
REQUIREMENTS:
Need to use a version of IDL w/ HDF-EOS routines installed.
NOTES:
None
EXAMPLE:
; Make sure HDF is supported on this platform
IF ( EOS_EXISTS() eq 0 ) then MESSAGE, 'HDF not supported!'
; Open the HDF file and get the file ID # (FID)
FID = EOS_GD_OPEN( 'gridfile.hdf', /READ )
IF ( FID lt 0 ) THEN MESSAGE, 'Error opening file!'
; Read a variable from a grid file
DATA = EOS_GETGR( fId, 'Latitude', GRIDNAME='GRID1' )
; Close the file
STATUS = EOS_GD_CLOSE( FID )
IF ( STATUS lt 0 ) THEN MESSAGE, 'Error closing file!'
MODIFICATION HISTORY:
bmy, 18 Sep 2002: TOOLS VERSION 1.51
bmy, 19 Dec 2002: TOOLS VERSION 1.52
- fixed typos
bmy, 04 Jun 2003: TOOLS VERSION 1.53
- fixed more typos
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
(See /n/home09/ryantosca/IDL/gamap2/file_io/eos_getgr.pro)
NAME:
EOS_GETSW
PURPOSE:
Convenience routine to read variables from an HDF-EOS
satellite swath data structure.
CATEGORY:
File & I/O, Scientific Data Formats
CALLING SEQUENCE:
DATA = EOS_GETSW( FID, NAME [, Keywords, _EXTRA=e ] )
INPUTS:
FID -> HDF File ID, as returned by routine EOS_SW_START.
NAME -> Name of the satellite swath dataset variable that
you want to extract from the HDF-EOS file.
KEYWORD PARAMETERS:
SWATHNAME -> Name of the HDF-EOS swath under which the data
is stored in the file. You can use the IDL HDF_BROWSER
routine to easily find the swath name.
_EXTRA=e -> Passes extra keywords to routine EOS_SW_READFIELD.
OUTPUTS:
DATA -> Array containing extracted data from the HDF-EOS file.
SUBROUTINES:
None
REQUIREMENTS:
Need to use a version of IDL w/ HDF-EOS routines installed.
NOTES:
None
EXAMPLE:
; Make sure HDF is supported on this platform
IF ( EOS_EXISTS() eq 0 ) then MESSAGE, 'HDF not supported!'
; Open the HDF file and get the file ID # (FID)
FID = EOS_SW_OPEN( 'swathfile.hdf', /READ )
IF ( FID lt 0 ) THEN MESSAGE, 'Error opening file!'
; Read a variable from a swath file
DATA = EOS_GETSW( fId, 'Latitude', SWATHNAME='swath1' )
; Close the file
STATUS = EOS_SW_CLOSE( FID )
IF ( STATUS lt 0 ) THEN MESSAGE, 'Error closing file!'
MODIFICATION HISTORY:
bmy, 18 Sep 2002: TOOLS VERSION 1.51
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
(See /n/home09/ryantosca/IDL/gamap2/file_io/eos_getsw.pro)
NAME:
EXTRACT_FILENAME
PURPOSE:
Extract the filename from a fully qualified filepath
CATEGORY:
File & I/O
CALLING SEQUENCE:
FILENAME = EXTRACT_FILENAME( FULLNAME [ , Keywords ] )
INPUTS:
FULLNAME --> a fully qualified filename containing path information.
KEYWORD PARAMETERS:
FILEPATH --> a named variable that returns the path of the
file. This can be used if both, the filename and the name
of the file will be used. Otherwise it is recommended to
use EXTRACT_PATH instead.
OUTPUTS:
A string containing the filename to be analyzed.
SUBROUTINES:
External Subroutines Required:
===============================
ADD_SEPARATOR (function)
REQUIREMENTS:
Requires routines from the TOOLS package.
NOTES:
See also EXTRACT_PATH
EXAMPLE:
PRINT, EXTRACT_FILENAME( '~/IDL/tools/extract_filename.pro')
extract_filename.pro
; Prints just the file name part of a longer path.
PRINT,EXTRACT_FILENAME( 'example.dat', filepath=filepath )
example.dat'
; will print 'example.dat', and filepath will contain ''
MODIFICATION HISTORY:
mgs, 18 Nov 1997: VERSION 1.00
mgs, 21 Jan 1999: - added extra check for use of '/' path
specifiers in Windows OS;
bmy, 19 Jan 2000: TOOLS VERSION 1.44
- replaced obsolete RSTRPOS( ) command with
STRPOS( /REVERSE_SEARCH ) for IDL 5.3+
- updated comments, few cosmetic changes
bmy, 13 Mar 2001: TOOLS VERSION 1.47
- Add support for MacOS operating system
bmy, 17 Jan 2002: TOOLS VERSION 1.50
- now call RSEARCH for backwards compatibility
with versions of IDL prior to v. 5.2
- use FORWARD_FUNCTION to declare RSEARCH
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
- now use ADD_SEPARATOR
- updated comments
(See /n/home09/ryantosca/IDL/gamap2/file_io/extract_filename.pro)
NAME:
EXTRACT_PATH
PURPOSE:
Extract the file path from a full qualified filename
CATEGORY:
File & I/O
CALLING SEQUENCE:
Path = EXTRACT_PATH( FULLNAME [, Keywords] )
INPUTS:
FULLNAME --> a fully qualified filename. If this input is
already a path it must end with the delimiter '/' (Unix),
'\' (Windows), or ':' (MacOS).
KEYWORD PARAMETERS:
FILENAME --> a named variable that returns the name of the
file. This can be used if both, the path and the name
of the file will be used. Otherwise it is recommended to
use EXTRACT_FILENAME instead.
OUTPUTS:
A string containing the path to the file given.
SUBROUTINES:
External Subroutines Required:
===============================
ADD_SEPARATOR (function)
RSEARCH (function)
REQUIREMENTS:
None
NOTES:
See also EXTRACT_FILENAME
EXAMPLE:
print,extract_path('~mgs/IDL/tools/extract_path.pro')
will print '~mgs/IDL/tools/'
print,extract_path('example.dat',filename=filename)
will print '', and filename will contain 'example.dat'
MODIFICATION HISTORY:
mgs, 18 Nov 1997: VERSION 1.00
mgs, 21 Jan 1999: - added extra check for use of '/' path
specifiers in Windows OS
bmy, 19 Jan 2000: TOOLS VERSION 1.44
- replaced obsolete RSTRPOS( ) command with
STRPOS( /REVERSE_SEARCH ) for IDL 5.3+
- updated comments, few cosmetic changes
bmy, 13 Mar 2001: TOOLS VERSION 1.47
- Add support for MacOS operating system
bmy, 17 Jan 2002: TOOLS VERSION 1.50
- now call RSEARCH for backwards compatibility
with versions of IDL prior to v. 5.2
- use FORWARD_FUNCTION to declare RSEARCH
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
- Now use ADD_SEPARATOR
- Updated comments
(See /n/home09/ryantosca/IDL/gamap2/file_io/extract_path.pro)
NAME:
FILE_EXIST
PURPOSE:
FILE_EXIST checks to see whether a specified file
can be found on disk, or if it does not exist.
CATEGORY:
File & I/O
CALLING SEQUENCE:
RESULT = FILE_EXIST( FILE [,OPTIONS])
INPUTS:
FILE (str) --> the name of the file to be checked
KEYWORD PARAMETERS:
PATH -> a path string (e.g. the IDL system variable !PATH)
or a list (string array) of directory names to be
searched for FILE. Under Unix, a trailing '/' is
attached to each entry; under Windows, a trailing
'\'; under MacOS, a trailing ':'. VMS isn't supported.
FULL_PATH -> returns the path of FILE if found. This is
not a true systemwide path but rather a combination
of a PATH element (which may be relative) and FILE.
DIRNAMES -> This keyword is now replaced by PATH, and
should not be used any more.
OUTPUTS:
RESULT -> =1 if the file is found or =0 otherwise
SUBROUTINES:
External Subroutines Required:
================================================
ADD_SEPARATOR (function) MFINDFILE (function)
REQUIREMENTS:
Requires routines from the TOOLS package.
NOTES:
(1) The PATH entries are expanded prior to use, so it is
possible to specify e.g. '~mgs/bla.pro'
(2) FILE_EXIST will always return the first file it
finds that matches your specification.
EXAMPLES:
(1)
IF ( FILE_EXIST( 'file_exist.pro' ) ) THEN PRINT, 'Found it!'
; Search for file_exist.pro
(2)
DIRS = [ '../', '~/DATA/' ]
OK = FILE_EXIST( 'test.dat', path=dirs, full=path )
IF ( OK ) THEN OPENR, U1, PATH
...
; Search for a file given a list of directories.
; If file is found, then open it for reading.
MODIFICATION HISTORY:
mgs, 26 Sep 1997: VERSION 1.00
mgs, 28 Sep 1997: - added expand_path() in order to digest ~-pathnames
- initializes FULL_PATH with a zero string
mgs, 06 Nov 1997: - replaced DIRNAMES by PATH and added
string seperation if PATH is a path
string with multiple entries
mgs, 05 Feb 1998: - bug fix: use expand_path also if only
filename is given
bmy, 13 Mar 2001: TOOLS VERSION 1.47
- now supports Windows, MacOS, and Unix
- cosmetic change, updated comments
bmy, 17 Jan 2002: TOOLS VERSION 1.50
- now call STRBREAK wrapper routine from
the TOOLS subdirectory for backwards
compatiblity for string-splitting;
bmy, 03 Oct 2003: TOOLS VERSION 1.53
- minor bug fix: FILE must be placed w/in
the call to EXPAND_PATH for IDL 6.0+
- deleted obsolete code from Jan 2002
bmy, 28 May 2004: TOOLS VERSION 2.02
- now call MFINDFILE instead of FINDFILE,
since MFINDFILE will call the new
FILE_SEARCH program for IDL 5.5+
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
- Now use ADD_SEPARATOR
- Updated comments
(See /n/home09/ryantosca/IDL/gamap2/file_io/file_exist.pro)
NAME:
FUTURE2BPCH
PURPOSE:
Converts future emission growth factor files from the obsolete
binary format to binary punch format (so that they can be
read by GAMAP).
CATEGORY:
File & I/O, BPCH Format
CALLING SEQUENCE:
FUTURE2BPCH, [ Keywords ]
INPUTS:
None
KEYWORD PARAMETERS:
INFILENAME -> Name of the input file containing fossil
fuel scale factors. If omitted, SCALEFOSS2BPCH
will prompt the user for a filename via a dialog box.
OUTFILENAME -> Name of the binary punch file containing
fossil fuel scale factors. Default is to add a
".bpch" extension to INFILENAME.
OUTPUTS:
None
SUBROUTINES:
External Subroutines Required
==================================================
CTM_TYPE (function) CTM_GRID (function)
NYMD2TAU (function) CTM_MAKE_DATAINFO (function)
CTM_WRITEBPCH EXTRACT_FILENAME (function)
REQUIREMENTS:
None
NOTES:
None
EXAMPLE:
FUTURE2BPCH, INFILENAME='scalefoss.liq.2x25.1998', $
OUTFILENAME='scalefoss.liq.2x25.1998.bpch'
; Converts scalefoss files to BPCH format.
MODIFICATION HISTORY:
bmy, 25 Jan 2006: VERSION 1.00
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
bmy, 02 Apr 2008: GAMAP VERSION 2.12
- Now read bpch as big-endian
(See /n/home09/ryantosca/IDL/gamap2/file_io/future2bpch.pro)
NAME:
GET_FREELUN (function)
PURPOSE:
Return next available logical unit number. Unlike
the internal GET_LUN procedure, this function is not
restricted to unit numbers above 100, and it will
detect any blocked unit number.
CATEGORY:
File & I/O
CALLING SEQUENCE:
lun = GET_FREELUN([LUN])
INPUTS:
none
KEYWORD PARAMETERS:
none
OUTPUTS:
The lowest available logical unit number. This number is
also returned in the LUN parameter for later use.
SUBROUTINES:
None
REQUIREMENTS:
None
NOTES:
None
EXAMPLE:
OPENW, GET_FREELUN( LUN ), FILENAME
; Open a file and get the next free unit number.
MODIFICATION HISTORY:
mgs, 17 Sep 1998: VERSION 1.00
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
- Updated comments, cosmetic changes
(See /n/home09/ryantosca/IDL/gamap2/file_io/get_freelun.pro)
NAME:
HDF_GETSD
PURPOSE:
Convenience routine to read scientific dataset variables
from Hierarchical Data Format (HDF) files
CATEGORY:
File & I/O, Scientific Data Formats
CALLING SEQUENCE:
DATA = HDF_GETSD( FID, NAME [, _EXTRA=e ] )
INPUTS:
FID -> HDF File ID, as returned by routine HDF_SD_START.
NAME -> Name of the scientific dataset variable that
you want to extract from the file.
KEYWORD PARAMETERS:
_EXTRA=e -> Passes extra keywords to routine HDF_SD_GETDATA.
OUTPUTS:
DATA -> Array containing extracted data from the HDF file.
SUBROUTINES:
None
REQUIREMENTS:
Need to use a version of IDL w/ HDF routines installed.
NOTES:
Taken from MOP02Viewer by Yottana Khunatorn (bmy, 7/17/01)
EXAMPLE:
; Make sure HDF is supported on this platform
IF ( HDF_EXISTS() eq 0 ) then MESSAGE, 'HDF not supported!'
; Open the HDF file and get the file ID # (FID)
FID = HDF_SD_START( 'fvdas_flk_01.ana.eta.20011027.hdf', /Read )
IF ( FID lt 0 ) then MESSAGE, 'Error opening file!'
; Read the UWND field from disk
DATA = HDF_GETSD( fId, 'UWND' )
; Close the file
HDF_SD_END, FID
MODIFICATION HISTORY:
bmy, 05 Nov 2001: VERSION 1.00
bmy, 23 Apr 2002: TOOLS VERSION 1.50
- updated documentation
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
(See /n/home09/ryantosca/IDL/gamap2/file_io/hdf_getsd.pro)
NAME:
HDF_GETSDATTR
PURPOSE:
Convenience routine to read attributes (global or variable-
associated) from Hierarchical Data Format (HDF) files.
CATEGORY:
File & I/O, Scientific Data Formats
CALLING SEQUENCE:
DATA = HDF_GETSDATTR( ID, NAME [ , Keywords ] )
INPUTS:
ID -> HDF File ID as returned by routine HDF_SD_START,
or scientific dataset ID, as returned by routine
HDF_SD_SELECT.
NAME -> Name of the attribute to be read from the HDF file.
KEYWORD PARAMETERS:
COUNT -> Returns the total number of values in the
specified attribute to the calling program.
HDF_TYPE -> Returns the HDF type of the attribute to the
calling program. HDF types are returned as a scalar
string. Possible returned values are DFNT_NONE,
DFNT_CHAR, DFNT_FLOAT32, DFNT_FLOAT64, DFNT_INT8,
DFNT_INT16, DFNT_INT32, DFNT_UINT8, DFNT_UINT16, and
DFNT_UINT32.
TYPE -> Returns the IDL type pf the attribute to the calling
program. The type of the attribute is returned as a
scalar string. Possible returned values are BYTE, INT,
LONG, FLOAT, DOUBLE, STRING, or UNKNOWN.
OUTPUTS:
DATA -> Array containing attribute data from the HDF file.
SUBROUTINES:
IDL HDF routines used:
==========================
HDF_SD_AttrInfo
HDF_SD_AttrFind (function)
REQUIREMENTS:
Need to use a version of IDL w/ HDF routines installed.
NOTES:
None
EXAMPLE:
; Make sure HDF is supported on this platform
IF ( NCDF_EXISTS() eq 0 ) then MESSAGE, 'HDF not supported!'
; Open the HDF file and get the file ID # (FID)
FID = HDF_SD_START( 'fvdas_flk_01.ana.eta.20011027.hdf', /READ )
IF ( FID lt 0 ) then MESSAGE, 'Error opening file!'
; Read the Ak, Bk, and PTOP attributes from the HDF file
; These are GLOBAL attributes associated w/ the file
AK = HDF_GETSDATTR( FID, 'ak' )
BK = HDF_GETSDATTR( FID, 'bk' )
PTOP = HDF_GETSDATTR( FID, 'ptop' )
; Close the HDF file
HDF_SD_END, FID
MODIFICATION HISTORY:
bmy, 30 Apr 2002: TOOLS VERSION 1.50
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
(See /n/home09/ryantosca/IDL/gamap2/file_io/hdf_getsdattr.pro)
NAME:
HDF_GETVD
PURPOSE:
Convenience routine to read VDATA variables
from Hierarchical Data Format (HDF) files
CATEGORY:
File & I/O, Scientific Data Formats
CALLING SEQUENCE:
VDATA = HDF_GETVD( FID, NAME [, _EXTRA=e ] )
INPUTS:
FID -> HDF File ID, as returned by routine HDF_OPEN.
NAME -> Name of the VDATA variable that you
want to extract from the file.
KEYWORD PARAMETERS:
_EXTRA=e -> Passes extra keywords to routine HDF_VD_READ.
OUTPUTS:
VDATA -> Array containing extracted data from the HDF file.
SUBROUTINES:
None
REQUIREMENTS:
Need to use a version of IDL w/ HDF routines installed.
NOTES:
Taken from MOP02Viewer by Yottana Khunatorn (bmy, 7/17/01)
EXAMPLE:
FID = HDF_OPEN( 'fvdas_flk_01.ana.eta.20011027.hdf', /Read )
IF ( FID lt 0 ) then Message, 'Error opening file!'
PTOP = HDF_GETVD( fId, 'PTOP' )
HDF_CLOSE, FID
; Opens an HDF-format file and gets the file ID. Then
; call HDF_GETSD to return the PTOP variable from the
; file. Then close the file and quit.
MODIFICATION HISTORY:
bmy, 05 Nov 2001: VERSION 1.00
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
(See /n/home09/ryantosca/IDL/gamap2/file_io/hdf_getvd.pro)
NAME:
HDF_SETSD
PURPOSE:
Convenience routine to write data into the Hierarchical Data
Format Scientific Dataset (HDF-SD) structure
CATEGORY:
File & I/O, Scientific Data Formats
CALLING SEQUENCE:
HDF_SETSD, FID, DATA, NAME [, Keywords ]
INPUTS:
FID -> HDF File ID, as returned by routine HDF_SD_START.
DATA -> Data (array or scalar) to be written to HDF-SD format.
NAME -> Name under which the data array will be saved
to the HDF file.
KEYWORD PARAMETERS:
LONGNAME -> Longer descriptive name for the data. This will
be saved as the "long_name" attribute. Default is ''.
RANGE -> A 2-element vector containing the [min,max] of
the data array. If not passed, RANGE will be computed
(but only for numeric data types). RANGE will be saved
to the HDF file as the "valid_range" attribute.
_EXTRA=e -> picks up extra keywords for HDF_SD_SETINFO, such
as FILL, UNIT, COORDSYS, etc...
OUTPUTS:
None
SUBROUTINES:
Uses the following IDL HDF routines:
===========================================
HDF_SD_Create (function) HDF_SD_SetInfo
HDF_SD_AddData HDF_SD_EndAccess
DATATYPE (function)
REQUIREMENTS:
Need to use a version of IDL w/ HDF routines installed.
NOTES:
(1) Since HDF supports the STRING type, we do not have to
treat BYTE data like ASCII characters (cf ncdf_set.pro)
EXAMPLE:
; Find out if HDF is supported on this platform
IF ( HDF_EXISTS() eq 0 ) then MESSAGE, 'HDF not supported!'
; Open the HDF file
FID = HDF_SD_START( 'myhdf.hdf', /Create )
IF ( FID lt 0 ) then Message, 'Error opening file!'
; Write data to disk
HDF_SETSD, FID, DATA, 'NOx', $
LONGNAME='Nitrogen Oxide',$
UNIT='v/v', $
FILL=0.0,
; Close HDF File
HDF_SD_END, FID
; Writes NOx data to an HDF file.
MODIFICATION HISTORY:
bmy, 17 Apr 2002: TOOLS VERSION 1.50
bmy, 11 Sep 2002: TOOLS VERSION 1.51
- Now call routine DATATYPE to determine
the type of the data so that we can
write all data types to the HDF file.
- Don't add the RANGE attribute to
the HDF file for a string type value.
- Updated comments
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
(See /n/home09/ryantosca/IDL/gamap2/file_io/hdf_setsd.pro)
NAME:
IS_DIR (function)
PURPOSE:
Tests if a directory exists
CATEGORY:
File & I/O
CALLING SEQUENCE:
RESULT = IS_DIR( PATH )
INPUTS:
PATH -> The variable to be tested.
KEYWORD PARAMETERS:
None
OUTPUTS:
RESULT -> =1 if directory exists, =0 otherwise
SUBROUTINES:
None
REQUIREMENTS:
None
NOTES:
None
EXAMPLE:
PRINT, IS_DIR( '~/IDL/tools/' )
1
; Test the existence of the ~/IDL/tools directory.
MODIFICATION HISTORY:
R.Bauer, 26 Jan 1999: INITIAL VERSION
- from Forschungszentrum Juelich GmbH ICG-1
bmy, 24 May 2007: TOOLS VERSION 2.06
- updated comments
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
(See /n/home09/ryantosca/IDL/gamap2/file_io/is_dir.pro)
NAME:
MAKE_MULTI_NC
PURPOSE:
Convert bpch files (matching a mask) to netCDF format.
The routine can work recursively : files in subdirectories
are also searched for.
This is a convenient generic wrapper for GAMAP routine
Bpch2coards.pro, to work on several files and directories at
once.
When working recursively, the directory tree in the input
directory is recreated as needed in the output directory.
CATEGORY:
File & I/O, BPCH Format, Scientific Data Formats
CALLING SEQUENCE:
MAKE_MULTI_NC
INPUTS:
none
KEYWORD PARAMETERS:
MASK -> string, default is '*', that is all files
INPUT_PARENT_DIR -> top directory of files to convert, if
not specified, a dialog window pops up
OUTPUT_PARENT_DIR -> top directory destination for netCDF
files, if not specified, a dialog window pops up.
RECURSION -> to search subdirectories. Default is 0
(OFF). Set to 1 to turn it on.
TOKEN -> set if you want to replace 'bpch' with 'nc' in all
part of the full name. Default is to have extension
".nc" added to file name only.
OUTPUTS:
None
REQUIREMENTS:
References routines from the GAMAP package.
NOTES:
To work recursively on the directories, I use FILE_SEARCH
with 2 arguments. This is not possible with MFINDFILE, and
works only with IDL 5.5 and above.
EXAMPLES:
indir = '/as/data/geos/GEOS_1x1/EDGAR_200607/'
outdir = '/home/phs/data/EDGAR_200607/nc/'
mask = '*1x1'
make_multi_nc, input_parent_dir=indir, out=outdir, mask=mask, /r
MODIFICATION HISTORY:
Thu Oct 29 16:40:28 2009, Philippe Le Sager
now pass _extra keyword to BPCH2COARDS (eg /NAMER
needed for nested data)
Thu Feb 19 11:12:16 2009, Philippe Le Sager
renamed MAKE_MULTI_NC. Added recursion keyword
(default is OFF). Now **AUTOMATICALLY** creates missing
subdirectory in output directory tree, when in
recursive mode.
Tue Jan 27 10:40:49 2009, Philippe Le Sager
v1, based on make_c_nc.pro to work on several
directories at once.
bmy, 30 Nov 2010: GAMAP VERSION 2.15
- Updated comments and category in header
bmy, 01 Feb 2012: GAMAP VERSION 2.16
- Skip processing ASCII text files
mps, 04 Mar 2015: - Bug fix: Now assign InFileName before checking
file type
(See /n/home09/ryantosca/IDL/gamap2/file_io/make_multi_nc.pro)
NAME:
MAKE_NC_RECURSIVE
PURPOSE:
Converts files matching a mask to netCDF format.
The routine works recursively. This is a convenience
wrapper for GAMAP routine bpch2coards.pro, to work on
several directories at once.
You just need to (1) create the same directory tree in the
output location, then (2) run the routine once.
CATEGORY:
File & I/O, BPCH Format, Scientific Data Formats
CALLING SEQUENCE:
MAKE_NC_RECURSIVE
INPUTS:
none
KEYWORD PARAMETERS:
MASK -> string, default is '*', that is all files
INPUT_PARENT_DIR -> top directory of files to convert, if
not specified, a dialog window pops up
NEW_PARENT_DIR -> top directory destination for netCDF
files, if not specified, a dialog window pops up
TOKEN -> set if you want to replace 'bpch' with 'nc' in all
part of the full name
OUTPUTS:
None
REQUIREMENTS:
References routines from the GAMAP package.
NOTES:
To work recursively on the directories, I use FILE_SEARCH
with 2 arguments. This is not possible with MFINDFILE, and
works only with IDL 5.5 and above.
EXAMPLE:
; after creating a subdirectories tree in outdir similar to the
; one in indir:
indir = '/as/data/geos/GEOS_1x1/EDGAR_200607/'
outdir = '/home/phs/data/EDGAR_200607/nc/'
mask = '*1x1'
make_nc_recursive, input_parent_dir=indir, new=outdir, mask=mask
MODIFICATION HISTORY:
Tue Jan 27 10:40:49 2009, Philippe Le Sager
v1, based on make_c_nc.pro to work on several
directories at once.
bmy, 30 Nov 2010: GAMAP VERSION 2.15
- Updated comments and categories in header
(See /n/home09/ryantosca/IDL/gamap2/file_io/make_nc_recursive.pro)
NAME:
MAKE_PDF
PURPOSE:
Wrapper program for the Unix utility "ps2pdf". Creates
a PDF file for each PostScript file located in the
a user-specified directory.
CATEGORY:
File & I/O
CALLING SEQUENCE:
MAKE_PDF, DIR
INPUTS:
DIR -> Directory where PostScript files reside.
PDF files will be written out to this directory.
KEYWORD PARAMETERS:
None
OUTPUTS:
None
SUBROUTINES:
External Subroutines Required:
===============================
ADD_SEPARATOR (function)
MFINDFILE (function)
REPLACE_TOKEN (function)
REQUIREMENTS:
You need "ps2pdf" installed on your system. This ships
with most versions of Unix or Linux.
Also requires routines from the GAMAP package.
NOTES:
PDF files will have the same names as the PostScript files
but with the *.pdf extension.
EXAMPLES:
MAKE_PDF, 'output'
; Create *.pdf Files from all *.ps files
; in the "output" directory.
MODIFICATION HISTORY:
bmy, 30 Nov 2010: GAMAP VERSION 2.15
- Initial version
bmy, 08 Jun 2011: - Make sure that the directory ends with a
path separator
(See /n/home09/ryantosca/IDL/gamap2/file_io/make_pdf.pro)
NAME:
MAKE_RESTART
PURPOSE:
Creates a restart file for GEOS-Chem.
CATEGORY:
File & I/O, BPCH Format
CALLING SEQUENCE:
MAKE_RESTART [, Keywords ]
INPUTS:
None
KEYWORD PARAMETERS:
OUTMODELNAME -> Name of the model grid onto which the data
will be regridded. Default is "GEOS4_30L".
OUTRESOLUTION -> Specifies the resolution of the model grid
onto which the data will be regridded. OUTRESOLUTION
can be either a 2 element vector with [ DI, DJ ] or
a scalar (DJxDI: 8=8x10, 4=4x5, 2=2x2.5, 1=1x1,
0.5=0.5x0.5). Default for all models is 4x5.
OUTFILENAME -> Name of the restart file that will be created.
Default is "restart.bpch".
TAU0 -> TAU value (hours from 0 GMT on 01 Jan 1985) that
that will be used to timestamp the restart file.
The default TAU0 value 140256.00 (corresponds to
0 GMT on 01 Jan 2001).
TRACERLIST -> A scalar value (or vector) of tracer numbers
to write to the restart file. Default is 1.
DATAVALUE -> Specifies the data value that will be assigned
to all grid boxes in the restart file. Default is 0e0.
DIAGN -> Specifies the diagnostic category name that will
be used to index data fields in the restart file.
Default is "IJ-AVG-$".
UNIT -> Use this keyword to overwrite the default unit
string (as specified in the "tracerinfo.dat" file)
for each data block in the restart file.
/NO_VERTICAL -> Set this switch to create a restart file
with 2-D data blocks (e.g. lon x lat) instead of 3-D
data blocks (e.g. lon x lat x alt).
DIM -> Allows you to manually set the dimensions of the
data blocks (for nested grids). DIM is passed to
CTM_MAKE_DATAINFO.
FIRST -> Allows you to manually set the starting indices
(in Fortran notation, starting from 1) of the data
block. FIRST is passed to CTM_MAKE_DATAINFO.
OUTPUTS:
None
SUBROUTINES:
External Subroutines Required:
===================================================
CTM_GET_DATABLOCK (function) CTM_GRID (function)
CTM_MAKE_DATAINFO (function) CTM_TYPE (function)
CTM_WRITEBPCH NYMD2TAU
REQUIREMENTS:
None
NOTES:
(1) You must make sure that your "diaginfo.dat" and
"tracerinfo.dat" file lists the diagnostic categories
and tracers that you wish to save to the restart file.
EXAMPLE:
MAKE_RESTART, OUTMODELNAME='GEOS4_30L', $
OUTRESOLUTION=2, $
OUTFILENAME='restart.2x25.2005010100', $
TAU0=NYMD2TAU( 20050101L ), $
DATAVALUE=100e0, $
TRACERLIST=[1,2,3,4,5,6], $
UNIT='ppbv', $
DIAGN='IJ-AVG-$'
; Create a GEOS-4 30-level 2x25 restart file for
; CO2 tracers 1-6, setting all tracers equal to
; 100 ppbv.
MODIFICATION HISTORY:
bmy, 19 Jul 2007: VERSION 1.00
bmy, 27 Oct 2010: GAMAP VERSION 2.15
- Added DIM and FIRST for setting the
region properly for nested grids
(See /n/home09/ryantosca/IDL/gamap2/file_io/make_restart.pro)
NAME:
MFINDFILE
PURPOSE:
Find all the files that match a given specification.
MFINDFILE is a wrapper for IDL routines FILE_SEARCH
(v5.5 and higher) and FINDFILE (v5.4 and lower).
CATEGORY:
File & I/O
CALLING SEQUENCE:
LISTING = MFINDFILE( MASK )
INPUTS:
FILEMASK -> a path and filename specification to look for.
KEYWORD PARAMETERS:
none
OUTPUTS:
A string list containing all the files that match the
specification.
SUBROUTINES:
External Routines Required:
===========================
ADD_SEPARATOR (function)
REQUIREMENTS:
None
NOTES:
(1) For IDL 5.5+, use FILE_SEARCH to return a listing of
files with the given path name specified by MASK. This
should work regardless of platform.
(2) For IDL 5.4- running under Unix, The built-in FINDFILE()
function has problems on whenever a lot of files are
matching the file specification. This is due to the fact
that filename expansion is done by the shell before
interpreting a command. Too many files cause too long
commands which are not accepted. This causes FINDFILE()
to return an empty list of candidates. (cf. www.dfanning.com)
Therefore, we implement a workaround where we issue a
"ls -1" command under the Unix shell. This isn't 100%
foolproof either but it's better than nothing.
(3) For IDL 5.5- running under other operating systems,
call the built-in IDL FINDFILE routine as usual.
EXAMPLE:
LIST = MFINDFILE( '~mgs/terra/chem1d/code/*.f' )
; returns all fortran files in Martin's chem1d directory.
MODIFICATION HISTORY:
mgs, 14 Sep 1998: VERSION 1.00
bmy, 14 Oct 2003: TOOLS VERSION 1.53
- Now use built-in FINDFILE() routine to
return file listing for IDL 5.3 and higher
bmy, 06 Nov 2003: TOOLS VERSION 2.01
- return to pre v1-53 algorithm
bmy, 28 May 2004: TOOLS VERSION 2.02
- For IDL 5.5+, now use FILE_SEARCH to return
a list of files corresponding to MASK
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
(See /n/home09/ryantosca/IDL/gamap2/file_io/mfindfile.pro)
NAME:
NCDF_GET
PURPOSE:
Convenience routine to read data into netCDF format.
CATEGORY:
File & I/O, Scientific Data Formats
CALLING SEQUENCE:
DATA = NCDF_GET( FID, NAME [, Keywords ] )
INPUTS:
FID -> netCDF File ID, as returned by routine NCDF_OPEN.
NAME -> Name under which the data array will be saved
to the netCDF file.
KEYWORD PARAMETERS:
VARINFO -> Returns a structure containing information
about the variable read in from disk. The structure
has the following form:
{ NAME : "", $
DATATYPE : "", $
NDIMS : 0L, $
NATTS : 0L, $
DIM : LONARR(NDIMS) }
LONGNAME -> Returns the value saved under the "long_name"
attribute in the netCDF file.
UNIT -> Returns the value of the "unit" attribute
saved in the netCDF file.
RANGE -> Returns the value of the "valid_range"
saved in the netCDF file
_EXTRA=e -> Picks up extra keywords got NCDF_VarGet.
OUTPUTS:
DATA -> Array containing extracted data from the netCDF file.
SUBROUTINES:
Uses the following IDL netCDF routines:
========================================
NCDF_VarId (function) NCDF_VarGet
NCDF_VarInfo (function) NCDF_AttGet
NCDF_AttName (function)
REQUIREMENTS:
Need to use a version of IDL w/ netCDF routines installed.
NOTES:
(1) Only looks for the "long_name", "unit", and "valid_range"
attributes. The user can extend this as he/she desires.
For a more general program, see ~/IDL/tools/ncdf_read.pro
by Martin Schultz.
EXAMPLE:
; Define array to write to file
ARRAY = DIST( 100, 50 )
; Find out if netCDF is supported on this platform
IF ( NCDF_EXISTS() eq 0 ) then MESSAGE, 'netCDF not supported!'
; Open netCDF file and get the file ID # (FID)
FID = NCDF_OPEN( 'myfile.nc' )
IF ( FID lt 0 ) then Message, 'Error opening file!'
; Read data from the netCDF file
; Return data attributes in the VARINFO array
; Also returns the text from the UNIT string
DATA = NCDF_GET( FID, 'BIOBSRCE::NOx', $
VARINFO=VARINFO, UNIT=UNIT )
; Close the netCDF file
NCDF_CLOSE, FID
MODIFICATION HISTORY:
bmy, 22 May 2002: TOOLS VERSION 1.50
bmy, 21 Oct 2003: TOOLS VERSION 1.53
- If we can't find a netCDF variable name,
then try again using a "sanitized" name
w/ all bad characters stripped out
bmy, 28 Jun 2007: TOOLS VERSION 2.06
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
- Added /VERBOSE keyword to reduce
warning messages
(See /n/home09/ryantosca/IDL/gamap2/file_io/ncdf_get.pro)
NAME:
NCDF_READ
PURPOSE:
Open a netCDF file and read data from it. The data is
returned as a structure whose tag names are the names of
the variables with blanks etc. replaced. If no variables
are specified with the VARIABLES keyword, only dimensional
information is returned. You can load all variables using
the ALL keyword. Other keyword options include OFFSET, COUNT, STRIDE,
NO_DIMENSIONS, NO_STRUCT, DIMNAMES, VARNAMES, VARDIMS, ATTRIBUTES.
Thus, this program includes ncdump functionality.
If no filename is given, a file selection dialog is
opened with the default mask '*.nc'. The name of the selected
file is returned in the TRUENAME keyword. A file selection
dialog also appears when the file cannot be found (see
OPEN_FILE.PRO). This can be turned off with the NO_DIALOG
keyword. The VERBOSE keyword provides information while
analyzing and reading the file.
AUTHOR:
Dr. Martin Schultz
Max-Planck-Institut fuer Meteorologie
Bundesstr. 55, D-20146 Hamburg
email: martin.schultz@dkrz.de
CATEGORY:
File & I/O, Scientific Data Formats
CALLING SEQUENCE:
NCDF_READ, result, filename=, truename=,
variables=, all=, varnames=,
vardimid=, vardims=, attributes=,
count=, offset=, stride=,
dimnames=, dims=, no_dimensions=,
no_struct=, no_dialog=, verbose=, title=
ARGUMENTS:
RESULT(out) -> a structure containing all the variable data
from the netCDF file. If only one variable is specified
and the NO_STRUCT keyword set, then RESULT will be an
array instead of a structure. Use the ALL keyword
to load all variables at once. Note, that the COUNT, OFFSET,
and STRIDE keywords can affect the size of RESULT.
RESULT is set to -1L if an error occurs before the structure
has been built. You can use CHKSTRU.PRO to test for this.
KEYWORD PARAMETERS:
FILENAME(in) -> the name of the netCDF file to be opened.
NCDF_READ uses OPEN_FILE to check the validity of
the file first. You can specify a search mask
instead of a filename in which case a file selection
dialog is displayed (unless you set the NO_DIALOG
keyword). The TRUENAME keyword contains the name
of the selected file or an empty string if the
file selection was canceled.
TRUENAME(out) -> the (fully qualified) name of the file selected
with the file selection dialog or an unaltered copy
of FILENAME if FILENAME is a valid filename.
VARIABLES(in) -> a string array containing the names of variables
for which data shall be read. Default is to read
only the dimensional information from the file.
(Currently) no warning is issued if a variable is not in the file.
ALL(in) -> set this keyword to load all variables stored in the
netCDF file. Generally, you cannot usethis keyword together with
COUNT, OFFSET, and STRIDE.
VARNAMES(out) -> a string array containing the names of all variables
as stored in the file. Note, that the tag names of e.g. the
result structure are filtered with the Valid_TagName function.
VARDIMID(out) -> a structure with integer arrays containing the
dimension ID's for all variables. See also VARDIMS which returns
the dimensions themselves.
VARDIMS(out) -> a structure with integer arrays containing the
dimensions for all variables in the netCDF file. These are not
kept in sync with potential COUNT, OFFSET, and STRIDE values,
but reflect the data sizes as stored in the file.
ATTRIBUTES(out) -> a structure holding the variable and global
attributes stored in the netCDF file (global attributes
are stored in tag name GLOBAL).
COUNT(in) -> an integer array containing the number of values to
be read for each dimension of the variables. Mapping of the
COUNT dimensions to the variable dimensions is achieved via
the first entry in the VARIABLES list and the COUNT parameter
will be applied to all variables that have that dimension.
Example: The first variable has dimensions LON, LAT, LEV,
the second variable has dimensions LON, LAT, and the third
variable has LAT, LEV. A COUNT of [40,20,10] would lead to
result dimensions of [40,20,10], [40,20], and [20,10].
OFFSET(in) -> an integer array containing the offsets for each
dimension of the variables to be read. Dimension mapping
is the same as for COUNT.
STRIDE(in) -> an integer array containing the stride for each
dimension of the variables to be read. Dimension mapping
is the same as for COUNT.
DIMNAMES(out) -> a string array containing the names of the
dimensions stored in the netCDF file.
DIMS(out) -> a long array containing the dimension values. Purely
for convenience. Use VARDIMS to retrieve the dimensions of
the variables.
TITLE(in) -> A title for the file selection dialog if an
incomplete or incorrect filename is specified. This
keyword is ignored if the no_dialog keyword is set.
NO_DIMENSIONS(in) -> set this keyword if you do not want to store
the dimensional variables from the file in the result structure.
DIMNAMES and DIMS will still be retrieved.
NO_STRUCT(in) -> if only one variable is selected with the
VARIABLE keyword, you can set this keyword to return only
the data for this variable as an array. This keyword implies
the functionality of NO_DIMENSIONS.
NO_DIALOG(in) -> set this keyword if you do not want interactive
behaviour when a file mask is given instead of a filename or
when the specified file does not exist.
VERBOSE(in) -> set this keyword to get detailed information while
reading the netCDF file.
SUBROUTINES:
Valid_TagName : replaces invalid characters in variable names so that
a structure can be built.
ncdf_attributes : retrieves global and variable attributes from netcdf
file and stores them as structure.
ncdf_dimensions : retrieves size and name for all dimensions in netcdf file.
ncdf_varnames : retrieves names and dimension information for all variables
in the netCDF file.
ncdf_mapdims : map dimension indices for COUNT, OFFSET, and STRIDE with
dimensions of first selected variable.
ncdf_TestDimensions : compute the COUNT, OFFSET, and STRIDE vectors that
must be applied for retrieving the data of one variable.
REQUIREMENTS:
uses OPEN_FILE and STRREPL.
NOTES:
Correct handling of dimensional information requires that the variables
storing the dimension values have the same name as the dimensions
themselves - a common feature in most netCDF standards.
I am still working on a netcdf file object which will be even
more powerful. At some point ncdf_read will only be a
procedure interface to this objec!
EXAMPLE:
ncdf_read,result,/All
; plot ozone vs. temperature
plot,result.temperature,result.ozone
MODIFICATION HISTORY:
mgs, 18 Sep 1999: VERSION 1.00
mgs, 29 Feb 2000: - added variables keyword
- added CATCH error handler
mgs, 21 Mar 2000: - bug fix for tag names
mgs, 09 May 2000: VERSION 2.00
- now only reads dimensions as default
- added ALL keyword to compensate
- returns dimnames and attributes
(makes ncdf_detail obsolete)
- added COUNT, OFFSET and STRIDE keywords
- added NO_DIMENSIONS and NO_DIALOG
keywords and more
mgs, 22 Aug 2000: - added title keyword
bmy, 22 May 2002: GAMAP VERSION 1.50
- Now replace ":", "=", "#" with "_" in
structure tag names, so IDL won't choke
bmy, 21 Oct 2003: GAMAP VERSION 1.53
- also replace "(" and ")" in tag names
- replace "$" with "S" (close to dollar sign)
- replace "*" with "A" (for Asterisk)
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
(See /n/home09/ryantosca/IDL/gamap2/file_io/ncdf_read.pro)
NAME:
NCDF_SET
PURPOSE:
Convenience routine to write data into netCDF format.
CATEGORY:
File & I/O, Scientific Data Formats
CALLING SEQUENCE:
NCDF_SET, FID, DATA, NAME, DIMS [, Keywords ]
INPUTS:
FID -> netCDF File ID, as returned by routine NCDF_CREATE.
DATA -> Data (array or scalar) to be written to netCDF file.
NAME -> Name under which the data array will be saved
to the netCDF file.
KEYWORD PARAMETERS:
LONGNAME -> Longer descriptive name for the data.
This will be saved as the "long_name" attribute.
RANGE -> A 2-element vector containing the [min,max] of
the data array. If not passed, RANGE will be computed
(but only if DATA is a numeric type). RANGE is saved
to the netCDF file as the "valid_range" attribute.
UNIT -> String containing the units of the data.
This will be saved as the "unit" attribute.
_EXTRA=e -> Picks up extra keywords
OUTPUTS:
None
SUBROUTINES:
Uses the following routines:
=====================================================
NCDF_Control NCDF_VarDef (function)
NCDF_AttPut DATATYPE (function from TOOLS)
NCDF_VarPut NCDF_VALID_NAME (function from TOOLS)
REQUIREMENTS:
Need to use a version of IDL w/ netCDF routines installed.
NOTES:
(1) For now, treat BYTE data like CHAR data. This is
most likely since netCDF does not support STRING types,
strings have to be stored as arrays of bytes.
EXAMPLE:
; Define array to write to file
ARRAY = DIST( 100, 50 )
; Find out if netCDF is supported on this platform
IF ( NCDF_EXISTS() eq 0 ) then MESSAGE, 'netCDF not supported!'
; Open netCDF file and get the file ID # (FID)
FID = NCDF_CREATE( 'myfile.nc', /CLOBBER )
IF ( FID lt 0 ) then Message, 'Error opening file!'
; Set dimensions for netCDF file
S = SIZE( ARRAY, /DIM )
DIM1 = NCDF_DIMDEF( FID, 'Length', S[0] )
DIM2 = NCDF_DIMDEF( FID, 'Width', S[1] )
; Go into netCDF DATA mode
NCDF_CONTROL, FID, /ENDEF
; Call NCDF_SET to write the array to the file
NCDF_SET, FID, ARRAY, 'My Data', [ DIM1, DIM2 ], $
LONGNAME='Data array created by me!', $
UNIT='unitless'
; Close the netCDF file
NCDF_CLOSE, FID
MODIFICATION HISTORY:
bmy, 19 Apr 2002: TOOLS VERSION 1.50
bmy, 10 Sep 2002: TOOLS VERSION 1.51
- Now call routine DATATYPE to determine
the type of the data so that we can
write to the netCDF file appropriately
- Don't add the RANGE attribute to
the netCDF file for a string type value.
- Updated comments
bmy, 21 Oct 2003: TOOLS VERSION 1.53
- now "sanitize" the netCDF variable name
w/ routine NCDF_VALID_NAME. The new netCDF
library in IDL 6.0+ chokes on bad characters.
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
(See /n/home09/ryantosca/IDL/gamap2/file_io/ncdf_set.pro)
NAME:
NCDF_VALID_NAME
PURPOSE:
Strips invalid characters from a string which is to be
used as a netCDF variable name. Based on original code
by Martin Schultz.
CATEGORY:
File & I/O, Scientific Data Formats
CALLING SEQUENCE:
RESULT = NCDF_VALID_NAME( ARG )
INPUTS:
ARG -> netCDF variable name string to be examined.
KEYWORD PARAMETERS:
None
OUTPUTS:
RESULT -> New netCDF name string with "bad" characters
replaced by "good" characters.
SUBROUTINES:
External Subroutines Used:
==========================
STRREPL (function)
REQUIREMENTS:
None
NOTES:
In IDL 6.0+, the netCDF library has been updated. Some
characters which used to be allowed in netCDF variable names
are no longer allowed. Therefore, use this function to
replace "bad" characters with "good" characters when
reading or writing to/from netCDF files.
EXAMPLE:
RESULT = NCDF_VALID_NAME( 'IJ-AVG-$::CO' )
PRINT, RESULT
; Prints "IJ-AVG-S__CO"
MODIFICATION HISTORY:
bmy, 21 Oct 2003: VERSION 1.00
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
(See /n/home09/ryantosca/IDL/gamap2/file_io/ncdf_valid_name.pro)
NAME:
OPEN_FILE
PURPOSE:
Open a file for input. This routine can automatically
decide whether to use DIALOG_PICKFILE, and it contains
basic error handling. After successful operation the
file with logical unit LUN will be open for input.
CATEGORY:
File & I/O
CALLING SEQUENCE:
OPEN_FILE,filemask,lun [,keywords]
INPUTS:
FILEMASK -> a filename with path information that may
contain wildcards ('*', '?')
LUN -> a named variable that will contain the logical unit
number upon return. If a unit number > 0 is passed,
an attempt is made to open the file with this number,
otherwise, a free unit is selected with /GET_LUN. In case
of an error, LUN contains -1. This can be used instead of
the RESULT keyword to detect errors (see below).
(both parameters are mandatory !)
KEYWORD PARAMETERS:
FILENAME -> a named variable that will contain the complete
filename upon return (i.e. the file selected with PICKFILE)
WRITE -> Set this keyword to open a file for read and write
operations. Normally, a file is opened for reading only.
RESULT -> a named variable that will return the error status
of the operation. A value of 0 indicates the file was
opened sucessfully, otherwise the value of !Error_State.Code
is returned.
PICKFILE -> logical flag to force use of the DIALOG_PICKFILE
routine, even if a complete filemask without wildcards was
passed.
TITLE -> the title of the pickfile dialog. Default is
'Choose a file'.
DEFAULTMASK -> A default filemask to be used when no filename
is given or the filename does not contain wildcards and
/PICKFILE is set. This mask will also be used if the
file cannot be opened because of 'FILE NOT FOUND' error.
NO_PICKFILE -> prevents the pickfile dialog for batch operation.
The filemask must not contain wildcards.
Normally a 'FILE NOT FOUND' condition leads to
a second attempt with the /PICKFILE flag set (recursive
call). Use this flag if you want to abort instead.
_EXTRA keywords are passed to the openr routine
(e.g. /F77_UNFORMATTED)
OUTPUTS:
SUBROUTINES:
REQUIREMENTS:
Uses EXTRACT_FILENAME function
NOTES:
EXAMPLE:
(1)
; Quick and dirty with pickfile dialog
OPEN_FILE,'*.dat',ilun
if (ilun le 0) then stop ; check error condition
(2)
; A few more options invoked
OPEN_FILE,'~/data/thisfile.dat',lun,default='*.dat', $
title='Choose a data file',filename=name
IF (LUN LE 0) THEN STOP ; check error condition
PRINT, FILENAME,' was opened successfully
; NOTE that filename does not have to be identical with
; '~/data/thisfile.dat' !
; readf,lun,...
CLOSE, LUN
FREE_LUN, LUN
MODIFICATION HISTORY:
mgs, 13 Aug 1998: VERSION 1.00
- extracted from ctm_read3dp_header.pro and
modified
mgs, 14 Aug 1998: - small bug fix: handle empty filename
string correctly
mgs, 22 Aug 1998: - added WRITE keyword to open writable files
mgs, 22 Oct 1998: - now always returns LUN=-1 in case of an error
mgs, 21 Jan 1999: - Added explicit F77_Unformatted keyword and set
Swap_If_Little_Endian or Swap_If_Big_Endian
automatically
mgs, 10 Feb 1999: - bug fix: swap_if was wrong way round
mgs, 12 May 1999: - ok. finally got the hang of byte swapping!
It's the machine architecture not the operating
system! Now changed it so that !VERSION.ARCH is
tested for 'x86'
mgs, 20 May 1999: - abandoned SWAP_IF completely and use explicit
SWAP_ENDIAN keyword in users grace now.
bmy, 14 Oct 2003: TOOLS VERSION 1.53
- For IDL 6.0+, if PATH is a null string, then
manually reset it to './'. This will avoid
the contents of the !PATH variable from being
listed in the dialog box.
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
cdh & phs, 30 Jun 2008: GAMAP VERSION 2.12
- declare dummy GET_LUN keyword. It
prevents passing GET_LUN w/ _extra to
openr/w, since LUN is already assigned in
all cases.
(See /n/home09/ryantosca/IDL/gamap2/file_io/open_file.pro)
NAME:
PROGRAM_DIR
PURPOSE:
Given a file, returns the directory in which the file resides.
CATEGORY:
File & I/O
CALLING SEQUENCE:
RESULT = PROGRAM_DIR( FILE [, Keywords ] )
INPUTS:
FILE -> Name of the file for which a directory search
will be performed.
KEYWORD PARAMETERS:
/FULL_PATH -> Set this switch to return the directory
name as an absolute path (e.g. /users/home/IDL/)
instead of a relative path (e.g. ~/IDL).
OUTPUTS:
None
SUBROUTINES:
External Subroutines Required:
==============================
EXPAND_PATH (function)
EXTRACT_FILENAME (function)
REQUIREMENTS:
None
NOTES:
(1) Unix is case-sensitive. It is recommended to keep
file names in all lowercase on Unix to avoid file
search confusion.
EXAMPLES:
(1)
PRINT, PROGRAM_DIR( 'myct.pro' )
~/IDL/tools/
; Finds the directory in which "myct.pro" resides.
(2)
PRINT, PROGRAM_DIR( 'myct.pro', /FULL_PATH )
/users/ctm/bmy/IDL/tools
; Same as the above example, but this time returns
; the directory as an absolute path name.
MODIFICATION HISTORY:
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
(See /n/home09/ryantosca/IDL/gamap2/file_io/program_dir.pro)
NAME:
READDATA
PURPOSE:
This subroutine reads in almost any ASCII file and returns
two arrays containing the names of the variables and their
values. The data is read line by line or in one step if
LINES is specified.
CATEGORY:
File & I/O
CALLING SEQUENCE:
READDATA, FILENAME, DATA, HEADER [, NCOL, NDAT ] [, KEYWORDS ]
INPUTS:
FNAME -> Name of fname to be read, e.g. 'flight12.dat'
KEYWORD PARAMETERS:
COLS -> number of columns to be read (must be used if no header
is read in, i.e. /NOHEADER is specified). Can be used
to read in a subset of columns if the file contains a header
line with variable names (i.e. if *not* /NOHEADER).
LINES -> number of lines to be read (not much faster for large
data sets, but allows to read in a subset of the data)
DELIM -> Separator between names in the header (default=',')
SKP1 -> Number of lines to be skipped before the variable
definition (default=0)
SKP2 -> .. after the variable definition (default=0)
SKIP -> same as SKP1 (SKP1 will overwrite SKIP. SKIP may be
not longer supported in future versions !)
AUTOSKIP -> for files that state the number of comment lines
in the first line. If keyword NOHEADER is not set, READDATA
expects a list of variable names as last comment line.
AUTOSKIP overrides settings of SKP1 and SKP2.
TRANSPOSE -> Normally, 1st array dimension is for variables,
2nd is for observations. /TRANSPOSE will reverse that
order (see note).
NOHEADER -> don't read a header (COLS must be specified in
this case !)
NODATA -> don't read data (stop after header). DATA parameter
must still be specified !
COMMENTS -> returns string array of all the comment lines in
the data file for later use
MAXCOMMENTS -> limits maximum number of comment lines to be
retrieved (default: 255)
QUIET -> Normally, READDATA prints the number of variables
found and number of data lines read. Use this option
to suppress all output.
/DOUBLE -> If set, will return data in double precision.
(Default is to return data in single precision.)
NAN_REPLACEMENT -> Allows you to provide a replacement value
for the following types of missing data fields: NaN,
nan, NA, na, Infinity, infinity, Missing, missing.
OUTPUTS:
DATA -> data array that was read in
NAMES -> string array of names in header
NCOL -> integer containing the number of columns
NDAT -> long integer containing the number of observations
COMMENTS -> string array containing all header lines.
If AUTOSKIP is set, skp1, and skp2 will contain the
actual amount of lines to skip (e.g. for re-storing
header information in EXPLORE)
SUBROUTINES:
External Subroutines Required:
==============================
OPEN_FILE
STRBREAK (function)
USAGE
REQUIREMENTS:
None
NOTES:
Default of the returned DATA array is: 1st index = variable,
2nd index = observation. Use the /TRANSPOSE option for reverse order
If /NOHEADER is used, then COLS must specify the actual number of
data columns in FNAME. Otherwise it can be used to read a subset of
the data from 0 to cols-1 columns.
IDL Parameters are optional. Of course, you should not readdata without
passing a DATA argument, but you can ignore the HEADER,NCOL, and NDAT
params.
EXAMPLES:
(1)
READDATA,'mydata.dat',DATA,HEADER,DELIM=' ',SKIP=5
... will read in the ASCII file mydata.dat and store the data in DATA.
The header information will be stored in HEADER. The header items are
seperated by blank spaces, and the first 5 lines should be ignored.
To pick a certain variable afterwards, type:
VAR = DATA(WHERE HEADER EQ 'MYVAR'),*)
(2)
READDATA,'noheader.dat',DATA,DELIM=';',NCOLS=3
... will read a three column ASCII file with no header information.
You can manually make up a header with
HEADER = ['VAR1','VAR2','VAR3']
or you can pass the HEADER argument and receive ['X1','X2','X3'] as
header.
(3)
READDATA,'mydata.dat',DATA,HEADER,DELIM=' ',SKP1=5,LINES=60,COLS=4, $
COMMENTS=COMMENTS
... will read in 60 lines and 4 columns of the ASCII file mydata.dat
and return 6 comment lines in COMMENTS (5 + variable names)
MODIFICATION HISTORY:
mgs 03/12/1997: - last update : 05/22/97
mgs 01 Aug 1997: - added template
mgs 15 Sep 1997: - added LINES option and removed some twitch in the
handling of TRANSPOSE. Unfortunately,
LINES does not improve the speed as
desired, but you can restrict the reading to
a smaller subset of the data.
mgs 26 Sep 1997: MAJOR REVIEW
- bug fixes in noheader option
- bug fixes in COLS and NCOL handling
- removed units option and created comments
keyword instead. program now reads in
all header lines into a string array
including the variable names line.
- automatic generation of a header if
/NOHEADER is specified
mgs 06 Nov 1997: - Added AUTOSKIP option for easier reading
of e.g. NASA formatted files.
mgs 01 Dec 1997: added MAXCOMMENTS keyword and limit
- skp1 now returns correct amount if
autoskip is set
mgs 30 Dec 1997: added NODATA keyword
mgs 21 Aug 1998: now uses open_file routine to allow wildcards
bmy 12 May 2005: added /DOUBLE keyword to force double precision
bmy & phs, 21 Aug 2007: GAMAP VERSION 2.10
- Now use STRBREAK, which is version independent
- Remove internal function USE_READDATA; we
now call USAGE to display program options
if the wrong # of arguments are passed
- Updated comments
bmy, 12 Dec 2012: GAMAP VERSION 2.16
- Now give the user the option to read in
data as characters and then to strip out
strings such as "NaN" or "NA" via the
- NAN_REPLACEMENT keyword
(See /n/home09/ryantosca/IDL/gamap2/file_io/readdata.pro)
NAME:
READ_BDT0001
PURPOSE:
Read a simple binary data file with size information
and variable names and units (format BDT0001).
CATEGORY:
File & I/O
CALLING SEQUENCE:
READ_BDT0001,filename,data,vardesc,nvars,nlines [,keywords]
INPUTS:
FILENAME -> Name of the file to read or a file mask that
will be used in the PICKFILE dialog (see OPEN_FILE)
If FILENAME is a named variable, the actual filename
will be returned and replace a template.
KEYWORD PARAMETERS:
NAMES -> a named variable will contain a string array with
NVARS variable names
UNITS -> ... a string array with NVARS physical units
COMMENTS -> A named variable that will return comment lines
stored in the data file. NOTE that comments are not
saved in vardesc.
DEFAULTMASK -> Default mask for PICKFILE dialog (see
OPEN_FILE).
FILE_ID -> A named variable will return the file identifier
string (80 characters). This string will be returned
even if the file is of wrong type and no data was read.
TYPE -> A named variable will contain the data type
_EXTRA keywords are passed on to OPEN_FILE
OUTPUTS:
DATA -> an array with NLINES * NVARS values. The type of the
data array depends on the information stored in the file.
VARDESC -> A variable descriptor structure (see GTE_VARDESC)
NVARS -> number of variables in file
NLINES -> number of data lines
SUBROUTINES:
REQUIREMENTS:
Uses OPEN_FILE
NOTES:
See also WRITE_BDT0001
Format specification:
file_ID : 80 byte character string
NVARS, NLINES, NCOMMENTS, TYPE : 4 byte integer (long)
NAMES : NVARS*40 byte character string
UNITS : NVARS*40 byte character string
COMMENTS : NCOMMENTS records of 80 byte length
DATA : 8 byte float (double) array NLINES*NVARS
EXAMPLE:
READ_BDT0001,'~/tmp/*.bdt',data,vardesc,comments=comments
; Will read a file that the user selects with the PICKFILE
; dialog. No information about the actual filename is
; returned.
FILE = '~/tmp/*.bdt'
READ_BDT0001,FILE,data,vardesc,nvars,nlines,file_id=file_id
; Does the same thing, but this time FILE will contain the
; actual filename. The number of variables and lines are
; returned in NVARS and NLINES, the file identifier string
; is returned in file_id
MODIFICATION HISTORY:
mgs, 24 Aug 1998: VERSION 1.00
mgs, 23 Dec 1998: VERSION 1.10:
- DATA now undefined if unsuccessful
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
bmy, 02 Apr 2008: GAMAP VERSION 2.12
- Now read data as big-endian
(See /n/home09/ryantosca/IDL/gamap2/file_io/read_bdt0001.pro)
NAME:
READ_BIN and PLOT_BIN
PURPOSE:
Read a simple binary 2-D file. The file must be F77
unformatted and contain the XDIM and YDIM information
as LONG integers in the first record.
CATEGORY:
File & I/O
CALLING SEQUENCE:
READ_BIN,FILENAME,DATA [,keywords]
PLOT_BIN,DATA [,keywords]
INPUTS:
FILENAME -> Name of the file to read
DATA (for PLOT_BIN) -> The data array as read with READ_BIN
KEYWORD PARAMETERS:
XDIM, YDIM -> return the dimensions of the data set.
_EXTRA -> used to pass extra keywords to OPEN_FILE. Probably
only useful with /SWAP_ENDIAN.
/PLOT -> Call PLOT_BIN directly.
(for PLOT_BIN)
MIN, MAX -> minimum and maximum to be used for conversion of
data to a byte array for display with TVIMAGE
TOP -> top value for BYTSCL
CT -> colortable numebr to use
/MAP -> set this keyword to overlay a map (isotropic cylindrical
projection)
OUTPUTS:
DATA -> The data array returned from READ_BIN
SUBROUTINES:
Uses OPEN_FILE and TVIMAGE
REQUIREMENTS:
None
NOTES:
Rather primitive program but demonstrates the principle use
of binary data files and TVIMAGE.
EXAMPLES:
READ_BIN,'~/mydata/*.bdat', DATA
PLOT_BIN, DATA, MIN=MIN(DATA,MAX=M), MAX=M
; is equivalent to
READ_BIN, '~/mydata/*.bdat', DATA, /PLOT
MODIFICATION HISTORY:
mgs, 15 Jan 1999: VERSION 1.00
mgs, 15 Jun 1999: - added header
- added PLOT keyword and _EXTRA
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
bmy, 02 Apr 2008: GAMAP VERSION 2.12
- Now read data as big-endian
(See /n/home09/ryantosca/IDL/gamap2/file_io/read_bin.pro)
NAME:
READ_H5DATASET
PURPOSE:
Convenience routine to read dataset variables
from Hierarchical Data Format version 5 (HDF5) files.
Also works for HDF-EOS files!
CATEGORY:
File & I/O, Scientific Data Formats
CALLING SEQUENCE:
DATAFIELD = READ_H5DATASET( FID, DATASET_NAME )
INPUTS:
FID -> HDF5 File ID, as returned by routine H5F_OPEN
DATASET_NAME -> Name of the scientific dataset variable that
you want to extract from the file.
KEYWORD PARAMETERS:
None
OUTPUTS:
DATAFIELD -> Array containing extracted data from the HDF file.
SUBROUTINES:
None
REQUIREMENTS:
Need to use a version of IDL w/ HDF5 library installed.
NOTES:
From Trevor Beck (trevor.beck@noaa.gov) for GOME-2.
EXAMPLE:
; Specify the file name
FILE = 'GOME_xxx_1B_M02_20070105012056Z_20070105030556Z_R_O_20080613081807Z.337p4_356p1.brs.hcho.he5'
; Make sure the file is a HDF5 file
IF ( H5F_IS_HDF5( FILE ) eq 0 ) then MESSAGE, 'Not an HDF-5 file!'
; Open the HDF file and get the file ID # (FID)
FID = H5F_OPEN( FILE )
IF ( FID lt 0 ) then MESSAGE, 'Error opening file!'
; Read the AMF field from disk
; NOTE: the swath name is "Column"
AMF = READ_H5DATASET( FID, "/Column/Amf" )
; Close the file
H5_CLOSE, FID
MODIFICATION HISTORY:
bmy, 28 May 2009: VERSION 1.00
(See /n/home09/ryantosca/IDL/gamap2/file_io/read_h5dataset.pro)
NAME:
READ_SONDE
PURPOSE:
Read climatological ozone sonde data as compiled by
Jennifer A. Logan at Harvard University.
If successful, the procedure returns a structure
with all information from a sondeXXX.* file.
Ozone concentrations are automatically converted to ppbv.
The data can be downloaded via ftp from io.harvard.edu
path= pub/exchange/sonde. Please read the README files!
CATEGORY:
Atmospheric Sciences, File & I/O
CALLING SEQUENCE:
READ_SONDE,filename,data
INPUTS:
filename -> Name of the file containing the sonde data.
This parameter can contain wildcards (*,?) in which case
a file selection dialog will be displayed. If filename
is a variable, it will be replaced by the actual filename
that was opened.
KEYWORD PARAMETERS:
MBAR -> return ozone concentrations in mbar rather than ppbv
STATIONS -> Can be used either as input or output for a list
of station codes, locations and names as retrieved with
procedure read_sondestations. STATIONS must be specified
(or is returned) as an array of structures.
OUTPUTS:
DATA -> A structure containing the following fields:
TITLE STRING 'Ozone sonde data'
STATION STRING
STATIONCODE INT
TIMERANGE STRING (See /n/home09/ryantosca/IDL/gamap2/atm_sci/read_sonde.pro)
NAME:
READ_SONDESTATIONS
PURPOSE:
Retrieve station codes and geographical locations for
ozone sonding stations as listed in file station.codes
from Jennifer A. Logan's ozone sonde climatology.
This routine is called from procedure READ_SONDE, and
only needs to be called explicitely if the station.codes
file resides neither in the current directory nor the
directory of the sonde data files.
The procedure will read the file station.codes once then
store the information in a common block for later use.
CATEGORY:
Atmospheric Sciences, File & I/O
CALLING SEQUENCE:
READ_SONDESTATIONS,stations [,filename]
INPUTS:
FILENAME (optional) -> if given, it specifies the path and filename
of the file that is normally called station.codes.
FILENAME may contain wildcards (*,?) in which case a
file selector dialog is displayed.
KEYWORD PARAMETERS:
None
OUTPUTS:
STATIONS -> An array with structures containing the stations
codes (integer), latitude, longitude, altitude (float),
amd name (string).
SUBROUTINES:
Uses OPEN_FILE and EXTRACT_FILENAME (used in OPEN_FILE)
REQUIREMENTS:
None
NOTES:
None
EXAMPLE:
read_sondestations,stations,'station.codes'
; if called for the first time, reads file station.codes
; and returns information for all stations in stations.
; NOTE: In this case, the filename argument could have been
; omitted.
MODIFICATION HISTORY:
mgs, 02 Nov 1998: VERSION 1.00
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
(See /n/home09/ryantosca/IDL/gamap2/atm_sci/read_sondestations.pro)
NAME:
ROUTINE_NAME (function)
PURPOSE:
return the name of the routine which calls this function.
CATEGORY
File & I/O
CALLING SEQUENCE:
rname = ROUTINE_NAME()
INPUTS:
none
KEYWORD PARAMETERS:
FILENAME -> returns the file in which the routine can be found
/CALLER -> returns information about the caller of the routine
instead of the routine itself
OUTPUTS:
The name of the caller routine is returned in lowercase
characters (can be used to construct a filename by adding
".pro")
SUBROUTINES:
None
REQUIREMENTS:
None
NOTES:
None
EXAMPLE:
From the command line:
PRINT, ROUTINE_NAME()
results in $main$
Very useful in conjunction with USAGE.PRO:
USAGE, ROUTINE_NAME()
displays help information on the current routine.
MODIFICATION HISTORY:
mgs, 27 Mar 1998: VERSION 1.00
mgs, 22 Apr 1998: - added FILENAME and CALLER keywords
mgs, 14 Jan 1998: - needed fix for filename when working on PC:
$MAIN$ allows no str_sep
bmy, 07 Aug 2002: TOOLS VERSION 1.51
- Now use routine STRBREAK to split the line
instead of STR_SEP. STR_SEP has been removed
from the IDL distribution in IDL 5.4+.
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
(See /n/home09/ryantosca/IDL/gamap2/file_io/routine_name.pro)
NAME:
SCALEFOSS2BPCH
PURPOSE:
Converts fossil fuel scale factor files from the obsolete
binary format to binary punch format (so that they can be
read by GAMAP).
CATEGORY:
File & I/O, BPCH Format
CALLING SEQUENCE:
SCALEFOSS2BPCH, [ Keywords ]
INPUTS:
None
KEYWORD PARAMETERS:
INFILENAME -> Name of the input file containing fossil
fuel scale factors. If omitted, SCALEFOSS2BPCH
will prompt the user for a filename via a dialog box.
OUTFILENAME -> Name of the binary punch file containing
fossil fuel scale factors. Default is to add a
".bpch" extension to INFILENAME.
OUTPUTS:
None
SUBROUTINES:
External Subroutines Required
==================================================
CTM_TYPE (function) CTM_GRID (function)
NYMD2TAU (function) CTM_MAKE_DATAINFO (function)
CTM_WRITEBPCH EXTRACT_FILENAME (function)
REQUIREMENTS:
None
NOTES:
None
EXAMPLE:
SCALEFOSS2BPCH, INFILENAME='scalefoss.liq.2x25.1998', $
OUTFILENAME='scalefoss.liq.2x25.1998.bpch'
; Converts scalefoss files to BPCH format.
MODIFICATION HISTORY:
bmy, 15 Jan 2003: VERSION 1.00
bmy, 23 Dec 2003: VERSION 1.01
- rewritten for GAMAP v2-01
bmy, 27 Jun 2006: VERSION 1.02
- Use more robust algorithm for getting
the year out of the file name
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
bmy, 02 Apr 2008: GAMAP VERSION 2.12
- Now read input file as big-endian
(See /n/home09/ryantosca/IDL/gamap2/file_io/scalefoss2bpch.pro)
NAME:
TESTPATH
PURPOSE:
Set !PATH variable to a limited set of directories for test
purposes (or restore the initial value)
CATEGORY:
File & I/O
CALLING SEQUENCE:
TESTPATH [,pathstring] [,/restore]
INPUTS:
PATHSTRING -> a string with the test directory name(s)
The !PATH variable will contain this string plus the
standard IDL library search path [which is hardwired
and may have to be changed after future updates].
Note: This parameter is ignored if the /RESTORE keyword
is set.
KEYWORD PARAMETERS:
/RESTORE -> restore initial value of !PATH. This is always
the value before the very first call to testpath! Use
of this keyword overrides the PATHSTRING parameter.
OUTPUTS:
None
SUBROUTINES:
None
REQUIREMENTS:
None
NOTES:
None
EXAMPLE:
testpath,'~mgs/IDL/test3d'
; sets !PATH to the given name + standard IDL libraries
testpath,/restore
; resets !PATH variable to original value
MODIFICATION HISTORY:
mgs, 16 Jun 1998: VERSION 1.00
mgs, 02 Nov 1998: - changed IDL path from idl5 to idl
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
(See /n/home09/ryantosca/IDL/gamap2/file_io/testpath.pro)
NAME:
WRITEDATA
PURPOSE:
write a 2 dimensional data array and header information to a file
CATEGORY:
File & I/O
CALLING SEQUENCE:
WRITEDATA,FILENAME,DATA [,HEADER,UNITS,CFACT,MCODE] [,keywords]
INPUTS:
FILENAME --> the name of the output file. If the output file
exists, the user will be prompted for a new name unless
keyword NO_WARN is set.
DATA --> The data array to be written.
HEADER --> An optional string array containing variable names.
These will be composed to a string using the DELIM delimiter.
Note that the HEADER information can also be passed in the
pre-formatted COMMENTS keyword parameter.
UNITS, CFACT, MCODE --> string arrays that will be added to the
file header concatenated with blank delimiters. These parameters
are optional and merely there to facilitate creating chem1d
model input files.
KEYWORD PARAMETERS:
TITLE --> A title string that will be the first header line.
It is also possible to pass a string array here, although for
more complicate file headers it is recommended to pre-format
the file header and pass it in the COMMENTS keyword.
DELIM --> A delimiter character for the HEADER (variable name)
items. Default is a blank ' '.
COMMENTS --> A string array containing all the lines of the file
header. Note that COMMENTS overrules the input of HEADER, UNITS,
CFACT, and MCODE as well as TITLE.
/NO_WARN --> Suppress warning message and user prompt for a new
filename if the file already exists.
OUTPUTS:
A file containing a file header and the data array written
line by line.
SUBROUTINES:
None
REQUIREMENTS:
None
NOTES:
None
EXAMPLE:
DATA = findgen(3,10)
HEADER = ['A','B','C']
writedata,'test.out',DATA,HEADER,TITLE='test file',DELIM=';'
This will create a file like:
test file
A;B;C
0.00000 1.00000 2.00000
3.00000 4.00000 5.00000
...
MODIFICATION HISTORY:
mgs, 25 Nov 1997: VERSION 1.00
mgs, 05 Apr 1999: - now uses formatted write statement
(looks like a bug in IDL for windows: sometimes no space
is printed between numbers if you simply print,data)
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
(See /n/home09/ryantosca/IDL/gamap2/file_io/writedata.pro)
NAME:
WRITE_BDT0001
PURPOSE:
Write a binary data file with size information and
variable names and units
CATEGORY:
File & I/O
CALLING SEQUENCE:
WRITE_BDT0001,filename,data,vardesc[,keywords]
INPUTS:
FILENAME -> Name of the file to write or a file mask
which will be used in the PICKFILE dialog. If the
filemask is a named variable, it will return the
actual filename.
DATA -> a 2D data array with dimensions LINES,VARIABLES
VARDESC -> A variable descriptor structure array (see
GTE_VARDESC). This array must contain one structure
for each variable and the structure must have the
tags NAME and UNIT. Alternatively, you can use the
NAMES and UNITS keywords to pass string arrays.
KEYWORD PARAMETERS:
NAMES -> A string array containing variable names. This
will only be used if no VARDESC structure array is
given.
UNITS -> A string array with physical units for each
variable. (see NAMES)
COMMENTS -> A string (or string array) with comment lines.
Only the first 80 characters of each line will be stored.
SELECTION -> An index array to select only a subset of
variables from the data set. Indices are truncated
to lie in the range 0..n_elements(names), which can
lead to multiple output of the same variable!
_EXTRA keywords are passed on to OPEN_FILE
Flags to determine the data type:
Default is to store the data in its current type. Use the
TYPE keyword to convert it to any other (numeric) type
or use one of the following keywords. The type information
is saved in the file, so READ_BDT0001 can automatically
read the data regardless of the format.
/BYTE -> convert data to byte format
/INT -> convert data to (2 byte) integer format
/LONG -> convert data to (4 byte) integer format
/FLOAT -> convert data to (4 byte) real format
/DOUBLE -> convert data to (8 byte) double prec. real format
/COMPLEX -> convert data to (8 byte) complex
/DCOMPLEX -> convert data to (16 byte) double complex
OUTPUTS:
None
SUBROUTINES:
Uses OPEN_FILE, STR2BYTE
REQUIREMENTS:
None
NOTES:
Format specification:
file_ID : 80 byte character string
NVARS, NLINES, NCOMMENTS, TYPE : 4 byte integer (long)
NAMES : NVARS*40 byte character string
UNITS : NVARS*40 byte character string
COMMENTS : NCOMMENTS records with 80 characters each
DATA : 8 byte float (double) array NLINES*NVARS
EXAMPLE:
WRITE_BDT0001,'~/tmp/*.bdt',data,vardesc
MODIFICATION HISTORY:
mgs, 24 Aug 1998: VERSION 1.00
mgs, 28 Aug 1998: - changed specs to allow comments
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
bmy, 02 Apr 2008: GAMAP VERSION 2.12
- Now write data as big-endian
(See /n/home09/ryantosca/IDL/gamap2/file_io/write_bdt0001.pro)
NAME:
WRITE_BIN
PURPOSE:
Save a 2-D data array into a binary file together with
its size info. Use read_bin.pro to read it.;
CATEGORY:
File & I/O
CALLING SEQUENCE:
WRITE_BIN, data, filename, _EXTRA=e
INPUTS:
DATA -> Array to save to binary file format.
FILENAME -> Name of the output binary file.
KEYWORD PARAMETERS:
_EXTRA=e -> Passes extra keywords to OPEN_FILE
OUTPUTS:
None
SUBROUTINES:
External Subroutines Required:
===============================
OPEN_FILE
REQUIREMENTS:
None
NOTES:
Use READ_BIN to read the data file.
EXAMPLES:
(1)
WRITE_BIN, DIST(20,20), 'myfile.bin'
; Writes a data array to a binary file.
(2)
WRITE_BIN, DIST(20,20), 'myfile.bin', /SWAP_ENDIAN
; Writes a data array to a binary file
; converts to BIG-ENDIAN (i.e. use this if
; you are running IDL on a PC.)
MODIFICATION HISTORY:
bmy & phs, 13 Jul 2007: GAMAP VERSION 2.10
bmy, 02 Apr 2008: GAMAP VERSION 2.12
- Now write data as big-endian
(See /n/home09/ryantosca/IDL/gamap2/file_io/write_bin.pro)