LIS/OTD 0.5 Degree High Resolution Full Climatology (HRFC)

v2.2 - Dataset size ~14.4Mb

The product is a 0.5 deg x 0.5 deg gridded composite of total (IC+CG) lightning bulk production, expressed as a flash rate density (fl/km²/yr). Climatologies from the 5-yr OTD (4/95-3/00) and 8-yr LIS (01/98-12/05) missions are included, as well as a combined OTD+LIS climatology and supporting base data (flash counts and viewing times). Best-available detection efficiency corrections and instrument cross-normalizations, as of the product generation date (1/28/06), have been applied.

The product is distributed in HDF (Hierarchical Data Format). HDF is accessible from C or Fortran using interfaces provided by NCSA (http://hdf.ncsa.uiuc.edu/tools.html), or through various commercial software packages, such as IDL or Noesys (both from http://www.rsinc.com), or  MATLAB (http://www.mathworks.com). All data in this product are stored in HDF Scientific Data Sets (SDS).

In presentations or publications, users are to acknowledge the dataset as follows:

The v2.2 gridded satellite lightning data were produced by the NASA LIS/OTD Science Team (Principal Investigator, Dr. Hugh J. Christian, NASA / Marshall Space Flight Center) and are available from the Global Hydrology Resource Center (http://ghrc.msfc.nasa.gov).

6. Long Description

• NOTE: Version 2.2 release included new documentation in PDF format, which can be viewed and downloaded here:
  ftp://ghrc.nsstc.nasa.gov/pub/doc/lisotd_gridded_products/lis_otd_gridded_products_documentation_v2.2.pdf
  
  
• Approach

Bulk lightning production, expressed in fl/km²/yr, is calculated as a "counting experiment", i.e., all flashes observed by the sensors in a given grid location are summed for the full period considered (OTD, 5 yr; LIS, 8 yr), then divided by the actual instrument viewing time of the grid location. Flashes are weighted during the summation by the best instantaneous estimate of each instrument’s flash detection efficiency (DE); e.g., if the DE estimate is 50%, a value of 2.0 is added to the flash summation grid. Instrument viewing dropouts due to temporary platform or instrument down-time are considered. Partial grazing of a grid location by the instrument FOV is considered in the LIS estimates, but not considered in the OTD estimates. Best available DE corrections are applied as a function of instrument, local hour, and for the OTD, spatial location (see below). The combined flash rate climatology merges the corrected OTD and LIS climatologies to reduce variance in the tropical estimates.

• Contents

The HDF file contains 12 Scientific Data Set (SDS) grids. These include:

HRFC_COM_FR        Combined Flash Rate [720x360, float, fl/km²/yr]          
HRFC_OTD_RF        OTD Raw Flashes [720x360, float, flashes]
HRFC_OTD_SF        OTD Scaled Flashes [720x360, float, flashes]
HRFC_OTD_VT        OTD Viewtime [720x360, float, km²-sec]
HRFC_OTD_FR        OTD Flash Rate [720x360, float, fl/km²/yr] 
HRFC_LIS_RF          LIS Raw Flashes [720x360, float, flashes]
HRFC_LIS_SF          LIS Scaled Flashes [720x360, float, flashes]
HRFC_LIS_VT          LIS Viewtime [720x360, float, km²-sec]
HRFC_LIS_FR          LIS Flash Rate [720x360, float, fl/km²/yr]
HRFC_OTD_DE       OTD Detection Eff. [144x72x24x3, float, %]
HRFC_LIS_DE         LIS Detection Eff. [144x72x24, float, %]
HRFC_AREA           Grid cell areas [720x360, km²]

Of these, HRFC_COM_FR is the primary data product; the combined LIS/OTD flash rate (FR) climatology. It may derived as: HRFC_COM_FR = (HRFC_OTD_SF + HRFC_LIS_SF) / (HRFC_OTD_VT + HRFC_LIS_VT) (for nonzero total viewing time grid cells). The supporting base grids (raw flash counts RF, detection efficiency-scaled flash counts _SF, and total viewing _VT) are provided only for reference or rebinning. Users are encouraged to view the viewing grids to understand the nature of the Low Earth Orbit satellite sampling, which is a component of the uncertainty at each grid location. The diurnally-applied detection efficiency corrections are also included (_DE; see below). A grid cell area (_AREA) product is also included, to simplify net production calculations (e.g., global annual production in fl/yr = sum(HRFC_COM_FR * HRFC_AREA)).

The dimension scales accompanying the SDS grids represent the center coordinate of each 0.5 degree grid cell (e.g., -179.75, -179.25 … 179.25, 179.75 deg lon, etc.). For unit conversion, a year is assumed to contain 365.25 days.

• Calibration

  Best-available calibrations for instrument detection efficiency (as of 1/28/06) have been applied in the v2.2 product. These consider variations in instrument sensitivity across the angular field of view, variability in the background scene radiance (the instruments are designed to reduce sensitivity as background radiance increases in order to preserve signal-to-noise ratio), actual cross-sensor ground validation, changing sensor thresholds and cross-normalization of climatological results between the two instruments. The procedure, documented in Boccippio et al [2002] and Christian et al [2003] is roughly as follows:

  1.      The diurnal distribution of cloud background radiance (upon which sensitivity depends) is estimated from LIS background scenes concurrent with lightning events [Boccippio et al, JAOT, 2002].

  2.      Laboratory calibration data [Koshak et al, 2000] are used with (1) to estimate the diurnal distributions of minimum detectable pixel radiance.

  3.      A pre-launch Optical Pulse Sensor (OPS) data base of pulse radiance distributions acquired from high altitude (U2) airplane measurements [Christian and Goodman, 1987] are used with (2) to predict the diurnal distribution of maximum possible LIS and OTD flash detection efficiency (following and expanding upon [Koshak et al, 2000]). The OTD prediction is computed for threshold settings used from 10/23/96-4/1/00.

  4.      The LIS nighttime flash detection efficiency is estimated to be 88%, based on ground validation by [Thomas et al, 2000] and [Koshak et al, AGU Fall Meeting, 2000]. The LIS maximum possible flash detection efficiency diurnal distribution is scaled by this value.

  5.      The results of (3) and the uncorrected climatological data yield a LIS:OTD flash detection efficiency ratio of 1.67 [Boccippio et al, JAM, 2000; Boccippio et al, JAOT, 2002]. The OTD maximum possible nighttime flash detection efficiency is set to (1/1.67)*0.88, and the OTD predicted diurnal flash detection efficiency cycle from (3) is scaled accordingly.

  6.      The OTD diurnal flash detection efficiency cycle during other threshold setting windows (04/01/95-06/09/95, 06/09/95-07/20/95, 07/20/95-10/23/96, 10/23/96-04/01/00) is adjusted slightly by relative differences determined from ground validation [Boccippio et al, JAOT, 2000]. Three grids are reported in the file; these date windows use grids (0,2,1,0) respectively.

  7.      The OTD flash detection efficiency over South America is further scaled using the climatological ratio of LIS to OTD detections. OTD flash detection efficiency in this region drops within the South Atlantic Anomaly as adaptive software noise filters are more active to remove radiation noise events in the CCD pixel array. The peak reduction in DE from this effect is estimated to be an additional 50%, centered over Sao Paulo, Brazil.

  Detection efficiency adjustments are thus prescribed as a function of (sensor, longitude, latitude, local hour, date in mission [OTD only]). These are included, for reference, in the HRFC_OTD_DE and HRFC_LIS_DE Scientific Data Sets. The bulk effect of the instantaneous corrections can be estimated as: BULK_OTD_DE = sum(HRFC_OTD_RF) / sum(HRFC_OTD_SF) (~ 47%) or BULK_LIS_DE = sum(HRFC_LIS_RF) / sum(BULK_LIS_SF) (~ 82%).

  Users wishing to cite this calibration procedure may use or modify the following:

  Observations in the LIS/OTD v2.2 reanalysis have been corrected by the LIS Science Team by estimated flash detection efficiency, applied as a function of sensor, local hour, date of mission, and (for the OTD) geographic location. For the entire dataset, these corrections correspond to average flash detection efficiencies of 47% (OTD) and 82% (LIS). The adjustments derive from a combination of laboratory calibration, ground-validation, and cross-normalization between the two instruments. Uncertainty in these corrections is estimated as +/- 10%. The calibration procedure is described in the dataset documentation.

7. Uncertainty

Consistency between modeled and ground-validated detection efficiency suggests that the applied values are known within about +/- 10%. This is thus the minimum uncertainty in the gridded data, arising from possible bias in the correction. Local variance arises from undersampling of a given grid location, and its relative impact increases as the actual local climatological flash rate decreases. This effect is likely dominant in the HRFC, but less important in the LRFC. A final bias component may arise from OTD sampling in the 3^rd-5^th years of its mission; problems with its host platform caused orbits viewing local times near local noon and midnight to be preferentially lost. This bias in diurnal viewing may yield a small bias in the bulk flash production estimate.

All three primary error sources (measurement uncertainty, sampling-related variance and sampling-related bias) are currently being quantified.

All OTD and LIS orbits undergo both automated and manual quality assurance. For the preliminary reanalysis, the most stringent orbit rejection criterion was applied: any orbit which was assigned a manual Q/A "warning" flag has been rejected from the reanalysis. In subsequent versions, these rejected orbits may be examined individually and, if acceptable, added into the climatology to improve sampling.

Each OTD flash is further assigned an automated quality index (the ‘Thunderstorm Area Count’, or ‘Density Index’), indicating its likelihood of being lightning rather than optical or radiation noise. For this reanalysis, only flashes with values of the metric >= 140 have been included. This is the same cutoff value used in all validation and science analysis published by the LIS science team to date. This filter removes most radiation noise from the climatology; a slight residual ‘ring’ artefact of very low spurious flash rates remains at the periphery of the South Atlantic Anomaly (southeast Pacific and southeast Atlantic).

There are no restrictions on the use of these data.

The High Resolution gridded dataset is being provided to illustrate "high contrast" lightning production regions. It is 25x less sampled than the LIS/OTD 2.5 Degree Low Resolution Full Climatology (LRFC), and hence quantitative values from individual grid locations should be used with extreme caution. Rebinning of the HRFC to custom lower resolutions (e.g., 1.0 deg, 5.0 deg) should be performed using the included scaled flashes (SF) and viewtimes (VT) grids, rather than the flash rate (RF) grid directly. E.g. to make a 5 deg x 5 degree climatology:

• Rebin HRFC_OTD_SF to 5x5 (i.e., accumulate high-res grid cells to low res grid cells)
• Rebin HRFC_LIS_SF to 5x5
• Rebin HRFC_OTD_VT to 5x5
• Rebin HRFC_LIS_VT to 5x5
• Compute the flash rate from the new grids: HRFC_COM_FR = (HRFC_OTD_SF + HRFC_LIS_SF) / (HRFC_OTD_VT + HRFC_LIS_VT) . (Perform the calculation only on grid cells that have combined viewtime > 0).

All subsequent versions of this dataset (both minor and major revisions) will maintain the same product definition, resolution and file format. Additional years of data from the LIS mission may be added into the climatology for future minor versions, but will not alter file format. Any actual definitional changes would be reflected by the creation of a new product.

Subsequent major version updates will only occur if/when the dataset is recomputed using new major version OTD or LIS orbit data. E.g:

Status OTD Orbit Version LIS Orbit Version HRFC Version

Preliminary                    1.1 4.0 0.x

Peer-reviewed               1.1 4.0 1.x
Subsequent 2 or higher
OR 5 or higher                  … 2.x, 3.x,

Current version is at v2.2, and covers years 1995-2005.

11.Version History

v0.1: 8/1/01

Preliminary Release; 5 yr. OTD (v1.1), 3 yr LIS (v4.0) orbit data

v1.0: 4/1/03   

First Official Release; 5 yr. OTD (v1.1), 5 yr LIS (v4.0) orbit data

v2.2: 9/1/06   

Second Official Release; 5 yr. OTD (v1.1), 8 yr LIS (v4.0) orbit data

 

Minor OTD calibration errors during 1995-1996 fixed

Calibration procedure now peer-reviewed

Instrument and Calibration/Validation:

Boccippio, DJ; Koshak, WJ; Blakeslee, RJ (2002): Performance assessment of the Optical Transient Detector and Lightning Imaging Sensor: I. Predicted diurnal variability. J. Atmos. Oc. Tech. 19, 1318-1332.

Thomas,RJ; Krehbiel,PR; Rison,W; Hamlin,T; Boccippio,DJ; Goodman,SJ; Christian,HJ (2000): Comparison of ground-based 3-dimensional lightning mapping observations with satellite-based LIS observations in Oklahoma. Geophys. Res. Lett. 27, 1703-1706.

Boccippio,DJ; Driscoll,KT; Koshak,WJ; Blakeslee,RJ; Boeck,WL; Mach,DA; Buechler,DE; Christian,HJ; Goodman,SJ (2000): The Optical Transient Detector (OTD): Instrument characteristics and cross-sensor validation. J. Atmos. Oc. Tech. 17, 441-458.

Koshak,WJ; Bergstrom,JW; Stewart,MF; Christian,HJ; Hall,JM; Solakiewicz,RJ (2000): Laboratory calibration of the Optical Transient Detector and Lightning Imaging Sensor. J. Atmos. Oc. Tech. 17, 905-915.

Christian,HJ; Blakeslee,RJ; Goodman,SJ; Mach,DM (eds.) (2000): Algorithm Theoretical Basis Document (ATBD) for the Lightning Imaging Sensor (LIS). Posted: 1 Feb 2000. (NASA / Marshall Space Flight Center, AL 35812)

Ushio,T; Driscoll,KT; Heckman,S; Boccippio,DJ; Koshak,WJ; Christian,HJ (1999): Initial comparison of the Lightning Imaging Sensor (LIS) with Lightning Detection and Ranging (LDAR). Proc. 11th Intl. Conf. on Atmospheric Electricity (ICAE), Guntersville, AL, 6-11 June. 738-741.

Christian,HJ; Blakeslee,RJ; Goodman,SJ; Mach,DA; Stewart,MF; Buechler,DE; Koshak,WJ; Hall,JM; Boeck,WL; Driscoll,KT; Boccippio,DJ (1999): The Lightning Imaging Sensor. Proc. 11th Intl. Conf. on Atmospheric Electricity (NASA), Guntersville, AL, 7-11 June. 746-749.

Kummerow, C; Barnes, W; Kozu, T; Shiue, J; Simpson, J (1998): The Tropical Rainfall Measuring Mission (TRMM) sensor package. J. Atmos. Oc. Tech. 15, 809-817.

Christian,HJ; Driscoll,KT; Goodman,SJ; Blakeslee,RJ; Mach,DA; Buechler,DE (1996): The Optical Transient Detector (OTD). Proc. 10th International Conference on Atmospheric Electricity, Osaka, Japan.

Christian,HJ; Blakeslee,RJ; Goodman,SJ (1992): Lightning imaging sensor for the Earth Observing System. (TM-4350.) NASA. 44 pages. Available from Center for Aerospace Information, P.O. Box 8757, Baltimore Washington International Airport, Baltimore, MD 21240.

Christian,HJ; Blakeslee,RJ; Goodman,SJ (1989): The Detection of Lightning from Geostationary Orbit. J. Geophys. Res. 94, 13329-13337.

Goodman,SJ; Christian,HJ; Rust,WD (1988): A comparison of the optical pulse characteristics of intracloud and cloud-to-ground lightning as observed above clouds. J. Appl. Met. 27, 1369-1381.

LIS/OTD-Enabled Science & Applications: 

Christian, HJ et al (2003): Global frequency and distribution of lightning as observed by the Optical Transient Detector. J. Geophys. Res, 108 4005, doi: 10.1029/2002JD002347.

Christian, HJ et al (2003): Global frequency and distribution of lightning as observed by the Optical Transient Detector. J. Geophys. Res, 108 4005, doi: 10.1029/2002JD002347.

Koike, M.; Kondo, Y.; Akutagawa, D.; Kita, K.; Nishi, N.; Liu, S.C.; Blake, D.; Kawakami, S.; Takegawa, N.; Ko, M.; Zhao, Y.; Ogawa, T. (2003) Reactive nitrogen over the tropical Western Pacific: Influence from lightning and biomass burning. J. Geophys. Res, 108, 8403, doi:10.1029/2001JD00823.

Boccippio,DJ (2002): Lightning scaling laws revisited. J. Atmos. Sci. 59, 1086-1104.

Bond, DW; Zhang, R; Tie, X.; Brasseur, G.; Huffines, G; Orville, R.E; Boccippio, DJ (2001): NO_x production by lightning over the continental United States. J. Geophys. Res, 106, 27701-27710.

Ushio,T; Heckman,S; Boccippio,DJ; Christian,HJ (2001): A survey of thunderstorm flash rates compared to cloud top height using TRMM satellite data. J. Geophys. Res., D, 106, 24089-24095.

Goodman, S. J. and D. J. Cecil, 2001.  Structure and Characteristics of Precipitation Systems Observed By TRMM, Preprints, 11^th Conference on Satellite Meteorology and Oceanography, 15-18 October, 2001, Madison, WI, Amer. Meteor. Soc., Boston, 464-467.

Chang, DE;Weinman, JA; Morales, CA; Olson, WS (2001): The effect of spaceborne microwave and ground-based continuous lightning measurements on forecasts of the 1998 Groundhog Day storm. Mon. Wea. Rev., 129, 1809-1833.

Boccippio, DJ; Heckman, S; Goodman, SJ (2001): A diagnostic analysis of the Kennedy Space Center LDAR network. 2: Cross-sensor studies. J. Geophys. Res. 106, 4787-4796.

Rodgers, E; Olson, W; Halverson, J; Simpson, J; Pierce, H (2000): Environmental forcing of supertyphoon Paka’s (1997) latent heat structure. J. Appl. Met., 39, 1983-2006.

Nesbitt,SW; Zipser,EJ; Cecil,DJ (2000): A census of precipitation features in the tropics using TRMM: Radar, ice scattering and lightning observations. J. Clim. 13, 4087-4106.

Williams,ER; Rothkin,K; Stevenson,D; Boccippio,DJ (2000): Global lightning variations caused by changes in thunderstorm flash rate and by changes in the number of thunderstorms. J. Appl. Met. 39, 2223-2230.

Boccippio,DJ; Goodman,SJ; Heckman,S (2000): Regional differences in tropical lightning distributions. J. Appl. Met. 39, 2231-2248

Jeker,DP; Pfister,L; Thompson,AM; Brunner,D; Boccippio,DJ; Pickering,KE; Wernli,H; Kondo,Y; Staehelin,J (2000): Measurements of nitrogen oxides at the tropopause: Attribution to convection and correlation with lightning. J. Geophys. Res., D 105, 3679-3700.

Goodman,SJ; Buechler,DE; Knupp,K; Driscoll,KT; McCaul,EW (2000): The 1997-98 El Nino event and related wintertime lightning variations in the Southeastern United States. Geophys. Res. Lett. 27, 541-544.

Buechler,DE; Driscoll,KT; Goodman,SJ; Christian,HJ (2000): Lightning activity within a tornadic thunderstorm observed by the Optical Transient Detector (OTD). Geophys. Res. Lett ., 27, 2253-2256.

Cecil, DJ; Zipser, EJ (2000): Relationships between tropical cyclone intensity and satellite-based indicators of inner core convection: 85 GHz ice-scattering signature and lightning. Mon. Wea. Rev., 111, 979-996.

Driscoll,KT (1999): A comparison between lightning activity and passive microwave measurements. 11th International Conf. on Atmospheric Electricity (ICAE), Guntersville, AL, 7-11 June 1999. 523-526.

Christian,HJ; Blakeslee,RJ; Boccippio,DJ; Boeck,WL; Buechler,DE; Driscoll,KT; Goodman,SJ; Hall,JM; Koshak,WJ; Mach,DM; Stewart,MF (1999): Global frequency and distribution of lightning as observed by the Optical Transient Detector (OTD). Proc. 11th Intl. Conf. on Atmospheric Electricity (ICAE), Guntersville, AL, 7-11 June. 726-729.

Boccippio,DJ; Wong,C; Williams,ER; Boldi,R; Christian,HJ; Goodman,SJ (1998): Global validation of single-station Schumann resonance lightning location. J. Atmos. Sp. Terr. Phys. 60, 701-712.

A set of Interactive Data Language (IDL) routines to extract and process the LIS/OTD High Resolution Full Climatology (HRFC), Low Resolution Full Climatology (LRFC), Low Resolution Diurnal Climatology (LRDC) and Low Resolution Annual Climatology (LRAC) are distributed with the data. The IDL syntax is roughly similar to C or FORTRAN and, with the exception of animation code, porting of these routines should be fairly straightforward. Full documentation can be found in the program files themselves; a summary is below.

NOTE: These routines are being provided as a courtesy to the user community. The GHRC and LIS Science Team cannot guarantee technical support or compatibility with IDL version updates or platform-specific implementations.

GETGRID.PRO:

GETGRID,HDF_NAME,SDS_NAME,GRID,[DIMS,DIMNAMES,DIM0…]

Retrieves a scientific data set (and optionally, its dimensions) from one of the HDF climatology files.

LONLAT_TO_XY.PRO:

Result = LONLAT_TO_XY([LON,LAT],RESOLUTION)

Converts a [lon,lat] pair to a grid [x,y] index, for a given grid resolution (0.5 or 2.5 deg).

Result = XY_TO_LONLAT([X,Y],RESOLUTION)

Converts an [x,y] index pair to a grid cell center [lon,lat], for a given grid resolution (0.5 or 2.5 deg)

SMOOTHING_ROUTINES.PRO:

Result = HRFC_XY_SMOOTH(N_GRIDCELLS,SENSOR_NAME)

Performs a spatial moving average on the 0.5 deg High Resolution Full  Climatology with size N_GRIDCELLS. This, e.g., can help smooth the values while preserving high-contrast information (as compared with  the Low Resolution Full Climatology). The routine loads the needed grids directly from the HDF file.

Result = LRAC_TIME_SMOOTH(N_DAYS,SENSOR_NAME)

Performs a temporal moving average with an N_DAYS window on the Low Resolution Annual Climatology. Strongly recommended for all use  of this climatology; with N_DAYS prescribed in multiples of 55 days.  The routine loads the needed grids directly from the HDF file.

Result = LRAC_XY_SMOOTH(OLDGRID,N_GRIDCELLS)

Performs a spatial moving average on the 2.5 deg results of LRAC_TIME_SMOOTH, to improve sampling.

MAKE_JGR.PRO:

Recomputes the statistics and graphics used in H.J. Christian et al, "Global frequency and distribution of lightning as observed by the Optical Transient Detector", submitted to J. Geophys. Res., to illustrate statistical processing of the climatological datasets.

MORE_EXAMPLES.PRO:

THREE_CLIMS

Renders three versions of the Full Climatology: the 0.5 degree HRFC, the HRFC smoothed with a 2.5 deg moving average, and the 2.5 degree LRFC.

ANIMATE_ANNUAL,XSIZE,YSIZE

Uses the IDL routine XINTERANIMATE to animate the combined climatology annual cycle, and optionally export the movie to MPEG. The movie will have pixel dimensions XSIZE and YSIZE.

ANIMATE_DIURNAL,XSIZE,YSIZE

Uses the IDL routine XINTERANIMATE to animate the combined climatology local hour diurnal cycle, and optionally export the movie to MPEG. The movie will have pixel dimensions XSIZE and YSIZE.

Questions regarding the science data processing, including gridding, viewing, calibration and variance should be addressed to Dennis.Boccippio@nasa.gov. Questions regarding the OTD or LIS missions themselves should be addressed to the OTD/LIS Principal Investigator, Hugh.Christian@nasa.gov.

Data can be ordered and questions addressed at http://ghrc.nsstc.nasa.gov/.

To order this data or for further information, please contact: Global Hydrology Resource Center User Services 320 Sparkman Drive Huntsville, AL 35805 Phone: 256-961-7932 E-mail: ghrc@eos.nasa.gov

---

NASA Information Contact: Michael Goodman, Global Hydrology and Climate Center GHRC Web Curator: GHRC Web Team Last update: Friday, 22-Sep-2006 13:24:51 CDT If you have trouble viewing or navigating this page, please contact GHRC User Services. U.S. Government Compliance report.