NASA - National Aeronautics and Space Administration
ABOUT CALIPSO PRODUCTS OUTREACH DOCUMENTS RESOURCES TOOLS CONTACTS        

CALIPSO Data User’s Guide - Lidar Level 1B v4.x Data Quality Statement


CALIPSO HOMERESOURCESUser’s Guide → Lidar Level 1 v4.10 Data Quality Statement

Summary Statement for the release of the CALIPSO Version 4.10 Lidar Level 1B Data Products, November 8, 2016

On 8 November 2016 the CALIPSO project began the release of version 4.10 of the CALIPSO lidar level 1B data products. The CALIPSO version 4.10 (V4.10) lidar level 1B data products are produced using the same greatly improved calibration algorithms that were first incorporated in the version 4.00 (V4.00) release in March 2014 (see below). The improvements in V4.10 release are in the ancillary data used for surface detection and calibration. The GTOPO30 Digital Elevation Model (DEM) used in V4.00 has been replaced by a substantially more accurate DEM developed by the CloudSat project. The Global Modeling and Assimilation Office (GMAO) Forward Processing for Instrument Team (FP-IT) meteorological forecast product has been replaced with the higher quality Modern Era Retrospective-Analysis for Research (MERRA-2) reanalysis product These changes to the V4.10 lidar level 1B products are described in more detail in the sections below.

Updated Digital Elevation Map (DEM)

Surface elevation has been referenced to the GTOPO30 DEM since beginning of the CALIPSO mission. Many inaccuracies with surface elevation have been identified, especially over Greenland and Antarctica and in remote mountainous areas. To address these differences, and to complement improvements to the surface detection techniques in the new level 2 products, GTOPO30 has been replaced with a new DEM developed by the CloudSat team. Use of the CloudSat DEM will also allow both missions to have a common surface reference. Pre-release analysis has shown a marked improvement in CALIPSO level 2 surface detection accuracy as a result of using the new DEM.

Inclusion of MERRA-2 Data

Increased accuracy and reliability of CALIOP calibration is achieved by replacing the Global Modeling and Assimilation Office (GMAO) Forward Processing for Instrument Teams (FP-IT) global meteorological product with the Modern Era Retrospective-Analysis for Research (MERRA-2) product. MERRA-2 assimilates temperature measurements from the Microwave Limb Sensor (MLS) on Aura, whereas the FP-IT distribution does not. The corrections provided by the MLS assimilation yield improved calibration coefficients, particularly in the polar regions. Analysis between current GMAO and MERRA-2 showed very little difference in ozone number densities, but measureable differences in molecular densities. The differences in molecular densities have been observed not to exceed 5%, but are focused particularly in high southern latitudes, and reflect changes in the temperatures reported in MERRA-2 relative to FPIT.

Summary Statement for the release of the CALIPSO Version 4.00 Lidar Level 1B Data Products, March 2014

On 31 March 2014 the CALIPSO project began the release of version 4.00 of the CALIPSO lidar level 1B data products. The version 4 lidar level 1 data products are currently being generated in reprocessing mode only, and are not yet accompanied by corresponding level 2 lidar data products. For the foreseeable future, a full suite of all CALIPSO version 3.3 data products (i.e., lidar, WFC and IIR) will continue to be produced in forward processing mode, with the version 3.3 level 2 products using version 3.3 level 1 products as inputs The CALIPSO version 4.00 lidar level 1B data products are generated using substantially upgraded analysis software that incorporates improved calibration algorithms for both 532 nm and 1064 nm during both nighttime and daytime data acquisition periods. These and other changes to the V4 lidar level 1B products are described in the sections below.

532 nm Nighttime Calibration

Significant changes were made in the CALIOP 532 nm nighttime calibration algorithm used in version 4.00. In the previous versions, this calibration was carried out by molecular normalization in the altitude range of 30-34 km, assuming the near-absence of aerosols at these altitudes (Powell et al., 2009). However, it has now been demonstrated that the aerosol loading in this region is larger than pre-launch expectations, particularly in the tropics (Vernier et al., 2009). Because the version 3 calibration made no allowance for these aerosols, biases (i.e., overestimates) were introduced into the calibration coefficients for the 532 nm nighttime data and these biases subsequently propagated into the 532 nm daytime and 1064 nm day and nighttime data. In version 4, the molecular normalization is applied at 36-39 km, which is near the upper limit of the CALIOP measurements. Independent data from SAGE II and GOMOS indicate this region is almost entirely free of particulates. To counter the loss of signal-to-noise ratio incurred by moving to the higher calibration altitude, the version 4 calibration algorithm now averages data over multiple nighttime granules (i.e., orbit segments). The relative uncertainties of the calibration coefficients are on the order of 1-2% overall, with the highest values occurring over the South Atlantic Anomaly and the polar regions. Owing to a coding error discovered in the version 3 calibration software, the 532 nm nighttime calibration uncertainties reported in the version 3 data products underestimated the actual uncertainties by a factor of ~3. This bug has been eliminated in the version 4 software.

The revised calibration approach produces a decrease of the zonal mean calibration coefficients by ~3% overall as compared to the version 3.x values. This in turn leads to an increase in the total attenuated backscatter coefficients by the same amount. The attenuated scattering ratios (SR) in the 30-34 km region are now increased by up to 5% over previous versions, indicating seasonal variations which are consistent with the predictions of Vernier et al. (2009). Stratospheric aerosol loading at these altitudes is now clearly captured by CALIOP measurements, showing up as spatial structures which are consistent with stratospheric dynamics. Similarly, the clear-air scattering ratios at 8-12 km which showed an anomalous dip (SR < 1) in the tropics in the version 3.x data products now no longer do so. Version 3 showed extended areas in the polar regions with SR < 1. Improved data filtering strategies in the version 4 calibration region have now largely removed these anomalies found in the previous versions.

Figure 1 maps.

Figure 1: monthly mean 532 nm calibration coefficients for October 2010 nighttime measurements. Calibration coefficients computed for the version 3.01 data release are shown at left. Calibration coefficients computed in pre-release testing of the version 4.0 algorithm are shown at right.

Figure 1 shows nighttime monthly mean 532 nm calibration coefficients for October 2010 computed for version 3.01 (left) and for a pre-release test of the version 4.00 algorithm (right). In the version 3.01 data, the presence of the stratospheric aerosol layer in the tropics is clearly visible as a region of high calibration coefficients located between the equator and 20° N. The South Atlantic Anomaly (SAA) is also visible as an oval of reduced calibration coefficients centered at ~20° S and ~40° W. By contrast, neither of these geophysical features appears in the version 4.00 data. In both images, the diminution of the calibration coefficients in the southern hemisphere, as the satellite approaches the night-to-day terminator, is indicative of changes in the on-board thermal environment that perturb the alignment between the laser transmitter and receiver. The median reduction in the calibration coefficients from version 3.01 to version 4.00 is ~3%.

References:

Powell, K. A. et al., 2009: “CALIPSO Lidar Calibration Algorithms: Part I - Nighttime 532 nm Parallel Channel and 532 nm Perpendicular Channel”, J. Atmos. Oceanic Technol., 26, 2015-2033, doi:10.1175/2009JTECHA1242.1.

Vernier, J.-P. et al., 2009: “Tropical stratospheric aerosol layer from CALIPSO lidar observations”, J. Geophys. Res., 114, D00H10, doi:10.1029/2009JD011946.

532 nm Daytime Calibration

The 532 nm daytime calibration is obtained by referencing the uncalibrated daytime signal to the calibrated nighttime signal in spatially matched regions where the diurnal variability in the background aerosol loading is assumed to be non-existent (Powell et al. 2010). This night-to-day calibration transfer region has been raised to a higher altitude in version 4 to avoid clouds and increase the number of samples used in calculating daytime calibration coefficients. With version 4 the transfer region is still 4 km deep, as with previous versions, but it now follows on top of the 400 K isentropic surface which is always above the tropopause and above the meteorologically active part of the atmosphere (Hoskins 1991), thus better satisfying the assumption that there is no diurnal variability in the transfer region. Version 4 daytime 532 nm calibration coefficients in the troposphere are ~5% smaller compared to those in version 3 due to a commensurate reduction in nighttime calibration coefficients. Subsequently, attenuated scattering ratios (or equivalently attenuated backscatter coefficients) are now higher in the troposphere by a few percent with largest increases of ~5% near the poles (Figure 2c).

Figure 2 maps.

Figure 2: Daytime clear-air attenuated scattering ratios for July 2010 in (a) version 3 and (b) version 4. Panel (c) shows the percent change in daytime clear-air attenuated scattering ratios from version 3 to version 4 (red indicates an increase). The South Atlantic Anomaly has been excluded.

A new correction has also been added to the level 1 processing of the two 532 nm channels in the daytime to increase the accuracy of measurements at high altitudes where signal levels are low. While the correction is small, its relative effect is significant at high altitudes where the backscatter signal is also small. Consequently, the correction has reduced the overestimate of scattering ratios at high altitudes in version 3 (Figure 2a), making the average attenuated scattering ratio in this region equal unity in version 4 as expected (Figure 2b).

In the version 3 release, the agreement between day and night clear-air attenuated scattering ratios at high altitudes was poor and varied with latitude, possibly due to diurnal variability in aerosol loading and clouds reducing the potential number of clear-air samples in the 8-12 km version 3 calibration transfer region (Figure 3a). Version 4 shows a marked improvement in the agreement between day and night clear-air attenuated scattering ratios above the version 4 calibration transfer region (to within 1 ± 3%), demonstrating (1) the validity of assuming that there is no diurnal variability in stratospheric aerosol loading at these altitudes, (2) that the day/night agreement of clear-air attenuated scattering ratios holds even above the night to day calibration transfer region where they are forced to agree, and (3) the version 4 calibration procedure adjusts for differences in day and night instrument behavior.

Figure 3 maps.

Figure 3: Ratio of day to night clear-air attenuated scattering ratios R’ at 24 - 30 km in (a) version 3 and (b) version 4 for January, April, July and October 2010. The South Atlantic Anomaly has been excluded.

The coarse estimates of daytime calibration uncertainties provided in version 3 have been replaced in version 4 with improved estimates derived using rigorous error propagation. Random uncertainties in the daytime calibration coefficients are less than 1% at all latitudes except near the poles, and are typically smaller than the corresponding nighttime calibration uncertainties because many more samples are averaged for the daytime calibrations. The version 4 calibration uncertainty estimates are continuous across the day-night terminators, and the daytime calibration uncertainties increase near the poles in version 4 to merge smoothly with their nighttime counterparts.

Figure 4 maps.

Figure 4: monthly mean 532 nm calibration coefficients for October 2010 daytime measurements. Calibration coefficients computed for the version 3.01 data release are shown at left. Calibration coefficients computed in pre-release testing of the version 4.0 algorithm are shown at right.

Figure 4 shows daytime monthly mean 532 nm calibration coefficients for October 2010 computed for version 3.01 (left) and for a pre-release test of the version 4.00 algorithm (right). The median reduction in the calibration coefficients from version 3.01 to version 4.00 is ~6%.

References:

Hoskins, B. J., 1991: “Towards a PV-Θ view of the general circulation”, Tellus, 43AB, 27-35, doi:10.1034/j.1600-0889.1991.t01-3-00005.x.

Powell, K. A. et al., 2010: “The CALIOP 532-nm Channel Daytime Calibration: Version 3 Algorithm”, Proceedings of the 25th International Laser Radar Conference, 5-9 July 2010, St.-Petersburg, Russia, 1367-1370, ISBN 978-5-94458-109-9.

1064 nm Daytime and Nighttime Calibration

Version 4 includes substantially redesigned algorithms for the 1064 nm daytime and nighttime calibrations (Vaughan et al., 2012). The CALIOP 1064 channel does not have sufficient molecular backscatter to allow referencing to clear air at high altitudes, so instead a scale factor is computed to match the 1064 nm signals from cirrus clouds to the corresponding 532 nm signals, with the assumption that the 1064/532 backscatter ratio from cirrus should be close to 1. The 1064 nm calibration coefficient is then the product of this 1064 nm scale factor and the 532 nm calibration coefficient. In Version 4, these 1064 nm scale factors are calculated and applied as a function of the granule elapsed time, to better represent the relative variations of the 532 and 1064 signals along orbit segments. The criteria for selecting the cirrus clouds that are used in the scale factor calculation have also been changed significantly in Version 4 (Vaughan et al., 2012). The new criteria provide a larger number of cloud samples in the southern latitudes, and remove water clouds, many polar stratospheric clouds and very thin cirrus from the reference ensemble. Accuracy of the 1064 nm calibrated attenuated backscatter is thus much improved over CALIOP Version 3 by minimizing latitudinal variation caused by measurement artifacts, and by selecting reference targets that can more reasonably be expected to have a 1064/532 signal ratio that is close to unity.

Figure 5 illustrates the changes in the 1064 nm scale factor between version 3 (left) and version 4 (right) for both nighttime data (top row) and daytime data (bottom row) for the month of October 2010. The version 3 scale factors show a large discontinuity between the very high nighttime values and the very low daytime values, along with abrupt changes in value between adjacent orbit segments. The version 4 scale factors, on the other hand, vary smoothly as a function of orbit elapsed time, with continuous values at both terminators. The magnitudes of the changes from V3 to V4 in the nighttime mean scale factors range between 20% lower (northern hemisphere) to 7% higher (southern hemisphere), with the median nighttime difference being 6% lower. The daytime mean scale factors range between 14% lower (northern hemisphere) to 19% higher (southern hemisphere), with the median daytime difference being 3% lower.

Figure 5 maps.

Figure 5: monthly mean 1064 nm calibration scale factors for October 2010. Version 3 scale factors are shown on the left; version 4 scale factors are shown on the right. The upper panels show nighttime data and the lower panels show daytime data.

Figure 6 illustrates the changes in the 1064 nm calibration coefficients between version 3 (left) and version 4 (right) for both nighttime data (top row) and daytime data (bottom row) for the month of October 2010. The data in the top row of Figure 6 represents the 532 nm nighttime calibration coefficients shown in Figure 1 multiplied by the nighttime 1064 nm calibration scale factors shown in the top row of Figure 5. Similarly, the data in the bottom row of Figure 6 represents the 532 nm daytime calibration coefficients shown in Figure 4 multiplied by the daytime 1064 nm calibration scale factors shown in the bottom row of Figure 5. The version 3 1064 nm calibration coefficients exhibit spatial artifacts and irregularities that arise from both the version 3 532 nm calibration coefficients and from the version 3 1064 nm scale factors. In contrast to the version 3 data, the version 4 1064 nm calibration coefficients produced by the new calibration and scale factor algorithms are seen to vary smoothly both latitudinally and longitudinally. The magnitudes of the changes from V3 to V4 in the nighttime mean 1064 nm calibration coefficients range between 22% lower (northern hemisphere) to 8% higher (southern hemisphere), with the median nighttime difference being 8% lower. The daytime mean 1064 nm calibration coefficients range between 20% lower (northern hemisphere) to 10% higher (southern hemisphere), with the median daytime difference again being 8% lower.

Figure 6 maps.

Figure 6: monthly mean 1064 nm calibration coefficients for October 2010. The 1064 nm calibration coefficients are the product of the 532 nm calibration coefficients and the 1064 nm scale factors. Version 3 calibration coefficients are shown on the left; version 4 calibration coefficients are shown on the right. The upper panels show nighttime data and the lower panels show daytime data.

References:

Vaughan, M. A. et al, 2012: “Chaos, Consternation and CALIPSO Calibration: New Strategies for Calibrating the CALIOP 1064 nm Channel”, in Reviewed and Revised Papers Presented at the 26th International Laser Radar Conference (ILRC 2012), 25-29 June 2012, Porto Heli, Greece, 655-658.

QC Flag 2

The rules for setting the bits in QC Flag_2 have been revised to provide more accurate indications of bad data. As of version 4.00,

  • a value of 0 in any bit indicates good data
  • a value of 1 in bits 11-16 always indicates bad data
  • for all other bits, a value of 1 indicates data values outside the normally expected range, which may or may not indicate bad data
    • Occasional randomly occurring values of 1 are generally due radiation-induced noise spikes, and do not indicate bad data. These will be most frequent when passing through the South Atlantic Anomaly.
    • Except as noted below, a value of 1 that persists for several minutes is probably an indication of bad data, and should be investigated before using the data.
    • A value of 1 in bits 20-22 may indicate the presence of Polar Stratospheric Clouds. These occur only in the polar winter, and may persist for more than 10 minutes. They do not indicate bad data.

Additional documentation for QC_Flag_2 can be found in the CALIPSO Data User’s Guide.

New Parameters

Nine new parameters describing the properties of the surface return were added to the version 4 lidar level 1B data product:

  • Surface_Saturation_Flag_532Par
  • Surface_Saturation_Index_532Par
  • Negative_Signal_Anomaly_Index_532Par
  • Surface_Saturation_Flag_532Perp
  • Surface_Saturation_Index_532Perp
  • Negative_Signal_Anomaly_Index_532Perp
  • Surface_Saturation_Flag_1064
  • Surface_Saturation_Index_1064

Surface_Saturation_Flag_xxxx

The version 4 level 1 analysis includes a new algorithm to estimate the likelihood that the signal return from the planetary surface is saturated. This information is reported separately for each of the three measurement channels using an 8-bit integer whose value indicates the likelihood that the surface backscatter signal is saturated. Flag values are as follows:

0 = Not_Saturated
1 = Possibly_Saturated
2 = Certainly_Saturated

Surface_Saturation_Index_xxxx

This SDS is a 16-bit integer used to identify the altitude array index of the maximum signal value for cases where Surface_Saturation_Flag_xxxx is either Possibly_Saturated or Certainly_Saturated. For cases where the Surface_Saturation_Flag_xxxx is Not_Saturated, the Surface_Saturation_Index_xxxx is set to -1. Otherwise, values will lie in the range between 277 (at ~8.2 km) and 577 (at -0.5 km).

Level 1B data users who use the surface signal to derive optical depths are advised that the values recorded in the lidar level 1B attenuated backscatter data may substantially under-represent the true signal values whenever the Surface_Saturation_Flag_xxxx is either Possibly_Saturated or Certainly_Saturated.

Negative_Signal_Anomaly_Index_xxxx

A phenomenon, dubbed a “negative signal anomaly”, occurs when the level 1B attenuated backscatter becomes anomalously negative at the onset of an abrupt, strongly scattering target such as the planetary surface or a dense cloud. This effect can occur in any of the three lidar channels and is present in all versions of the CALIOP level 1B product. At this time only one range bin is believed to be affected in profiles containing the anomaly. Additional documentation is provided in the CALIPSO Data User’s Guide. The cause of the negative signal anomaly is currently under investigation by the CALIPSO team. Occurrences of this phenomenon are reported using a 16-bit integer in the Negative_Signal_Anomaly_Index_xxxx SDS. For those profiles where the negative surface anomaly is not present, the Negative_Signal_Anomaly_Index_xxxx is set to -1. Otherwise, values will lie in the range between 277 (at ~8.2 km) and 577 (at -0.5 km).

Level 1B data users are advised to exclude from their analyses attenuated backscatter in range bins identified by the Negative Signal Anomaly Index. Users of the 532 nm total attenuated backscatter data should consider the Negative Signal Anomaly Index for both parallel and perpendicular channels.



Last Updated: November 30, 2016





NASA
Last Updated: June 19, 2020
Curator: Charles R. Trepte
NASA Official: Charles R. Trepte

+ No Fear Acts
+ FOIA
+ Privacy

+ Office Of Inspector General
+ Agency Financial Reports

+ Contact NASA