NASA - National Aeronautics and Space Administration

DETAILED Data Quality Document for the CALIPSO Lidar Level 1.5 Standard Version 1.00 and Expedited Version 3.50 Data Products

Last Updated: October 09, 2020

Data Version: 3.50   Data Version: 1.00
Data Date Range: March 1, 2019 to September 30, 2020   Data Date Range: June 13, 2006 to September 30, 2020
Data Release Date: March 1, 2019   Data Release Date: April 10, 2019
Data Version: 3.51   Data Version: 1.01
Data Date Range: October 01, 2020 to present   Data Date Range: October 01, 2020 to present
Data Release Date: October 01, 2020   Data Release Date: October 01, 2020

10-09-2020: A minor version bump (+0.01) has been applied to all CALIPSO data products due to a required upgrade to the operating system on the CALIPSO production cluster. All algorithms were re-compiled to process in this new environment with no change to the underlying science algorithms or inputs.

The level 1.5 product was initially released as a beta product for expedited, near-real time purposes, using the version 3 expedited data products as input. The spring 2019 level 1.5 data product releases encompass two major changes:

1. Initial release of a new standard level 1.5 product — The new V1.00 standard level 1.5 product is based entirely on version 4 CALIOP data, thus providing accurately calibrated attenuated backscatter coefficients and the benefits of the most sophisticated CALIOP scene classification algorithms. This product represents the highest quality level 1.5 data product.

2. Updates to the expedited level 1.5 product — The V3.50 expedited level 1.5 product, still based on expedited version 3 CALIOP data, has been updated according to the changes detailed below to ensure compatibility with the new standard level 1.5 product.

The feature clearing and averaging algorithms are identical between the V1.00 standard and V3.50 expedited level 1.5 products, with the exception of how stratospheric features are handled (discussed below). The products share identical science data sets (SDSs), with the exception of aerosol extinction information. Extinction coefficients and extinction QC flags are not included in V1.00 standard because quality-screening procedures for their use are still under development. Details of major changes to the level 1.5 data products are given below.

Top altitude increased to 30.1 km
The top altitude has been extended from 20.2 km (the top altitude in the beta version) to 30.1 km so that stratospheric features can be represented by the product. Due to the fundamental resolutions of CALIOP level 1B data, the resolution of level 1.5 data in the 20.2-30.1 km region is coarser than that of lower altitudes (Table 1). Subsequently, differences in attenuated backscatter standard deviations, uncertainties, and medians are expected between these regions due to the differences in sample numbers.

Altitude Region Spatial Resolution
-0.5 to 20.2 km 20 km horizontal x 60 m vertical
20.2 to 30.1 km 20 km horizontal x 180 m vertical
Table 1. Spatial resolution of the level 1.5 product

Attenuated backscatter 1064 nm added
Attenuated backscatter coefficients at 1064 nm have been added to accompany the 532 nm data already provided. Associated 1064 nm calibration constants and molecular model attenuated backscatter coefficients are also added.

Cloud clearing and surface contamination clearing improved
In order to avoid cloud contamination near cloud edges, the cloud mask derived from the CALIOP vertical feature mask is now dilated vertically and horizontally by the distances in Table 2. Attenuated backscatter coefficients within the dilated cloud mask are removed prior to subsequent averaging. Similarly, attenuated backscatter within 30 m of the surface is removed to avoid surface contamination.

Altitude Region Vertically Horizontally
20.1 - 30.2 km 180 m 5/3 km
8.2 - 20.1 km 60 m 1 km
-0.5 - 8.2 km 30 m 1/3 km
Table 2. Cloud mask dilation amount

Figure 1 below shows the impact of cloud mask dilation on level 1.5 attenuated backscatter. The topmost panel shows enhanced attenuated backscatter due to residual undetected clouds. By dilating the cloud mask, these residual cloudy samples are removed (middle panel).

Figure 1. Example of level 1.5 total attenuated backscatter without cloud mask dilation (top panel) and with cloud mask dilation (middle panel) for the granule 2014-08-05T09-08-14ZN. The bottom panel shows the level 2 vertical feature mask from which the cloud mask is derived.

Laser energy information added
New science data sets have been added to characterize the CALIOP single-shot laser energies within each level 1.5 profile:


Laser energy information is valuable for quality screening level 1.5 data because CALIOP has been experiencing low energy laser shots since September 2016, primarily over the South Atlantic Anomaly region that have significantly degraded the quality of affected profiles. The most effective method for screening level 1.5 profiles affected by low energy is to exclude profiles based on Minimum_Laser_Energy_532. Currently, the CALIPSO science team recommends removing profiles having a Minimum_Laser_Energy_532 < 0.08 J. More information is given on the CALIOP Data Advisory Page.

Extinction QC flag revised for comprehensive reporting
Extinction QC flags are bit-mapped integers that summarize the final state of the level 2 extinction retrieval process. They are also valuable metrics for quality screening. Because multiple extinction samples are aggregated into a level 1.5 range bin, the level 2 extinction QC flags must be summarized in a useful manner for quality screening. Previously, the maximum value of the extinction QC flag was reported within each level 1.5 range bin. Though useful for identifying the most egregious error states of the extinction retrieval process, it did not provide a comprehensive summary of the final states of all solutions within the level 1.5 range bin.

In this new release, the bitwise OR is calculated from all aerosol extinction QC flags within each range bin, thereby allowing all contributing extinction QC flags to be represented. Consequently, the SDS has been renamed from “Extinction_QC_Flag_532_Maximum” to “Extinction_QC_Flag_532”. This SDS only appears in the V3.50 expedited level 1.5 product.

L2_Feature_Type definitions revised
The feature type definitions reported in the SDS L2_Feature_Type have been revised to accommodate the new aerosol subtypes that have been added to the version 4 level 2 data products (dusty marine, PSC aerosol, volcanic ash, sulfate/other). The new definitions are compared against those of the beta version below. Note that aerosol layers in the V3.50 expedited level 1.5 product will never contain these new aerosol subtypes (they will be classified as different types) because these classifications do not exist in V3 level 2 products.

Value L2_Feature_Type (V3.50 Expedited, V1 Standard) L2_Feature_Type (Beta Product)
0 invalid (bad or missing data) invalid (bad or missing data)
1 totally attenuated totally attenuated
2 surface surface
3 subsurface subsurface
4 cloud cloud
5 clean marine clean marine
6 dust dust
7 polluted continental/smoke polluted continental/smoke
8 clean continental clean continental
9 polluted dust polluted dust
10 elevated smoke smoke/ biomass burning
11 dusty marine mixed aerosol
12 PSC aerosol cloud-cleared mixed aerosol
13 volcanic ash cloud-cleared clean marine
14 sulfate/other cloud-cleared dust
15 mixed aerosol cloud-cleared polluted continental/smoke
16 cloud-cleared clean marine cloud-cleared clean continental
17 cloud-cleared dust cloud-cleared polluted dust
18 cloud-cleared polluted continental/smoke cloud-cleared elevated smoke
19 cloud-cleared clean continental “clear air”
20 cloud-cleared polluted dust cloud-cleared “clear air”
21 cloud-cleared elevated smoke overcast
22 cloud-cleared dusty marine  
23 cloud-cleared PSC aerosol  
24 cloud-cleared volcanic ash  
25 cloud-cleared sulfate/other  
26 cloud-cleared mixed aerosol  
27 “clear air”  
28 cloud-cleared “clear air”  
29 overcast  
Table 3. Numeric value and definitions of L2_Feature_Type for the V3.50 expedited and V1.00 standard releases compared to that of the beta release.

Differences in treatment of stratospheric features; stratospheric aerosol subtypes included in V1.00 standard level 1.5
Stratospheric features are now handled differently by the level 1.5 algorithm depending on the version of the input data being used. Whereas V3 level 2 scene classification regards all layers detected above the tropopause as “stratospheric features”, the V4 scene classification algorithm performs cloud-aerosol discrimination above the tropopause. This allows identification of aerosol layers and their subtypes in the lower stratosphere, requiring the level 1.5 algorithm to handle them differently.

The convention for handling stratospheric features in V3.50 expedited level 1.5 (using V3 level 2 input data) is identical to the previous beta version: medium and high-confidence stratospheric features between 60°S and 60°N are assigned an L2_Feature_Type of “clear-air”. All other stratospheric features including every stratospheric feature detected poleward of 60° are removed as though they are clouds.

The convention for V1.00 standard level 1.5 (using V4 level 2 input data) is to keep all stratospheric aerosol layers classified as volcanic ash, sulfate/other, and smoke. Aerosol layers having the subtype “PSC aerosol” are removed.

New ancillary science data sets
Land water mask and tropopause height SDSs have been added.

Filename format changed for V3 expedited
The filename of the beta level 1.5 expedited product ended with “D.hdf”. The “D” has been replaced with “Z” for the V3.50 expedited filename because expedited granules contain both day and night observations. The “Z” stands for Zulu time.

Last Updated: November 22, 2021
Curator: Charles R. Trepte
NASA Official: Charles R. Trepte

+ No Fear Acts
+ Privacy

+ Office of Inspector General
+ Office of Special Councel
+ Agency Financial Reports

+ Contact NASA