NASA - National Aeronautics and Space Administration

Detailed Data Quality Summary for the CALIPSO Version 1.00 Lidar Level 3 Cloud Occurrence Monthly Data Product

Data Version: 1.00
Data Release Date: December 3, 2018
Data Date Range: June 13, 2006 to December 31, 2016
Production Strategy: Standard

The CALIPSO lidar level 3 (L3) three-dimensional (3D) cloud occurrence product reports global distributions of cloud occurrence aggregated monthly on a uniform 3-dimensional (3D) spatial grid. All clouds reported in the CALIPSO lidar level 2 products are initially considered. Level 3 quality assurance (QA) filters are used to identify and reject low confidence or otherwise suspicious layers reported in the level 2 data. In addition to the total number of quality-assured cloud samples detected within each grid cell, the cloud occurrence product reports separate totals for opaque clouds and transparent clouds, partitioned according to cloud thermodynamic phase (i.e., ice, water, and unknown phase) and accounts for the clouds rejected by the QA filters. Ice cloud information is further augmented by including histograms of layer optical depth (OD) for each grid cell. Background meteorological data (e.g., temperature, pressure, and relative humidity) and surface information such land/water surface types and snow sea ice extent coverage are also included.

Data are reported on standard grid dimensions of 2° latitude (85 elements) by 2.5° longitude (144 elements) by 60 m altitude (344 elements) over an altitude range of -0.4 km to 20.2 km above mean sea level. For each calendar month, three files are generated. These files report cloud occurrence statistics derived from daytime observations (using daytime granules only), nighttime observations (using nighttime granules only) and all observations (using both daytime and nighttime granules).

Due to an elevated frequency of low energy laser shots within and near the Southern Atlantic Anomaly (SAA) region since late May 2017, the generation of all CALIPSO L3 products, including the L3 cloud occurrence product, ceases after December 2016. The CALIPSO lidar science working group is currently working on understanding how this issue might affect all levels of CALIPSO data products reported within the SAA.

Level 2 Input Data

The version 1.00 of L3 3D cloud occurrence product is created from the V4.10 level 2 (L2) 5 km merged cloud layer product and V4.10 L2 5 km cloud profile product as shown in the Table 1.

Lidar Level 2 Input Data for Level 3 Cloud Occurrence Product, V1.00
Version Input Product(s) Data Date Range
V4.10 CAL_LID_L2_05kmMLay-Standard-V4-10*.hdf
June 2006 - December 2016

Unique Information in the CALIOP 3D Cloud Occurrence Product

The CALIOP 3D cloud occurrence product reports cloud occurrence counts in each 3D grid cell. Based on the number of cloud counts and cloud-free counts within each altitude bin, vertical profiles of volume cloud occurrence frequency are readily derived for each latitude-longitude bin.

The volume cloud fraction is different from the 2D cloud fraction typically derived from passive sensors measurements. For each 2D latitude-longitude grid cell, passive sensor cloud fractions report a scalar value quantifying how frequently clouds are observed anywhere within the vertical column above the surface. The CALIOP volume cloud occurrence frequency translates this scalar value into a range-resolved vector that expresses passive sensor “cloud fraction” as a function of altitude. Note, however, that the CALIOP 3D cloud occurrence product cannot be used to derive the passive-sensor-like 2D cloud occurrence, since the temporal information required for passive-sensor-like gridding is lost during aggregation process.

Figure 1. A sketch showing a hypothetical A-train trajectory assuming the CALIOP had 6 vertical ranges only and CALIOP and MODIS had the same horizontal resolution and perfectly collocated. Orange grid cells represent clouds detected simultaneously by the CALIOP and MODIS. The bottom image shows the cloud fraction 5/6 that would be reported by the MODIS.

This difference between cloud fraction and volume cloud occurrence frequency is illustrated with a simple sketch in Figure 1. Passive sensor cloud fraction is the ratio between cloudy pixels and the total number of pixels observed. As illustrated in the bottom image, there are six horizontal pixels (and collocated vertical profiles), with clouds observed in pixels/profiles 1, 2, 4, 5, 6. The passive sensor cloud fraction as would be reported by MODIS is thus 5/6.

The CALIOP L3 cloud occurrence product follows a different aggregation method. Cloud occurrence information is composited vertically, so that (as illustrated in the top right image in Figure 1) the merged profile reports 1, 1, 2, 0, 1, 1 cloud samples in each of 6 successive range bins. Since each range bin was composited over 6 along-track samples, the values reported in the corresponding profile of volume cloud occurrence frequency are 1/6, 1/6, 2/6, 0, 1/6 and 1/6 in each vertical grid, and the volume cloud occurrence frequency for the entire swath is 6/36.

The use of this different aggregation method makes it impossible to derive the passive sensor-like 2D cloud fraction from the information reported in the CALIPSO L3 3D cloud occurrence product, because it is impossible to detangle individual profile information from the vertically merged profiles. In other words, it is impossible to know which individual profiles contain clouds based only on the vertically merged profiles.

Zonal Cloud Fraction Patterns

The zonal cloud fraction describes the cloud fraction as a function of latitude and altitude. This requires an integral of longitudinal samples at each latitude and altitude bin.

Based on counts of cloud samples at each grid, zonal all cloud fraction could be defined as the ratio between accepted cloud samples and all observed samples. The all observed samples include cloud samples, which is also a sum of accepted cloud samples and rejected cloud samples, and cloud free samples. The formula is given below:

Cloud occurrence frequencies for any thermodynamic phase such as water, ice and unknown clouds can be calculated using a similar formula. For example, water cloud zonal occurrence frequency can be estimated by replacing the numerator in the above equation with the number of accepted water cloud samples and keeping the denominator the same. The accepted water cloud samples are derived by adding the total of transparent and opaque water clouds; e.g.,

Figure 2. Zonal cloud fraction patterns derived from the July 2008 CALIPSO L3 cloud occurrence product for all clouds (top left), ice clouds (top right), water clouds (bottom left) and unknown clouds (bottom right). Cloud occurrence less than 1% is represented in white.

Figure 2 shows zonal cloud occurrence frequency patterns derived from the July 2008 CALIPSO L3 cloud occurrence product. Ice clouds are found mainly at high-altitudes in the tropics and in polar regions. There is a significant contribution from polar stratosphere clouds (PSCs) over the Antarctic during this month. The highest concentration of water clouds occurs at low altitudes over the Southern Ocean. The unknown cloud zonal cloud fraction is quite small relative to the ice and water fractions, since the majority of unknown clouds identified in L2 algorithm fail to pass the QA tests enforced in the L3 cloud selection process.

With its sensitive detection capability, the CALIOP readily detects thin cirrus clouds (OD < 0.3) that are largely unrecognized by most passive sensors. When comparing the zonal cloud fractions derived from CALIOP measurements to those from passive sensors, it is therefore essential to use the appropriate OD range. The L3 cloud occurrence product enables this option by reporting histograms of layer OD for ice clouds. The observed ice cloud optical depths in each grid cell are partitioned as shown in Table 2.

Table 2: bin sizes for L3 3D cloud occurrence product ice cloud optical depth histograms
Bin Number Optical Depth (τ) Range
1 τ < 0.01
2 0.01 ≤ τ < 0.03
3 0.03 ≤ τ < 0.10
4 0.10 ≤ τ < 0.30
5 0.30 ≤ τ < 1.00
6 τ ≥ 1.00 and transparent
7 opaque

Optical depth thresholds can be created by combining sample counts and ice cloud histogram information. For example, if assuming that clouds with optical depths less than 0.3 are all ice clouds, the fraction of all cloud samples with OD ≥ 0.3 is simply the difference between all accepted cloud samples and number of ice cloud samples with OD < 0.3. The number of ice cloud samples with OD < 0.3 in any grid cell is just the sum of first four bins of the ice cloud histogram in that cell. Consequently, the zonal cloud fraction for ice clouds with OD ≥ 0.3 is computed using the formula below.

Figure 3. Zonal cloud occurrence frequency patterns for all clouds (left) and clouds with OD ≥ 0.3 only (right).

The left panel in Figure 3 shows the zonal occurrence frequency pattern for all clouds, while the right panel shows the occurrence frequency pattern for clouds with OD ≥ 0.3. Comparing these two images suggests that optically thin cirrus clouds are mainly observed in the tropical upper troposphere and as polar stratospheric clouds (PSCs) over Antarctica.

Known Issues

In the initial release of V1.00, there are a few issues which need further evaluations to find better solutions. Here we provide a list of known issues and our suggestions.

  1. Inconsistent counts of opaque ice samples

    Opaque ice clouds samples could be either found from Ice_Cloud_Opaque_Samples or the last bin in the histogram Ice_Cloud_Optical_Depth_Histogram. In the current algorithm, there are small differences in the two counts which lead to about 3% opaque ice cloud fraction difference.

    This problem occurs when a thick opaque 5 km cloud merges with several thin single shot cloud layers (and the lowest layer is also opaque) embedded in it. The Ice_Cloud_Opaque_Samples and Ice_Cloud_Optical_Depth_Histogram counts slightly differently for those transparent single shot cloud layers embedded in this opaque 5 km cloud layers.

    How to reconcile the issue needs an extensive study and might need to change the underlying assumptions.

    To avoid confusion, the last bin of the histogram is reserved with the filled value -9999. The current algorithm should not affect optically thin ice cloud counts especially the first 5 bins. If a user is interested in evaluating opaque ice clouds, please use the Ice_Cloud_Opaque_Samples.

  2. Low altitude water clouds over the Southern Ocean might be underestimated

    To avoid overestimation of low altitude broken marine clouds with small spatial scales, a quality assurance filter is introduced to remove water clouds with cloud top less than 8. 2 km identified from horizontal averaging equals 5, 20 or 80 km.

    Recently Deladillo et al. [2018] showed that optically thin water clouds are ubiquitously observed over the Southern Ocean. Needless to say, those water clouds would probably be missed from single shot feature detection algorithm. A horizontal averaging is needed to increase the single-to-noise ratio thus to detect those faint water clouds. Excluding those optically thin water clouds might result an underestimation of water cloud samples over the Southern Ocean.

    Figure 4. Zonal water cloud fraction without and with the low water cloud filter.

    From the two zonal water cloud fractions without and with the low water cloud filter as shown in Figure 4, it can be seen that the filter removed about 20% of water clouds at low altitudes over the Southern Ocean.

    The final decision has to be made based on a detailed analysis evaluating the risk of overestimation of small-scale broken marine clouds if excluding the filter and underestimation of large-scale thin water clouds scenario if including the filter. It also might be helpful to compare the patterns with another passive sensor observation.


Delgadillo, R., Voss, K. J., & Zuidema, P. (2018): “Characteristics of optically thin coastal Florida cumuli derived from surface-based lidar measurements”. Journal of Geophysical Research: Atmospheres, 123, 10,591-10,605.

Last Updated: November 22, 2021
Curator: Charles R. Trepte
NASA Official: Charles R. Trepte

+ No Fear Acts
+ Privacy

+ Office of Inspector General
+ Office of Special Councel
+ Agency Financial Reports

+ Contact NASA