NASA - National Aeronautics and Space Administration

Detailed Data Quality Summary for the CALIPSO Version 1.00 Lidar Level 3 Ice Cloud Data Product

Data Version: 1.00
Data Release Date: May 2018
Data Date Range: June 13, 2006 to December 31, 2016

The lidar level 3 (L3) ice cloud product reports global distributions of ice cloud extinction coefficient and ice water content histograms on a uniform spatial grid with monthly measurements. It also includes background meteorological data, cloud sample counts and surface information. Each month, there are three files describing statistics during day (with day granules only), night (night granule only) and all (with both day and night granules).

Due to an elevated frequency of low energy shots within and near the Southern Atlantic Anomaly region since late May 2017, all level 3 products including this level 3 ice cloud product have been stopped processing after December 2016. The lidar science working group is currently working on understanding how this issue might affect all levels of the CALIPSO products.

Level 2 Input Data

The version 1.00 of the level 3 ice cloud product is processed from the V4.10 level 2 5 km cloud profile product as shown in the following table.

Lidar Level 2 Input Data for Level 3 Ice Cloud Product, V1.00
Level 2 Version Data Date Range
V4.10 June 13, 2006 - December 31, 2016

A Brief Description of the Algorithm

In order to properly aggregate the statistics from the level 2 cloud profile product one must consider four different conditions within each 5 km column; cloud free, cloud, totally attenuated and lidar detected surface and/or subsurface. These different features are identified using the Atmospheric_Volume_Description parameter contained in the lidar level 2 product.

A “cloud free” region includes either clear sky or aerosol. A “cloud” region contains features classified as clouds prior to the application of any quality filters. Depending on the opacity of a 5 km column, i.e., feature type 7 no signal (totally attenuated) presenting or not, the column can either have a “totally attenuated region” or “lidar detected surface/subsurface region”. If the column is opaque, “totally attenuated region” spans altitudes from where the backscatter signal is totally attenuated to the low boundary of vertical range which is around - 0.5 km. Typically the start of the totally attenuated region is the apparent base of the lowest detected cloud layer. In this case, there is no lidar detected surface or subsurface region. On the contrary, if the column is transparent, the lidar detected surface and subsurface region is from the actual lidar detected surface to the low boundary of vertical range which is around -0.5 km.

For those instances when clouds are detected in the column, the algorithm further aggregates the statistical distribution of the ice/water phase decoded from Atmospheric_Volume_Description parameter. The level 3 ice cloud product reports the number of clouds classified as unknown/not determined, water, and ice clouds prior to quality filtering. The algorithm segregates counts for both randomly orientated ice and horizontally orientated ice as ice clouds.

Further, if the clouds identified in the column are defined as ice the algorithm then applies quality filters and rejects/accepts ice cloud samples depending whether they passed a series of quality filters. Only the extinction coefficient and ice water content of “accepted” ice clouds were included into the histograms and median calculations.

Quality Filter Information

The quality filters are used to select high confidence ice clouds with confident extinction retrievals. There are two types of filters: quality assurance (QA) filters and quality control (QC) filters. The QA filters are chosen to select high confidence ice clouds in terms of feature type and ice/water phase. And the QC filters make sure the extinction retrieval was confident.

QA Filter Acceptance Criteria:

  1. CAD Score Filter: feature type confidence is low, medium or high (featureQA = 1, 2, 3). This is equivalent to choosing clouds with CAD score between 20 and 100.
  2. Cloud Phase Filter: Only randomly oriented ice clouds with a high confidence are considered. The level 2 5 km cloud profile product reports cloud phase and confidence at 30 m bins, while extinction and ice water content are reported at 60 m resolution. To account for these different bin averages the algorithm requires that each 30 m bin contained in the parent 60 m bin must be high confidence randomly oriented clouds, otherwise the entire 60 m bin is filtered out.

QC Filter Acceptance Criteria:

  1. Extinction QC Flag Filter: Only extinction QC flag = 0, 1, 2, 16 or 18 are allowed. The extinction QC flag refers to the one derived from 532 nm channel only. These include both constrained and unconstrained retrievals, and both transparent and opaque clouds. Extinction QC flag other than those values are ignored since these retrievals indicate a greater propensity to exhibit erroneously large or negative values, or no solution possible within allowable lidar ratio bounds. More information about this flag could be found on the Level 2 Cloud Profile Product Data Summary.
  2. Extinction QC Uncertainty Filter: The extinction QC uncertainty should be less than 99.9 km-1 to ensure convergent solution. An uncertainty of 99.9 km-1 is a flag value assigned by the retrieval algorithm, and occurs when the extinction uncertainty estimate begins to diverge to infinity while iterating downward through the profile (i.e. high to low altitude). In this case the retrieval results become unreliable and the algorithm excludes ice cloud samples having extinction uncertainty equal to 99.9 km-1, as well as all samples below as the untrustworthy extinction solution is propagated to lower altitudes. All excluded ice clouds are counted as rejected ice cloud samples.
  3. Overlying Optical Depth Filter: The overlying optical depth (OD) should be less than or equal to 2. As one moves to the profile from high to low (i.e. the altitude decreases) the signal gradually attenuates due to both the background atmosphere and any embedded features, the overlying optical depth increases so does the uncertainty. To mitigate an increased uncertainty an overlying OD threshold 2 is set. The overlying OD is derived by summing the 532 nm extinction coefficient over the defined vertical grids. The algorithm continually derives the OD as it progresses downward. At the point in which the OD reaches the threshold all ice clouds samples at that altitude and below in the profile are ignored and counted as rejected ice cloud samples.
  4. Overlying Feature Filter If an invalid feature (feature type = 0) or water cloud (feature type = 2 and ice/water phase = 2) is detected, based on feature information decoded from the Atmospheric_Volume_Description, then any ice cloud samples identified beneath is considered suspicious and is rejected.

Figure 1 shows how the quality filters impact ice cloud occurrence. While the zonal patterns with/without filters look similar, the quality filters rejected ice clouds observed at low altitude region around tropical region and cirrus cloud edges or cirrus fringes.

Figure 1. Zonal patterns of the ice cloud occurrence without and with quality filters are shown on the left top and bottom. Cloud occurrence less than 1% is represented in white. The percentage of removal which is the ratio between the pattern with filters and that without filters is plotted on the right. The patterns are derived from 2008 July month night L3 ice cloud product data.

Cirrus Fringe

As mentioned in “Filter for Misclassified Cirrus” in the level 3 aerosol product description web page, the tenuous edges of cirrus clouds are misclassified as aerosol by the CALIPSO Cloud Aerosol Discrimination (CAD) algorithm. In the V4.10 level 2 algorithm, those misclassified aerosols are changed to low confidence cloud, none confidence phase random orientated ice clouds with a special CAD score 106.

For the lidar level 3 ice cloud algorithm these ice clouds with a CAD of 106 have been counted but rejected.

Spatial Grids of Level 3 Tropospheric Aerosol and Level 3 Ice Cloud Product

The spatial grid of level 3 ice cloud product is designed to be compatible with the level 3 tropospheric aerosol product. The spatial resolution for level 3 ice cloud product is 2.5° longitude by 2.0° latitude and 120 m altitude. For level 3 aerosol product, it is 5.0° longitude by 2.0° latitude and 60 m altitude. Both products start to count from the same lower altitude boundary.

Figure 2 illustrates the vertical grid resolution and vertical range of lidar level 2 aerosol/cloud profile product, lidar level 3 tropospheric aerosol product and lidar level 3 ice cloud product. If a user is interested in combining the lidar level 3 tropospheric and ice cloud products, they simply define a grid resolution, e.g, longitude 5.0° by latitude 2.0° by altitude 120 m, and then aggregate ice cloud information and aerosol information from their default grid resolutions to this new grid resolution.

Figure 2. Vertical grid resolution and vertical range of level 2 aerosol/cloud profile product, level 3 tropospheric aerosol product and level 3 ice cloud product. The numbers on the left column of level 2 aerosol/cloud profile product are the lidar altitudes which are the centers of vertical bins. The dots on the right column show the center point. Similarly, the numbers on the left columns of level 3 product are the lidar altitude at the center points as shown as black dots on the right columns. Note the center of level 3 ice cloud product vertical grid is the center of every two neighboring vertical grids of level 3 tropospheric aerosol product.

Histogram Bins

In the level 3 ice cloud product, the ice cloud extinction coefficients retrieved from 532 nm channel and ice water content are described by two full histograms with 44 bins.

The histogram bin structure is specially designed to accommodate retrieved extinction coefficient and ice water content which spans several orders of magnitude. As shown in Table 1, among those 44 bins, 42 bins contain the nominal range of expected values from bin 2 to 43. Expected values for extinction coefficients are -0.1 km-1 to 10.0 km-1 and those for ice water content are between -0.01 g·m-3 and 1.0 g·m-3. Negative and positive outliers are recorded in histogram bins 1 and 44, respectively. Small magnitude values with less confidence are stored in bin 17 and 18. The 40 bins (bins 2-16 and bins 19-43) are uniform in common logarithm space with base 10. Each bin size is 0.2, thus five bins represent 1 order of magnitude. Note the bin boundaries reported here are in linear scale thus they are not uniformly spaced. Negative extinction coefficients and ice water contents are expected within the distribution of level 2 retrieved solutions and are thereby included in the level 3 product to accurately represent the retrieved population in the level 2 data product.

Table 1: Histogram Bin Structures

Bin number Extinction Coefficient Range (km-1) IWC Range (g·m-3)
1 (-3.402 x 1038, -1.0 x 10-1) (-3.402 x 1038, -1.0 x 10-2)
2-16 [-1.0 x 10-1, -1.0 x 10-4) [-1.0 x 10-2, -1.0 x 10-5)
17 [-1.0 x 10-4, 0) [-1.0 x 10-5, 0)
18 [0, 1.0 x 10-4) [0, 1.0 x 10-5)
19-43 [1.0 x 10-4, 1.0 x 10+1) [1.0 x 10-5, 1.0 x 100)
44 [1.0 x 10+1, 3.402 x 1038) [1.0 x 100, 3.402 x 1038)

Figure 3 shows the ice cloud extinction coefficient and ice water content histograms at three altitude bins around the tropical region at night in July 2008. The tropical region is defined as a region between latitude 23.5° S to 23.5° N. The peak of higher altitude distribution has a smaller extinction coefficient and ice water content. This suggests that the in-cloud extinction coefficient and ice water content increase as the altitude decreases. Also, the area of each distribution gives the number of observed ice clouds at that altitude range. There are many more ice clouds observed between 13 km and 14 km than those between 9 km and 10 km.

Figure 3. Histograms of ice cloud extinction coefficients and ice water content observed at three altitude ranges around the tropical region. The X-axis is in logarithm scale. The Y-axis is in linear scale. The vertical dashed line separates negative and positive values.

In-cloud IWC and Grid-averaged IWC

In the lidar level ice cloud product, the medians ice cloud extinction coefficient or ice water content (IWC) in each 3D spatial grid is reported as parameter Extinction_Coefficient_532_Median and Ice_Water_Content_Median. As explained above, there are 44 bins describing the full distribution of ice water content. The valid range of ice water content is from -0.01 g·m-3 to 1.0·g m-3. The first bin (bin 0) and the last bin (bin 43) are for ice water content less than -0.01 g·m-3 and larger than 1.0 g·m-3. There are also two bins 16 and 17 counting for very small ice water content whose magnitudes are less than 10-5. These coefficients are typically less confident in the retrieval algorithm. However, those extreme small values are included when reporting the median ice water content. Note these two parameters describe the extinction coefficient/ice water content within ice clouds, in other words, during ice clouds only.

If a user is interested in calculating the mean in-cloud ice water content, it is recommended using the set of equations, with parameters Ice_Water_Content_Histogram, Ice_Water_Content_Bin_Boundary (middle boundary for example). Here the number of the accepted ice clouds does not exactly equal to Ice_Cloud_Accepted_Samples since we do not include small or big outliers in bin 0, 16, 17 and 43.

It may be necessary to consider ice cloud occurrence when deriving the grid-averaged ice water content. In this case, the occurrence of ice cloud in each 3D grid is estimated as the ratio between number of accepted ice clouds (excluded outliers) with the sum of cloud and cloud free samples. The following set of equations can derive grid-averaged ice water content using the parameters Ice_Water_Content_Histogram, Ice_Water_Content_Bin_Boundary, Cloud_Samples and Cloud_Free_Samples:

Figure 4 shows the in-cloud averaged ice water content and grid-averaged ice water content. As shown on the left, the in-cloud averaged ice water content describes the capacity of ice water content an ice cloud can hold at a specific altitude/temperature. This capacity increases as the altitude decreases and the temperature increases. Considering the ice clouds occurrence drops as the altitude decreases, the grid-averaged will increase first and then decrease with altitude as shown on the right of Figure 4.

Figure 4. Zonal pattern of in-cloud averaged ice water content and grid-averaged ice water content at night in July 2008. The latitude bin is 2° and the altitude bin is 120 m. The X-axis shows the laitutde and Y-Axis altitude. The ice water content is shown in logarithm. The unit is mg·m-3.

The same discussions also apply to extinction coefficient. To derive the grid-averaged extinction coefficient, use the same formula and replace the parameters Ice_Water_Content_Histogram and Ice_Water_Content_Bin_Boundary with Extinction_Coefficient_532_Histogram and Extinction_Coefficient_532_Bin_Boundary.

Excluded Profiles

There may be instances when for a given 5 km profile (the fundamental resolution used by the level 2 data) there is clear-air throughout the entire column without valid lidar detected surface (no feature type 5 neither). This occurs when the lowest cloud or aerosol layer in a 5 km profile touches the surface but yet fails to pass the surface thickness threshold test set up in the lidar level 2 algorithm. For these cases, the entire 5 km profile is excluded. These removed columns are added to the Number_of_Bad_Profiles parameter contained in the file. In July 2008 month, the occurrences were about 0.08% during day and 0.007% at night. Since the numbers are so small removing those bad profiles would not affect the level 3 monthly statistics.

Information of Processed Level 2 5 km Cloud Profile Product files

Information of the number of Level 2 5 km Cloud Profile Product files is reported as Number_of_Level2_Files_Analyzed in metadata. The list of those files is also provided in List_of_Input_Files. As shown in Figure 5, the number of processed day granules typically equals that of night granules in each month. Occasionally there would be one count off due to the possibility including the last granule from the previous month for processing either day or night granules. This granule has records for two consecutive days where some belonging to the previous month and some to the current month.

It is worthy noticing that there are no level 3 ice cloud product files in February 2016. During that month, the CALIPSO was turned off due to a GPS clock problem.

Figure 5. The numbers of processed L2 5 km cloud profile product files for the standard V1.00 level 3 ice cloud product.

In the metadata, a user could also find detailed information about grid information and filter information in the Program_Configuration if interested.

Last Updated: November 22, 2021
Curator: Charles R. Trepte
NASA Official: Charles R. Trepte

+ No Fear Acts
+ Privacy

+ Office of Inspector General
+ Office of Special Councel
+ Agency Financial Reports

+ Contact NASA