Detailed Data Quality Summary for the CALIPSO Version 1.00 Lidar Level 3 Ice Cloud Data Product
Data Release Date:
Data Date Range:
June 13, 2006 to December 31, 2016
The lidar level 3 (L3) ice cloud product reports global distributions of ice
cloud extinction coefficient and ice water content histograms on a uniform spatial
grid with monthly measurements. It also includes background meteorological data,
cloud sample counts and surface information. Each month, there are three files
describing statistics during day (with day granules only), night (night granule only)
and all (with both day and night granules).
Due to an elevated frequency of low energy shots within and near the Southern
Atlantic Anomaly region since late May 2017, all level 3 products including this
level 3 ice cloud product have been stopped processing after December 2016. The
lidar science working group is currently working on understanding how this issue
might affect all levels of the CALIPSO products.
Level 2 Input Data
The version 1.00 of the level 3 ice cloud product is processed from the V4.10 level 2
5 km cloud profile product as shown in the following table.
Lidar Level 2 Input Data for Level 3 Ice Cloud Product, V1.00
Level 2 Version
Data Date Range
June 13, 2006 - December 31, 2016
A Brief Description of the Algorithm
In order to properly aggregate the statistics from the level 2 cloud profile
product one must consider four different conditions within each 5 km column;
cloud free, cloud, totally attenuated and lidar detected surface and/or
subsurface. These different features are identified using the
parameter contained in the lidar level 2 product.
A “cloud free” region includes either clear sky or aerosol.
A “cloud” region contains features classified as clouds prior to the
application of any quality filters. Depending on the opacity of a 5 km column,
i.e., feature type 7 no signal (totally attenuated) presenting or not, the column
can either have a “totally attenuated region” or
“lidar detected surface/subsurface region”. If the column is opaque,
“totally attenuated region” spans altitudes from where the backscatter
signal is totally attenuated to the low boundary of vertical range which is
around - 0.5 km. Typically the start of the totally attenuated region is the
apparent base of the lowest detected cloud layer. In this case, there is no
lidar detected surface or subsurface region. On the contrary, if the column is
transparent, the lidar detected surface and subsurface region is from the actual
lidar detected surface to the low boundary of vertical range which is around -0.5 km.
For those instances when clouds are detected in the column, the algorithm further
aggregates the statistical distribution of the ice/water phase decoded from
parameter. The level 3 ice cloud product reports the number of clouds classified
as unknown/not determined, water, and ice clouds prior to quality filtering. The
algorithm segregates counts for both randomly orientated ice and horizontally
orientated ice as ice clouds.
Further, if the clouds identified in the column are defined as ice the algorithm
then applies quality filters and rejects/accepts ice cloud samples depending whether they
passed a series of quality filters. Only the extinction coefficient and ice water
content of “accepted” ice clouds were included into the histograms and median calculations.
Quality Filter Information
The quality filters are used to select high confidence ice clouds with
confident extinction retrievals. There are two types of filters: quality assurance
(QA) filters and quality control (QC) filters. The QA filters are chosen to select
high confidence ice clouds in terms of feature type and ice/water phase. And the
QC filters make sure the extinction retrieval was confident.
QA Filter Acceptance Criteria:
CAD Score Filter: feature type confidence is low, medium or high
(featureQA = 1, 2, 3). This is equivalent to choosing clouds with
between 20 and 100.
Cloud Phase Filter: Only randomly oriented ice clouds with a high
confidence are considered. The level 2 5 km cloud profile product reports cloud
phase and confidence at 30 m bins, while extinction and ice water content are
reported at 60 m resolution. To account for these different bin averages the
algorithm requires that each 30 m bin contained in the parent 60 m bin must be
high confidence randomly oriented clouds, otherwise the entire 60 m bin is
QC Filter Acceptance Criteria:
Extinction QC Flag Filter: Only extinction QC flag = 0, 1, 2, 16 or 18
are allowed. The extinction QC flag refers to the one derived from 532 nm channel
only. These include both constrained and unconstrained retrievals, and both
transparent and opaque clouds. Extinction QC flag other than those values are
ignored since these retrievals indicate a greater propensity to exhibit erroneously
large or negative values, or no solution possible within allowable lidar ratio
bounds. More information about this flag could be found on the
Level 2 Cloud Profile Product Data Summary.
Extinction QC Uncertainty Filter: The extinction QC uncertainty should
be less than 99.9 km-1 to ensure convergent solution. An uncertainty of
99.9 km-1 is a flag value assigned by the retrieval algorithm, and
occurs when the extinction uncertainty estimate begins to diverge to infinity
while iterating downward through the profile (i.e. high to low altitude). In this
case the retrieval results become unreliable and the algorithm excludes ice cloud
samples having extinction uncertainty equal to 99.9 km-1,
as well as all samples below as the untrustworthy extinction solution is propagated
to lower altitudes. All excluded ice clouds are counted as rejected ice cloud samples.
Overlying Optical Depth Filter: The overlying optical depth (OD) should
be less than or equal to 2. As one moves to the profile from high to low (i.e.
the altitude decreases) the signal gradually attenuates due to both the background
atmosphere and any embedded features, the overlying optical depth increases so does
the uncertainty. To mitigate an increased uncertainty an overlying OD threshold 2
is set. The overlying OD is derived by summing the 532 nm extinction coefficient
over the defined vertical grids. The algorithm continually derives the OD as it
progresses downward. At the point in which the OD reaches the threshold all ice
clouds samples at that altitude and below in the profile are ignored and counted
as rejected ice cloud samples.
Overlying Feature Filter If an invalid feature (feature type = 0)
or water cloud (feature type = 2 and ice/water phase = 2) is detected, based on
feature information decoded from the
then any ice cloud samples identified beneath is considered suspicious and is rejected.
Figure 1 shows how the quality filters impact ice cloud occurrence. While the
zonal patterns with/without filters look similar, the quality filters rejected
ice clouds observed at low altitude region around tropical region and cirrus
cloud edges or cirrus fringes.
Figure 1. Zonal patterns of the ice cloud occurrence without and with
quality filters are shown on the left top and bottom. Cloud occurrence less than
1% is represented in white. The percentage of removal which is the ratio between
the pattern with filters and that without filters is plotted on the right.
The patterns are derived from 2008 July month night L3 ice cloud product data.
As mentioned in
“Filter for Misclassified Cirrus”
in the level 3 aerosol product description web page, the tenuous edges of cirrus
clouds are misclassified as aerosol by the CALIPSO Cloud Aerosol Discrimination
(CAD) algorithm. In the V4.10 level 2 algorithm, those misclassified aerosols
are changed to low confidence cloud, none confidence phase random orientated
ice clouds with a special CAD score 106.
For the lidar level 3 ice cloud algorithm these ice clouds with a CAD of 106
have been counted but rejected.
Spatial Grids of Level 3 Tropospheric Aerosol and Level 3 Ice Cloud Product
The spatial grid of level 3 ice cloud product is designed to be compatible
with the level 3 tropospheric aerosol product. The spatial resolution for level 3
ice cloud product is 2.5° longitude by 2.0° latitude and 120 m altitude.
For level 3 aerosol product, it is 5.0° longitude by 2.0° latitude and
60 m altitude. Both products start to count from the same lower altitude boundary.
Figure 2 illustrates the vertical grid resolution and vertical range of lidar
level 2 aerosol/cloud profile product, lidar level 3 tropospheric aerosol product
and lidar level 3 ice cloud product. If a user is interested in combining the lidar
level 3 tropospheric and ice cloud products, they simply define a grid resolution,
e.g, longitude 5.0° by latitude 2.0° by altitude 120 m, and then aggregate
ice cloud information and aerosol information from their default grid resolutions
to this new grid resolution.
Figure 2. Vertical grid resolution and vertical range of level 2
aerosol/cloud profile product, level 3 tropospheric aerosol product and level 3
ice cloud product. The numbers on the left column of level 2 aerosol/cloud profile
product are the lidar altitudes which are the centers of vertical bins. The dots
on the right column show the center point. Similarly, the numbers on the left
columns of level 3 product are the lidar altitude at the center points as shown
as black dots on the right columns. Note the center of level 3 ice cloud product
vertical grid is the center of every two neighboring vertical grids of level 3
tropospheric aerosol product.
In the level 3 ice cloud product, the ice cloud extinction coefficients
retrieved from 532 nm channel and ice water content are described by two full
histograms with 44 bins.
The histogram bin structure is specially designed to accommodate retrieved
extinction coefficient and ice water content which spans several orders of
magnitude. As shown in Table 1, among those 44 bins, 42 bins contain the nominal
range of expected values from bin 2 to 43. Expected values for extinction
coefficients are -0.1 km-1 to 10.0 km-1 and those for ice
water content are between -0.01 g·m-3 and 1.0 g·m-3.
Negative and positive outliers are recorded in histogram bins 1 and 44, respectively.
Small magnitude values with less confidence are stored in bin 17 and 18. The 40 bins
(bins 2-16 and bins 19-43) are uniform in common logarithm space with base 10.
Each bin size is 0.2, thus five bins represent 1 order of magnitude. Note the bin
boundaries reported here are in linear scale thus they are not uniformly spaced.
Negative extinction coefficients and ice water contents are expected within the
distribution of level 2 retrieved solutions and are thereby included in the
level 3 product to accurately represent the retrieved population in the
level 2 data product.
Table 1: Histogram Bin Structures
Extinction Coefficient Range (km-1)
IWC Range (g·m-3)
(-3.402 x 1038, -1.0 x 10-1)
(-3.402 x 1038, -1.0 x 10-2)
[-1.0 x 10-1, -1.0 x 10-4)
[-1.0 x 10-2, -1.0 x 10-5)
[-1.0 x 10-4, 0)
[-1.0 x 10-5, 0)
[0, 1.0 x 10-4)
[0, 1.0 x 10-5)
[1.0 x 10-4, 1.0 x 10+1)
[1.0 x 10-5, 1.0 x 100)
[1.0 x 10+1, 3.402 x 1038)
[1.0 x 100, 3.402 x 1038)
Figure 3 shows the ice cloud extinction coefficient and ice water content
histograms at three altitude bins around the tropical region at night in
July 2008. The tropical region is defined as a region between latitude 23.5°
S to 23.5° N. The peak of higher altitude distribution has a smaller extinction
coefficient and ice water content. This suggests that the in-cloud extinction
coefficient and ice water content increase as the altitude decreases. Also,
the area of each distribution gives the number of observed ice clouds at that
altitude range. There are many more ice clouds observed between 13 km and 14 km
than those between 9 km and 10 km.
Figure 3. Histograms of ice cloud extinction coefficients and ice water
content observed at three altitude ranges around the tropical region. The X-axis
is in logarithm scale. The Y-axis is in linear scale. The vertical dashed line
separates negative and positive values.
In-cloud IWC and Grid-averaged IWC
In the lidar level ice cloud product, the medians ice cloud extinction coefficient or
ice water content (IWC) in each 3D spatial grid is reported as parameter
Extinction_Coefficient_532_Median and Ice_Water_Content_Median. As explained above,
there are 44 bins describing the full distribution of ice water content. The valid range of ice water content
is from -0.01 g·m-3 to 1.0·g m-3. The first
bin (bin 0) and the last bin (bin 43) are for ice water content less than -0.01 g·m-3
and larger than 1.0 g·m-3. There are also two bins 16 and 17
counting for very small ice water content whose magnitudes are less than 10-5. These
coefficients are typically less confident in the retrieval algorithm. However,
those extreme small values are included when reporting the median ice water content. Note these
two parameters describe the extinction coefficient/ice water content within ice clouds, in other
words, during ice clouds only.
If a user is interested in calculating the mean in-cloud ice water content,
it is recommended using the set of equations, with parameters Ice_Water_Content_Histogram,
Ice_Water_Content_Bin_Boundary (middle boundary for example). Here the number of
the accepted ice clouds does not exactly equal to Ice_Cloud_Accepted_Samples
since we do not include small or big outliers in bin 0, 16, 17 and 43.
It may be necessary to consider ice cloud occurrence when deriving the
grid-averaged ice water content. In this case, the occurrence of ice cloud in
each 3D grid is estimated as the ratio between number of accepted ice clouds
(excluded outliers) with the sum of cloud and cloud free samples. The following
set of equations can derive grid-averaged ice water content using the parameters
Ice_Water_Content_Histogram, Ice_Water_Content_Bin_Boundary, Cloud_Samples and
Figure 4 shows the in-cloud averaged ice water content and grid-averaged ice water content. As shown on
the left, the in-cloud averaged ice water content describes the capacity of ice water content an ice cloud
can hold at a specific altitude/temperature. This capacity increases as the
altitude decreases and the temperature increases. Considering the ice clouds
occurrence drops as the altitude decreases, the grid-averaged will increase first
and then decrease with altitude as shown on the right of Figure 4.
Figure 4. Zonal pattern of in-cloud averaged ice water content and grid-averaged
ice water content at night in July 2008. The latitude bin is 2° and the altitude bin is
120 m. The X-axis shows the laitutde and Y-Axis altitude. The ice water content is shown in
logarithm. The unit is mg·m-3.
The same discussions also apply to extinction coefficient. To derive the
grid-averaged extinction coefficient, use the same formula and replace the parameters
Ice_Water_Content_Histogram and Ice_Water_Content_Bin_Boundary with
Extinction_Coefficient_532_Histogram and Extinction_Coefficient_532_Bin_Boundary.
There may be instances when for a given 5 km profile (the fundamental resolution
used by the level 2 data) there is clear-air throughout the entire column without
valid lidar detected surface (no feature type 5 neither). This occurs when the
lowest cloud or aerosol layer in a 5 km profile touches the surface but yet fails
to pass the surface thickness threshold test set up in the lidar level 2 algorithm.
For these cases, the entire 5 km profile is excluded. These removed columns are
added to the Number_of_Bad_Profiles parameter contained in the file. In July 2008
month, the occurrences were about 0.08% during day and 0.007% at night. Since the
numbers are so small removing those bad profiles would not affect the level 3
Information of Processed Level 2 5 km Cloud Profile Product files
Information of the number of Level 2 5 km Cloud Profile Product files is
reported as Number_of_Level2_Files_Analyzed in metadata. The list of those files
is also provided in List_of_Input_Files. As shown in Figure 5, the number of
processed day granules typically equals that of night granules in each month.
Occasionally there would be one count off due to the possibility including the
last granule from the previous month for processing either day or night granules.
This granule has records for two consecutive days where some belonging to the
previous month and some to the current month.
It is worthy noticing that there are no level 3 ice cloud product files in
February 2016. During that month, the CALIPSO was turned off due to a GPS clock
Figure 5. The numbers of processed L2 5 km cloud profile product files
for the standard V1.00 level 3 ice cloud product.
In the metadata, a user could also find detailed information about grid information
and filter information in the Program_Configuration if interested.