NASA - National Aeronautics and Space Administration
ABOUT CALIPSO PRODUCTS OUTREACH DOCUMENTS RESOURCES TOOLS CONTACTS        

CALIPSO: Data User's Guide - Data Product Descriptions - Lidar Level 2 5 km Cloud and Aerosol Profile Products


CALIPSO HOMECALIPSO's User Guide HOMEData Summaries → Lidar Level 2 5 km Cloud and Aerosol Profile Products

Introduction

The CALIPSO Cloud and Aerosol Profile Products report profiles of particle extinction and backscatter and additional profile information (e.g., particulate depolarization ratios) derived from these fundamental products. Layer optical depths are reported in the Cloud and Aerosol Layer Products. The layer optical depths are derived from the same retrievals that are used to compute the extinction and backscatter profiles in the profile products. All of these extinction products are produced using the same basic algorithm (Young and Vaughan, 2009).

The version 3 CALIPSO data processing system now generates profile products for clouds and aerosols at the same spatial resolution. The cloud and aerosol profile products are reported at a uniform spatial resolution of 60-m vertically and 5-km horizontally, over a nominal altitude range from 20-km to -0.5-km for clouds, and over a broader altitude range of 30-km to -0.5-km for aerosols. Profile data for all features detected in the stratosphere are reported in the aerosol profile product, thus users seeking to obtain CALIPSO retrievals of backscatter and extinction profiles of polar stratospheric clouds are directed to the aerosol profile products. Due to constraints imposed by CALIPSO's on-board data averaging scheme, the vertical resolution of the aerosol profile data varies as a function of altitude. In the tropospheric region between 20-km to -0.5-km, the aerosol profile products are reported at a resolution of 60-m vertically, and in the stratospheric region (above 20-km), the aerosol profile products are reported at a resolution of 180-m vertically. In the text below we provide brief descriptions of individual data fields reported in the CALIPSO cloud and aerosol profile products. Where appropriate, we also provide an assessment of the quality and accuracy of the data in the current release.

Don't forget to check out the detailed data quality summary for this data product. (At end of page).


Standard and Expedited Data Set Definitions

Standard Data Sets:

Standard data processing begins immediately upon delivery of all required ancillary data sets. The ancillary data sets used in standard processing (e.g., GMAO meteorological data from the National Snow and Ice Data Center) must be spatially and temporally matched to the CALIPSO data acquisition times, and thus the time lag latency between data onboard acquisition and the start of standard processing can be on the order of several days. The data in each data set are global, but are produced in files by half orbit, with the day portion of an orbit in one file and the night portion of the orbit in another.

Expedited Data Sets:

Expedited data are processed as soon as possible after following downlink from the satellite and delivery to Langley Research Center (LaRC). Latency between onboard acquisition and analysis expedited processing is typically on the order or 6 to 28 hours. Expedited processing uses the most recently current available set of ancillary data (e.g., GMAO meteorological profiles) and calibration coefficients available, which may lag the CALIPSO data acquisition time/date by several days.

Expedited data files contain at the most, 90 minutes of data. Therefore, each file may contain both day and night data.

NOTE: Users are strongly cautioned against using Expedited data products as the basis for research findings or journal publications. Standard data sets only should be used for these purposes.

The differences between expedited processing and standard processing are explained in more detail in Adapting CALIPSO Climate Measurements for Near Real Time Analyses and Forecasting (PDF).


Data Descriptions

In the text below we provide brief descriptions of individual data fields reported in the CALIPSO cloud and aerosol profile products. Where appropriate, we also provide an assessment of the quality and accuracy of the data in the current release. The data descriptions are grouped into several major categories, as follows:

Additionally all the science data sets (SDSs) are listed in the table to the right, click on the SDS name to go directly to the description.

Science Data Set (SDS) Data Maturity
Latitude NA
Longitude NA
Profile_Time NA
Profile_UTC_Time NA
Day_Night_Flag NA
Column_Optical_Depth_Cloud_532 Provisional
Column_Optical_Depth_Cloud_Uncertainty_532 Provisional
Column_Optical_Depth_Aerosols_532 Provisional
Column_Optical_Depth_Aerosols_Uncertainty_532 Provisional
Column_Optical_Depth_Stratospheric_532 Provisional
Column_Optical_Depth_Stratospheric_Uncertainty_532 Provisional
Column_Optical_Depth_Aerosols_1064 Provisional
Column_Optical_Depth_Aerosols_Uncertainty_1064 Provisional
Column_Optical_Depth_Stratospheric_1064 Provisional
Column_Optical_Depth_Stratospheric_Uncertainty_1064 Provisional
Column_Feature_Fraction ValStage1
Column_Integrated_Attenuated_Backscatter_532 ValStage1
Column_IAB_Cumulative_Probability ValStage1
Tropopause_Height external
Tropopause_Temperature external
Temperature external
Pressure external
Molecular_Number_Density external
Relative_Humidity external
Surface_Elevation_Statistics ValStage1
Surface_Winds external
Samples_Averaged NA
Aerosol_Layer_Fraction ValStage1
Cloud_Layer_Fraction ValStage1
Atmospheric_Volume_Description ValStage1
Extinction_QC_Flag_532 Provisional
Extinction_QC_Flag_1064 Provisional; aerosol products only
CAD_Score ValStage1
Total_Backscatter_Coefficient_532 Provisional
Total_Backscatter_Coefficient_Uncertainty_532 Provisional
Perpendicular_Backscatter_Coefficient_532 Provisional
Perpendicular_Backscatter_Coefficient_Uncertainty_532 Provisional
Particulate_Depolarization_Ratio_Profile_532 Provisional
Particulate_Depolarization_Ratio_Uncertainty_532 Provisional
Extinction_Coefficient_532 Provisional
Extinction_Coefficient_Uncertainty_532 Provisional
Aerosol_Multiple_Scattering_Profile_532 Provisional
Backscatter_Coefficient_1064 Provisional; aerosol products only
Backscatter_Coefficient_Uncertainty_1064 Provisional; aerosol products only
Extinction_Coefficient_1064 Provisional; aerosol products only
Extinction_Coefficient_Uncertainty_1064 Provisional; aerosol products only
Aerosol_Multiple_Scattering_Profile_1064 Provisional; aerosol products only


Column Time Parameters

Profile Time
Time, expressed in International Atomic Time (TAI). Units are in seconds, starting from January 1, 1993. For the 5 km profile products, three values are reported: the time for the first pulse included in the 15 shot average; the time at the temporal midpoint (i.e., at the 8th of 15 consecutive laser shots); and the time for the final pulse.

Profile UTC
Similar to Profile Time, but for time expressed in Coordinated Universal Time (UTC), and formatted as 'yymmdd.ffffffff', where 'yy' represents the last two digits of year, 'mm' and 'dd' represent month and day, respectively, and 'ffffffff' is the fractional part of the day.

Day Night Flag
Indicates the lighting conditions at an altitude of ~24 km above mean sea level; 0 = day, 1 = night.

Column Geolocation Information

Latitude
Geodetic latitude, in degrees, of the laser footprint. For the 5 km profile products, three values are reported: the footprint latitude for the first pulse included in the 15 shot average; the footprint latitude at the temporal midpoint (i.e., at the 8th of 15 consecutive laser shots); and the footprint latitude for the final pulse.

Longitude
Longitude, in degrees, of the laser footprint. For the 5 km profile products, three values are reported: the footprint longitude for the first pulse included in the 15 shot average; the footprint longitude at the temporal midpoint (i.e., at the 8th of 15 consecutive laser shots); and the footprint longitude for the final pulse.

Column Optical Properties

Column Optical Depth Cloud 532 (Provisional)
Column Optical Depth Aerosols 532 (Provisional)
Column Optical Depth Aerosols 1064 (Provisional)
Column Optical Depth Stratospheric 532 (Provisional)
Column Optical Depth Stratospheric 1064 (Provisional)

Optical depth of all clouds, aerosol, or stratospheric layers within a 5 km column. The optical depths are obtained by integrating the 532 nm cloud/aerosol/stratospheric extinction profile, reported in these profile products. For aerosols and stratospheric layers the optical depth is provided at both 532 and 1064 nm wavelengths.

The column optical depth uncertainties provide the only unique QA indicator for each column optical depth product; no other QA or data quality flags are provided. Therefore, users are strongly encouraged to make use of the Extinction_QC ,CAD_Score, cloud water phase, cloud water phase QA values (both found in the Feature_Classification_Flags in the layer products or the Atmospheric_Volume_Description in the profile products), Column_IAB_Cumulative_Probability, and feature opacity information to make their own quality assessment of the column depths.

The column optical depths are a column integral product. Any large uncertainties or poor extinction retrievals from layers within the column (i.e. clouds, aerosols, or stratospheric features) will propogate downward and may impact the quality of all (i.e. cloud, aerosol, and stratospheric) the column optical depths in the column. The following paragraphs outline notes with regard to specific data products that users should be aware of when using column optical depth data.

Opacity: We remind data users that the CALIPSO lidar is only capable of penetrating to the surface if the total column optical depth is less than ~5. (Note that this value takes into account the contribution of multiple scattering.) If the column is opaque to the lidar, then the reported column optical depths are a measure to the apparent base of the lowest feature observed. Feature opacity can be determined by inspecting the extinction QC flag for the lowest extinction coefficient in any 5-km column; if bit 5 (value = 16) is set then the feature is totally attenuating.

Extinction QC: The extinction QC values in the column should be examined to determine if any of the extinction retrievals were bad. Users are reminded that any poor extinction retrievals in the column may impact the quality of all column optical depths. In general, solutions where the final lidar ratio is unchanged (extinction QC = 0) or the extinction solution is constrained (extinction QC = 1) yield physically plausible solutions more often. Conversely, retrievals tend to be more uncertain in those cases where the lidar ratio for either wavelength must be reduced.

CAD_Score and feature sub-type: Features with low absolute CAD_Scores, "special" CAD_Scores, or uncertain aerosol type classifications may impact the quality of the column optical depths. For example, if the top-most feature in the column has a low absolute CAD_Score it is possible that the assigned lidar ratio may be incorrect; this would impact the extinction retrieval for that feature which would lead to an incorrect rescaling of all the data below that feature.

Cloud phase: If there are clouds in the column that are found to have horizontal oriented ice (HOI) crystals it is likely that the quality of the column optical depths are low. The anomalously high backscatter from HOI clouds generally makes the extinction retrieval more difficult. Because all the data below the HOI cloud is rescaled by the retrieved optical depth, the extinction data below could be suspect.


Column Optical Depth Cloud Uncertainty 532 (Provisional)
Column Optical Depth Aerosols Uncertainty 532 (Provisional)
Column Optical Depth Stratospheric Uncertainty 532 (Provisional)
Column Optical Depth Aerosols Uncertainty 1064 (Provisional)
Column Optical Depth Stratospheric Uncertainty 1064 (Provisional)

Estimated uncertainty in the column optical depth at each wavelength, computed according to the formulas give in the CALIPSO Version 3 Extinction Uncertainty Document (PDF). Ignoring multiple scattering concerns for the moment, errors in column optical depth calculations typically arise from three main sources: signal-to-noise ratio (SNR) within a layer, calibration accuracy, and the accuracy of the lidar ratio specified for use in the solution. Except for constrained solutions, where a lidar ratio estimate can be obtained directly from the attenuated backscatter data, lidar ratio uncertainties are almost always the dominant contributor to optical depth uncertainties, and the relative error in the layer optical depth will always be at least as large as the relative error in the layer lidar ratio.

Calculation of the layer optical depth uncertainty is an iterative process. On some occasions when the SNR is poor, or an inappropriate lidar ratio is being used, the iteration will attempt to converge asymptotically to positive infinity. Whenever this situation is detected, the iteration is terminated, and the layer optical depth uncertainty is assigned a fixed value of 99.99. Any time an uncertainty of 99.99 is reported, the extinction calculation should be considered to have failed. The associated optical depths cannot be considered reliable, and should therefore be excluded from all science studies.

Note: optical depth uncertainties are reported as absolute errors, not relative errors.


Column Integrated Attenuated Backscatter 532 (ValStage1)
The integral with respect to altitude of the 532 nm total attenuated backscatter coefficients. The limits of integration are from the onset of the backscatter signal at ~40-km, down to the range bin immediately prior to the surface elevation specified by the digital elevation map. This quantity represents the total attenuated backscatter measured within a column. Physically meaningful values of the column integrated attenuated backscatter (hereafter, γ′column) range from ~0.01 sr (completely clear air), to greater than 1.5 sr (e.g., due to anomalous backscatter from horizontally oriented ice crystals; see Hu et al. (Optics Express 15, 2007)).

Column IAB Cumulative Probability (ValStage1)
The cumulative probability of measuring a total column integrated attenuated backscatter value equal to the value computed for the current profile. Values in this field range between 0 and 1. The cumulative probability distribution function, shown below in Figure 1, was compiled using all CALIOP total column IAB measurements acquired between 15 June, 2006 and 18 October, 2006.

Figure 3: Distribution of γ′column at 532 nm
Column IAB distribution at 532 nm.

Column Meteorological Data

Tropopause Height (external)
Mean tropopause height, in kilometers above local mean sea level; derived from the GEOS-5 data product provided to the CALIPSO project by the GMAO Data Assimilation System

Tropopause Temperature (external)
Mean tropopause temperature, in degrees C; derived from the GEOS-5 data product provided to the CALIPSO project by the GMAO Data Assimilation System

Surface Winds (external; aerosol products only)
Provides the mean zonal and meridional component of the surface wind speed computed over the horizontal distance spanned by the averaged profile; units are meters per second.

Profile Meteorological Data

Temperature (external)
Mean temperature, in degrees C, reported for the midpoint of each range bin in the profile; derived from the GEOS-5 data product provided to the CALIPSO project by the GMAO Data Assimilation System

Pressure (external)
Mean pressure, in hectopascals, reported for the midpoint of each range bin in the profile; derived from the GEOS-5 data product provided to the CALIPSO project by the GMAO Data Assimilation System

Molecular Number Density (external)
Mean molecular number density, in molecules per cubic meter, reported for the midpoint of each range bin in the profile; derived from the GEOS-5 data product provided to the CALIPSO project by the GMAO Data Assimilation System

Relative Humidity (external)
Mean relative humidity, reported for the midpoint of each range bin in the profile; derived from the GEOS-5 data product provided to the CALIPSO project by the GMAO Data Assimilation System

Surface Elevation Statistics

Surface Elevation Statistics (ValStage1)
Provides the minimum, maximum, mean, and standard deviation of the surface elevation obtained from the GTOPO30 digital elevation map (DEM) for the horizontal distance spanned by the averaged profile; units are kilometers.

Feature Spatial Information Within Column

Column Feature Fraction (ValStage1)
The fraction of the 5-km horizontally averaged profile, between 30-km and the DEM surface elevation, which has been identified as containing a feature (i.e., either a cloud, an aerosol, or a stratospheric layer)

Samples Averaged
Specifies the number of full resolution samples averaged for each profile range bin; for the purposes of this computation, 'full resolution' is taken to mean 30 meters vertically, and a single shot (~1/3-km) horizontally. Thus a single range bin below an altitude of ~20.2 km (resolution = 60-m vertical, 5-km horizontal) will have at most 480 samples averaged (i.e., for those layers that required 80-km averaging for detection, 240 shots horizontally by two 30-m range bins vertically).

Aerosol Layer Fraction (ValStage1)
Cloud Layer Fraction (ValStage1)

Reports the fraction (by area) within each 5 km horizontal * 60 m vertical range bin containing aerosols or clouds in the profile products. The Aerosol and Cloud Layer Fractions are conceptually identical and the same procedure is used to calculate both quantities.

Since the array elements of the profile products can be larger than the native resolution of the extinction retrieval (5 km * 30 m vs. 5 km * 60 m in the lower troposphere), and because atmospheric features can be identified at horizontal resolutions of 1 km and 1/3 km, the atmospheric composition within each profile range bin is not guaranteed to be homogeneous. Thus, the Aerosol and Cloud Layer Fractions report the fraction (by area) of each 5 km * 60 m profile range bin identified as containing aerosols or clouds by the Scene Classification Algorithms. By referencing Cloud Layer Fraction, the fractional amount of cloud clearing performed within each profile range bin of aerosol backscatter and extinction can be determined.

Figure 4: Cloud clearing scenarios for strongly scattering clouds detected at single shot resolution. Red indicates clouds detected at 1/3 km resolution, blue indicates clouds found at 1 km or coarser resolution, yellow indicates an aerosol layer found at 5 km resolution, and white indicates clear air. Scenarios: Clouds embedded in aerosol (upper panel), clouds embedded in clear air (middle panel), and dense clouds embedded in within a weakly scattering cloud layer (lower panel). Each row extends 5 km horizontally and 30 m vertically. Each column extends 1/3 km horizontally.
Cloud/Aerosol Layer fraction example.

Shown in Figure 4 are 3 possible scenarios illustrating how the Cloud Layer Fraction would be reported for each 5 km x 60 m cloud profile range bin. There are at most 30 single shot (1/3 km x 30 m) cloud layers in each 5 km x 60 m cloud profile range bin - fifteen 1/3 km horizontally and two 30 m vertically. In the top panel, red indicates clouds found at 1/3 km resolution and the yellow indicates an aerosol layer found at a 5 km horizontal resolution after the 1/3 km clouds had been removed. In this case, the cloud fraction for the top row would be 11/30 = 0.36. In the middle panel, no features were detected at any coarser spatial resolution after the 1/3 km features were removed. The Atmospheric Volume Description for the 5 km average would report the cell as being "clear air", but the cloud fraction for the top row would still be 11/30 = 0.36. In the lower panel, a cloud was detected in the data remaining after all 1/3 km features had been removed. In this case the cloud fraction would be 1 for all rows shown.

The Aerosol and Cloud Layer Fractions must be values between zero and one, yet both layer fractions are reported as integers between 0 and 30. For example, a value for the Aerosol_Layer_Fraction reported as 11 would indicate a fraction of 11/30 = 0.367.


Atmospheric Volume Description (ValStage1)

Atmospheric Volume Description is a profile descriptive flag containing theFeature Classification Flags associated with each 5 km x 60 m (or 5 km x 180 m) range bin in the Profile Products. The Feature Classification Flags provide assessments of (a) feature type (e.g., cloud vs. aerosol vs. stratospheric layer); (b) feature subtype; (c) layer ice-water phase (clouds only); and (d) the amount of horizontal averaging required for layer detection. Note that the interpretation of final three bits in the atmospheric volume description (i.e., the averaging required for detection) is slightly different from the interpretation that would be used for the feature classification flags. These differences are summarized in the table below.

Value Atmospheric volume description Feature Classification flag
0 not applicable not applicable
1 5 km 1/3 km
2 20 km 1 km
3 80 km 5 km
4 5 km w/ subgrid feature detected at 1/3 km 20 km
5 20 km w/ subgrid feature detected at 1/3 km 80 km
6 80 km w/ subgrid feature detected at 1/3 km not used
7, 8 not used not used


How profile descriptive flags are stored

Atmospheric Volume Description, CAD Score and Extinction QC [532|1064] are all profile descriptive flags that are stored in the Level 2 Profile Products in the same manner explained here.

Ideally, each profile descriptive flag would be an array of the size [# altitude bins, # profiles] with each array element providing a complete description of the range-resolved atmospheric state. However, because the range resolution of the Level 1 profile data below ~8.3-km is 30 m, and because the feature-finder, scene classification, and extinction algorithms all operate at this finer spatial resolution, providing a genuinely complete description of the atmospheric state for each 60 m Level 2 range bin requires that the profile descriptive flags be stored as 3-D arrays of the size [#profiles, # altitude bins, 2]. The first dimension, [ : , : , 1], corresponds to the standard altitude array of the Profile Products. Thus, below 8.3 km, the first dimension contains the descriptive flags of the higher of the two full resolution (30 m) bins that comprise the single 60 m bin reported in the Profile Products. Meanwhile, below 8.3 km, the second dimension [: , : , 2] contains the descriptive flags for the lower of the two 30 m range bins. Above 8.3 km, where the range resolution of the Level 1 data is 60 m or greater, the descriptive flags for each single 60 m (or 180 m) range bin are replicated in both array elements.

Figure 5: Wholly fictitious but heuristically useful schematic of layer detection results for a data segment extending 80-km horizontally and 480-m vertically. Yellow/orange/brown indicates an aerosol layer detected at horizontal averaging resolutions of, respectively, 80, 20 or 5 km. Shades of blue likewise represent clouds at 80, 20, and 5 km. Red represents a surface detected layer at 5 km, and the white regions are (presumably) clear air, where no features were found. The right-hand side of the figure shows the atmospheric volume descriptor for columns 1 and 16.
Atmospheric volume description example.

Profile QA Information

Extinction QC Flag 532 (Provisional)
Extinction QC Flag 1064 (Provisional; aerosol products only)

Note that this field is stored in a similar format as the Atmospheric Volume Descriptor.

The extinction QC flags are bit-mapped 16-bit integers, reported for each layer and for each wavelength for which an extinction retrieval was attempted. Aerosol extinction is computed for both wavelengths; cloud extinction is only reported at 532 nm. The information content of each bit is as follows

Bit Value Interpretation
1 0 unconstrained retrieval; initial lidar ratio unchanged during solution process
1 1 constrained retrieval
2 2 Initial lidar ratio reduced to prevent divergence of extinction solution
3 4 Initial lidar ratio increased to reduce the number of negative extinction coefficients in the derived solution
4 8 Calculated backscatter coefficient exceeds the maximum allowable value
5 16 Layer being analyzed has been identified by the feature finder as being totally attenuating (i.e., opaque)
6 32 Estimated optical depth error exceeds the maximum allowable value
7 64 Solution converges, but with an unacceptably large number of negative values
8 128 Retrieval terminated at maximum iterations
9 256 No solution possible within allowable lidar ratio bounds
16 32768 Fill value or no solution attempted

The bit assignments are additive, so that (for example) an extinction QC value of 18 represents an unconstrained retrieval (bit 1 is NOT set) for which the lidar ratio was reduced to prevent divergence (+2; bit 2 is set), and for which the feature finder has indicated that the layer is opaque (+16; bit 5 is set). For the version 2.01 release, bits 10-15 are not used. Complete information about the conditions under which each extinction QC bit is toggled can be found in the CALIPSO Extinction Retrieval ATBD (PDF)


CAD Score (ValStage1)

Note that this field is stored in a similar format as the Atmospheric Volume Descriptor.

The cloud-aerosol discrimination (CAD) score, which is reported in the 1-km and 5-km layer products, and now in version 3, the 5 km cloud and aerosol profile products, provides a numerical confidence level for the classification of layers by the CALIOP cloud-aerosol discrimination algorithm. The CAD algorithm separates clouds and aerosols based on multi-dimensional histograms of scattering properties (e.g., intensity and spectral dependence) as a function of geophysical location. In areas where there is no overlap or intersection between these histograms, features can be classified with complete confidence (i.e., |CAD score| = 100).

In the current release (version 3), the CAD algorithm uses newly developed five-dimensional (5D) probability density functions (PDFs), rather than the three-dimensional (3D) PDFs used in previous versions. In addition to the parameters used in the earlier 3D version of the algorithm (layer mean attenuated backscatter at 532 nm, layer-integrated attenuated backscatter color ratio, and altitude), the new 5D PDFs also include feature latitude and the layer-integrated volume depolarization ratio. Detailed descriptions of the CAD algorithm can be found in Sections 4 and 5 of the CALIPSO Scene Classification ATBD (PDF). Enhancements made to incorporate the 5D PDFs used in version 3 release are described in Liu et al., 2010 (PDF). For further information on the CAD algorithm architecture and the three-dimensional (3D) PDFs used in versions 1 and 2 of the data products, please see Liu et al., 2004 (PDF) and/or Liu et al., 2009.

The standard CAD scores reported in the CALIPSO layer products range between -100 and 100. The sign of the CAD score indicates the feature type: positive values signify clouds, whereas negative values signify aerosols. The absolute value of the CAD score provides a confidence level for the classification. The larger the magnitude of the CAD score, the higher our confidence that the classification is correct. An absolute value of 100 therefore indicates complete confidence. Absolute values less that 100 indicate some ambiguity in the classification; that is, the scattering properties of the feature are represented to some degree in both the cloud PDF and in the aerosol PDF. In this case, a definitive classification cannot be made; that is, although we can provide a "best guess" classification, this guess could be wrong, with a probability of error related to the absolute value of the CAD score. A value of 0 indicates that a feature has an equal likelihood of being a cloud and an aerosol. Users are encouraged to refer to the CAD score when the cloud and aerosol classification results are used and interpreted.

Beginning with the version 2.01 release, several "special" CAD score values have been added. These are listed in the table below. Each of these new values represents a classification result that is based on additional information beyond that normally considered in the standard CAD algorithm.

CAD score Interpretation
-101 negative mean attenuated backscatter encountered; layer is most likely an artifact, and its spatial and optical properties should be excluded from all science analyses.
101 initially classified as aerosol, but layer integrated depolarization mandates classifying layer as cloud (version 2 only; obsolete in version 3)
102 layer exhibits very high integrated backscatter and very low depolarization characteristic of oriented ice crystals (version 2 only; obsolete in version 3)
103 layer integrated attenuated backscatter at 532 nm is suspiciously high; feature authenticity and classification are both highly uncertain
104 layers with CAD scores of 104 are boundary layer clouds that were found to be opaque at the initial 5-km horizontal averaging resolution used by the layer detection algorithm; however, these layers are not uniformly filled with high-resolution clouds (i.e., layers detected at a 1/3-km horizontal resolution), and the 532 nm mean attenuated backscatter coefficient of the data that remains after cloud clearing is negative. Studies examining the spatial properties and distributions of clouds can safely include the spatial properties of these layers; however, the associated measured and derived optical properties should be excluded from all science studies.
105 a CAD score of 105 designates a layer detected at one of the coarser averaging resolutions (20-km or 80-km) for which the initial estimates of measured properties have been negatively impacted by either (a) the attenuation corrections applied to account for the optical depths of overlying layers, or (b) the extension of the layer base altitude are boundary layer clouds that were found to be opaque at the initial 5-km horizontal averaging resolution used by the layer detection algorithm; however, these layers are not uniformly filled with high-resolution clouds (i.e., layers detected at a 1/3-km horizontal resolution), and the 532 nm mean attenuated backscatter coefficient of the data that remains after cloud clearing is negative. Studies examining the spatial properties and distributions of clouds can safely include the spatial properties of these layers; however, the associated measured and derived optical properties should be excluded from all science studies.

Profile Extinction and Backscatter Coefficients

Total Backscatter Coefficient 532 (Provisional)
Backscatter Coefficient 1064 (Provisional; aerosol products only)
Particulate total backscatter coefficients reported for each profile range bin in which the appropriate particulates (i.e., clouds or aerosols) were detected; those range bins in which no particulates were detected contain fill values (-9999). Units are kilometers-1 steradians-1. For the 532 nm data, the particulate total backscatter coefficients are derived from the sum of the parallel and perpendicular backscatter measurements recorded aboard the CALIPSO satellite (i.e., β532 total = β532 parallel + β532 perp).

Total Backscatter Coefficient Uncertainty 532 (Provisional)
Total Backscatter Coefficient Uncertainty 1064 (Provisional; aerosol products only)
Uncertainty in the particulate total backscatter coefficients reported for each profile range bin in which the appropriate particulates were detected; these are absolute uncertainties, not relative, thus the units are identical to the units of the total backscatter coefficients (i.e., kilometers-1 steradians-1); those range bins in which no particulates were detected contain fill values (-9999). Uncertainties are computed according to the procedures described in the CALIPSO Version 3 Extinction Uncertainty Document (PDF).

Perpendicular Backscatter Coefficient 532
Particulate backscatter coefficients derived from the 532 nm perpendicular channel measurements, reported for each profile range bin in which the appropriate particulates (i.e., clouds or aerosols) were detected; those range bins in which no particulates were detected contain fill values (-9999). Units are kilometers-1 steradians-1.

Perpendicular Backscatter Coefficient Uncertainty 532
Uncertainty in the perpendicular channel backscatter coefficients reported for each profile range bin in which the appropriate particulates were detected; these are absolute uncertainties, not relative, thus the units are identical to the units of the total backscatter coefficients (i.e., kilometers-1 steradians-1); those range bins in which no particulates were detected contain fill values (-9999). Uncertainties are computed according to the procedures described in the CALIPSO Version 3 Extinction Uncertainty Document (PDF).

Extinction Coefficient 532 (Provisional)
Extinction Coefficient 1064 (Provisional; aerosol products only)
Particulate extinction coefficients reported for each profile range bin in which the appropriate particulates (i.e., clouds or aerosols) were detected; those range bins in which no particulates were detected contain fill values (-9999). Units are kilometers-1.

Extinction Coefficient Uncertainty 532 (Provisional)
Extinction Coefficient Uncertainty 1064 (Provisional; aerosol products only)
Uncertainty in the particulate extinction coefficients reported for each profile range bin in which the appropriate particulates were detected; these are absolute uncertainties, not relative, thus the units are identical to the units of the particulate extinction coefficients (i.e., kilometers-1); those range bins in which no particulates were detected contain fill values (-9999). Uncertainties are computed according to the procedures described in the CALIPSO Version 3 Extinction Uncertainty Document (PDF)

Particulate Depolarization Profiles

Particulate Depolarization Ratio Profile 532 (Provisional)
The particulate depolarization ratio, δp(z), is a post-extinction quantity, calculated from ratio of the layer integrated perpendicular and parallel polarization components of particulate backscatter coefficient at a given altitude z, using
Particulate Depol.
Here β⊥,P and β||,P perpendicular and parallel components of particulate backscatter coefficient at 532 nm, respectively. The quality of the estimate for δp is determined not only by the SNR of the backscatter measurements in parallel and perpendicular channels, but also the accuracy of the range-resolved two-way transmittance estimates within the layer. The two-way transmittances due to molecules and ozone can be well characterized via the model data obtained from the GMAO. The two-way transmittances due to particulates, however, are only as accurate as the CALIOP extinction retrieval. Opaque cirrus cloud layers can be particularly prone to errors in the particulate depolarization ratio, as very large attenuation corrections are applied to the weak signals at the base of the layers, and on those occasions where one channel or the other becomes totally attenuated, this situation can generate very large, negative particulate depolarization ratio estimates. For layers that are not opaque, δp is generally reliable. However, in weakly scattering layers, the quality of the daytime estimate can be degraded by a factor of 2-4 due to the larger background noise compared with the nighttime estimate.

Particulate Depolarization Ratio Uncertainty 532 (Provisional)
The uncertainties reported for the particulate depolarization ratios provide an estimate for random error in the particulate depolarization ratio for each range bin (i.e., the ratio of perpendicular and parallel components of retrieved particulate backscatter coefficient within the feature). Based on an assessment of several days of test data (January 1-3, 2007), the uncertainty for aerosol profile products is typically (a median value) ~0.18 and ~0.7 for nighttime and daytime measurements, respectively. For cloud profile products, it is typically ~0.33 and ~0.58 during night and day, respectively.

Multiple Scattering Profiles

Aerosol Multiple Scattering Profile 532 (aerosol products only)
Aerosol Multiple Scattering Profile 1064 (aerosol products only)
Cloud Multiple Scattering Profile 532 (cloud products only)
The multiple scattering profiles, η532(z) and η1064(z), are specified at each wavelength according to layer type and subtype. Values range between 0 and 1; 1 corresponds to the limit of single scattering only, with smaller values indicating increasing contributions to the backscatter signal from multiple scattering. Multiple scattering effects are different in aerosols, ice clouds, and water clouds. A discussion of multiple scattering factors for ice clouds and several aerosol types can be found in Winker, 2003 (PDF). Multiple scattering in water clouds is discussed in Winker and Poole (1995).

Ice clouds: for the CALIOP viewing geometry, simulations show multiple scattering effects are nearly independent of range and, as parameterized in the CALIOP retrieval algorithm (Winker et al. 2009), are nearly independent of extinction. In version 2, ice clouds were assigned a range-independent multiple scattering factor of η532 = 0.6. Validation comparisons indicate this is an appropriate value and the same value is used in version 3.

Water clouds: in version 2 the multiple scattering factor for water clouds was set to unity, resulting in large errors in retrievals of extinction and optical depth. In version 3, a value of η532 = 0.6 is used. Based on Monte Carlo simulations of multiple scattering, this value appears to be appropriate for semitransparent water clouds (τ < 1). (It is purely coincidental this is the same value used for ice clouds.) For denser water clouds (τ > 1) the multiply-scattered component of the signal becomes much larger than the single-scattered component, η532 becomes dependent on both cloud extinction and range into the cloud, and the retrieval becomes very sensitive to errors in the multiple scattering factor used. In these cases the multiple scattering cannot be properly accounted for in the current retrieval algorithm and retrieval results are unreliable.

Aerosols: simulations of multiple scattering effects on retrievals of aerosol layer optical depth indicate the effects are small in most cases. There is uncertainty in these estimates, however, due to poor knowledge of aerosol scattering phase functions. Validation comparisons conducted to date do not indicate significant multiple scattering effects on aerosol extinction profile retrievals. Multiple scattering effects may become significant in dense aerosol layers (σ > 1 /km), but in these cases retrieval errors are usually dominated by uncertainties in the lidar ratio or failure to fully penetrate the layer. In version 3, as in version 2, multiple scattering factors for both wavelengths are set to unity.

IWC Profiles

Ice Water Content Profile (Provisional; cloud products only)

Ice water content (IWC) is reported for all ice clouds detected in the CALIPSO measurements. IWC values of 0-0.54 gm,-3 account for 99.5% of the values measured by CALIOP. Above 8.25 km, IWC values > 0.767 gm-3 cannot be measured.

Background: Cloud ice water content is a provisional data product that is calculated as a parameterized function of the CALIOP retrieved extinction by ice cloud particles:

(1) IWC Equation.

Here, σ is the 532 nm volume extinction coefficient in km-1, and c0 = 119 gm-3 and c1 = 1.22 are coefficients derived from an observed empirical relationship between lidar extinction and an extensive set of in situ measurements of cloud particle properties from numerous field campaigns [1]. The relationship between 532 nm extinction and IWC was developed using IWC data between 0-1.0 gm-3 with temperatures between -70 and 0 °C. Cloud ice amount has been shown to vary with temperature, cloud particle size distribution, and by location inside a cloud. A temperature-dependent parameterization is being considered and tested for the next CALIOP data release. The effect of particle size distribution on IWC as seen by CALIOP is also currently being evaluated by comparison with in situ cloud data. Preliminary results show that CALIOP IWC has sufficient spatial resolution and precision to realistically resolve cloud morphology. A more detailed preliminary evaluation of the CALIOP version 3 IWC is available as an ILRC extended abstract [2], which includes CALIOP IWC probability distributions and example browse images. For a brief discussion containing critical information needed to intelligently use CALIOP IWC, please see the "data screening" section, below.

Resolution
IWC is reported at 60 m vertically, with a horizontal spatial resolution of 5 km along-track, and effectively the width of the laser beam across-track.
Precision
The precision of IWC is directly linked to the precision of the associated extinction retrieval. The precision of the extinction retrieval is ultimately limited by signal-to-noise ratio, and this varies between night and day and according to the overhead two-way 532 nm transmission. Therefore, the precision of CALIOP IWC has to be evaluated for each individual case. The team is currently developing a best-case precision estimate for nighttime high altitude Cirrus clouds.
Accuracy
Because this is a provisional data product, assessment of IWC accuracy is currently underway. This assessment can be approached in two different ways; (1) by establishing the accuracy of the 532 nm extinction on which it is based, or (2) by assessing the IWC product directly. Direct comparison of CALIOP IWC with other measured IWC values includes evaluation of both the extinction retrieval and the IWC parameterization.
Data screening
CALIOP IWC is a highly derived data product. Besides the parameterization, it relies on these activities:
Cloud determination
Bits 6 and 7 in the atmospheric volume descriptor indicates a feature of type 2=cloud, determined using 5-dimensional probability distribution functions as described in [3].
Cloud phase determination
The atmospheric volume descriptor indicates that the cloud phase is 1=randomly oriented ice (ROI) or 3=horizontally oriented ice (HOI) as determined for cloud particles (type=2) using temperature and depolarization [4]. Although extinction is available, IWC is not calculated for cloud particle phase 2=water. Users should use caution with HOI data because the preferred horizontal orientation of ice particles causes anomalously large backscatter that makes the extinction retrievals more difficult. This in turn may affect the accuracy of the IWC.
Extinction retrievals
Users that wish to understand the nuances and details of CALIOP extinction retrievals are referred to [5]. Extinction retrievals that are constrained provide the most accuracy. Unconstrained extinction retrievals require an a priori estimate of the extinction to backscatter ratio. If the a priori is adjusted significantly by the extinction retrieval algorithm then the retrieval becomes less accurate. The extinction quality flag provides this information about the extinction retrievals. An IWC user may wish to use retrievals with a converged (extinction flag=1) or near a priori (extinction flag=0) solution.

To summarize, users that do not wish to dig more deeply are recommended to use data of type=2 with phase=1 (atmospheric volume descriptor) and an extinction quality flag value of 0 or 1. IWC data from the highest cloud layer (highest signal-to-noise ratio), relative uncertainty of < 2 and IWC of < 0.026 gm-3 are likely to be the most reliable.

References:

  1. Heymsfield, A. J., D. Winker and G.-J. van Zadelhoff, 2005: "Extinction-ice water content-effective radius algorithms for CALIPSO", Geophysical Research Letters, 32, L10807, pp. 1-4.
  2. Avery, M., D. Winker, M. Vaughan, S. Young, R. Kuehn, Y. Hu, J. Tackett, B. Getzewich, Z. Liu, A. Omar, K. Powell, C. Trepte, and K.-P. Lee, (2010), "A first look at CALIOP/CALIPSO cloud ice water content", submitted to the 25th International Laser and Radar Conference.
  3. Liu, Z., et. al. 2009: "The CALISPO Lidar Cloud and Aerosol Discrimination: Version 2 Algorithm and Initial Assessment of Performance", Journal of Atmospheric and Oceanic Technology, 26, pp. 1198-1213.
  4. Hu, Y., et. al., 2009: "CALIPSO/CALIOP Cloud Phase Discrimination Algorithm", Journal of Atmospheric and Oceanic Technology, 26, pp. 2293-2309.
  5. Young, S. A. and M. A. Vaughan, 2009: "The Retrieval of Profiles of Particulate Extinction from Cloud-Aerosol Lidar Infrared Pathfinder Satellite Observations (CALIPSO) Data: Algorithm Description", Journal of Atmospheric and Oceanic Technology, 26, pp. 1105-1119.
  6. Protat, A., J. Delanoe, E.J. O’Connor and T.S.L’Ecuyer, 2010: "The evaluation of CloudSat and CALIPSO ice microphysical products using ground-based cloud radar and lidar observations", Journal of Atmospheric and Oceanic Technology, accepted, pp. 1-50.

Ice Water Content Profile Uncertainty (Provisional; cloud products only)

IWC uncertainty has a range of 0-99.99 gm-3, and is derived directly from the extinction retrieval uncertainty. The IWC fractional uncertainty probability distribution is shown in [1]. This is the estimated CALIOP measurement uncertainty, and does not characterize uncertainty in the IWC parameterization. Users wishing to understand extinction retrieval uncertainty should consult the summary descriptions at the top of this page.

Comparison of CALIOP IWC with in situ data from various field campaigns, and with CPL lidar data is underway. Further, we are working on comparisons between CALIOP IWC and IWC from CLOUDSAT and MLS. We anticipate sharing the results from these comparisons in future quality summary updates.

Future IWC parameterizations may include temperature dependency, based on further in situ data comparisons. Currently suggested temperature-dependant parameterizations do not produce good results for high-altitude tropical clouds [6] and therefore are not used.

References:

  1. Heymsfield, A. J., D. Winker and G.-J. van Zadelhoff, 2005: "Extinction-ice water content-effective radius algorithms for CALIPSO", Geophysical Research Letters;, 32, L10807, pp. 1-4.
  2. Avery, M., D. Winker, M. Vaughan, S. Young, R. Kuehn, Y. Hu, J. Tackett, B. Getzewich, Z. Liu, A. Omar, K. Powell, C. Trepte, and K.-P. Lee, (2010), "A first look at CALIOP/CALIPSO cloud ice water content", submitted to the 25th International Laser and Radar Conference.
  3. Liu, Z., et. al. 2009: "The CALISPO Lidar Cloud and Aerosol Discrimination: Version 2 Algorithm and Initial Assessment of Performance", Journal of Atmospheric and Oceanic Technology, 26, pp. 1198-1213.
  4. Hu, Y., et. al., 2009: "CALIPSO/CALIOP Cloud Phase Discrimination Algorithm", Journal of Atmospheric and Oceanic Technology, 26, pp. 2293-2309.
  5. Young, S. A. and M. A. Vaughan, 2009: "The Retrieval of Profiles of Particulate Extinction from Cloud-Aerosol Lidar Infrared Pathfinder Satellite Observations (CALIPSO) Data: Algorithm Description", Journal of Atmospheric and Oceanic Technology, 26, pp. 1105-1119.
  6. Protat, A., J. Delanoe, E.J. O’Connor and T.S. L’Ecuyer, 2010: "The evaluation of CloudSat and CALIPSO ice microphysical products using ground-based cloud radar and lidar observations", Journal of Atmospheric and Oceanic Technology, accepted, pp. 1-50.

File Metadata Parameters

Product ID
an 80-byte (max) character string specifying the data product name. For all CALIPSO Level 2 lidar data products, the value of this string will be "L2_Lidar".

Date Time at Granule Start
a 27-byte character string that reports the date and time at the start of the file orbit segment (i.e., granule). The format is yyyy-mm-ddThh:mm:ss.ffffffZ.

Date Time at Granule End
a 27-byte character string that reports the date and time at the end of the file orbit segment (i.e., granule). The format is yyyy-mm-ddThh:mm:ss.ffffffZ.

Date Time at Granule Production
This is a 27-byte character string that defines the date at granule production. The format is yyyy-mm-ddThh:mm:ss.ffffffZ.

Number of Good Profiles
This is a 32-bit integer specifying the number of good attenuated backscatter profiles contained in the granule.

Number of Bad Profiles
This is a 32-bit integer specifying the number of bad attenuated backscatter profiles contained in the granule.

Initial Subsatellite Latitude
This field reports the first subsatellite latitude of the granule.

Initial Subsatellite Longitude
This field reports the first subsatellite longitude of the granule.

Final Subsatellite Latitude
This field reports the last subsatellite latitude of the granule.

Final Subsatellite Longitude
This field reports the last subsatellite longitude of the granule.

Orbit Number at Granule Start
This field reports the orbit number at the granule start time.

Orbit Number at Granule End
This field reports the orbit number at the granule stop time.

Orbit Number Change Time
This field reports the time at which the orbit number changes in the granule.

Path Number at Granule Start
This field reports the path number at the granule start time.

Path Number at Granule End
This field reports the path number at the granule stop time.

Path Number Change Time
This field reports the time at which the path number changes in the granule.

Lidar Level 1 Production Date Time
For each CALIOP Lidar Level 2 data product, the Lidar Level 1 Production Date Time field reports the file creation time and date for the CALIOP Level 1 lidar data file that provided the source data used in the Level 2 analyses.

Number of Single Shot Records in File
for internal use only

Number of Average Records in File
for internal use only

Number of Features Found
for internal use only

Number of Cloud Features Found
for internal use only

Number of Aerosol Features Found
for internal use only

Number of Indeterminate Features Found
for internal use only

Lidar Data Altitude
This field defines the lidar data altitudes (583 range bins) to which Lidar Level 1 profile products are registered.

GEOS Version
This is a 64-byte character that reports the version of the GEOS data product provided by the GMAO.

Classifier Coefficients Version Number
Version number of the classifier coefficients file that stores the five-dimensional probability distribution functions used by the cloud-aerosol discrimination (CAD) algorithm

Classifier Coefficients Version Date
Creation date of the classifier coefficients file that stores the five-dimensional probability distribution functions used by the cloud-aerosol discrimination (CAD) algorithm

Production Script
Provides the configuration information and command sequences that were executed during the processing of the CALIOP Lidar Level 2 data products. Documentation for many of the control constants found within this field is contained in the CALIPSO Lidar Level 2 Algorithm Theoretical Basis Documents


Data Quality Statements

Lidar Level 2 Cloud and Aerosol Profile Information
Half orbit (Night and Day) lidar cloud and aerosol profile data and ancillary data
Release Date Version Data Date Range Maturity Level
June 2013
Expedited Data
3.30 June 1, 2013 to present Provisional
April 2013
Standard Data
3.30 March 1, 2013 to present Provisional
December 2011 3.02 November 1, 2011 to present Provisional
May 2010 3.01 June 13, 2006 to February 16, 2009
March 17, 2009 to October 31, 2011
Provisional
October 2008 2.02 September 14, 2008 to October 29, 2009 Beta
January 2008 2.01 June 13, 2006 to September 13, 2008 Beta
 


Summary Statement for the release of the CALIPSO Lidar Level 1B, Level 2, and Level 3 Products Version 3.30, April 2013

The Version 3.30 CALIOP Lidar Level 1, Level 2, and Level 3 data products incorporate the updated GMAO Forward Processing – Instrument Teams (FP-IT) meteorological data, and the enhanced Air Force Weather Authority (AFWA) Snow and Ice Data Set as ancillary inputs to the production of these data sets, beginning with data date March 1, 2013.

Impacts on CALIOP data products caused by the transition to GEOS-5 FP-IT are predicted to be minimal, based on a comparison of CALIOP Version 3.02 against CALIOP Version 3.30, summarized below. Additional details are given in the following document: Impacts of Change in GEOS-5 Version on CALIOP Products (PDF).

GEOS-5 Changes and CALIOP Impact Summary:
  • Level 1B nighttime calibration:
    GEOS-5 molecular number densities in the CALIOP nighttime calibration region increased by roughly 0.6% on average which caused the nighttime calibration coefficients to decrease on average by -0.6%. Since attenuated backscatter is inversely proportional to the calibration coefficient, nighttime attenuated backscatter will increase by 0.6% on average.

  • Level 1B daytime calibration:
    GEOS-5 molecular number densities in the CALIOP daytime calibration region increased by 0.1% near the equator and increased by up to 0.4 - 0.7% near the poles which caused daytime calibration coefficients to decrease by <-0.2% near the equator and decrease by roughly -0.8% near the poles. Daytime attenuated backscatters will thereby increase by these same magnitudes.

  • Level 2 layer detection:
    GEOS-5 molecular number densities increased in the CALIOP night and day calibration regions subsequently increasing night and day attenuated backscatters, causing the number of layers detected to increase slightly. For the two months examined, the number of aerosol and cloud layers increased by < 0.8% and < 0.2%, respectively.

  • Level 2 layer classification:
    GEOS-5 tropopause height decreased by ∼1 km at 30°S and 40°N and decreased by 1.5 km over the Antarctic in September 2011. Since CALIOP classifies layers detected above the tropopause as stratospheric features, about 3 - 5% of stratospheric features were instead classified as either cloud or aerosol. These changes are considered minor except in Sep. 2011 over the Antarctic where a 1 - 1.5 km reduction in tropopause height caused 100% of cloud and aerosol layers to be re-classified as stratospheric features. This latter effect may occur seasonally over the Antarctic.

  • Level 3 aerosol extinction and aerosol optical depth:
    GEOS-5 molecular number densities increased by small amounts in the CALIPSO calibration regions and by smaller amounts at other altitudes, slightly increasing the number of aerosol layers detected and increasing their attenuated backscatter. Consequent small increases in aerosol extinction and aerosol optical depth are much smaller than uncertainties in these parameters.


Summary Statement for the release of the CALIPSO Lidar Level 2 Cloud and Aerosol Profile Products Version 3.02, December 2011

The CALIPSO Team is releasing Version 3.02 which represents a transition of the Lidar, IIR, and WFC processing and browse code to a new cluster computing system. No algorithm changes were introduced and very minor changes were observed between V 3.01 and V 3.02 as a result of the compiler and computer architecture differences. Version 3.02 is being released in a forward processing mode beginning November 1, 2011.


Summary Statement for the release of the CALIPSO Lidar Level 2 Cloud and Aerosol Profile Products Version 3.01, May 2010

The quality and depth of the information provided in version 3 of the CALIOP Lidar Level 2 Cloud and Aerosol Profile Products are substantially improved over previous releases. The improvements are attributed to a number of factors, including refinements to layer base detection (PDF), more reliable separation of clouds and aerosols (PDF), implementation of a new algorithm for determining cloud thermodynamic phase, and bug fixes in the CALIOP extinction retrieval code. Data usability is greatly enhanced by the addition of new diagnostic and quality assurance parameters (e.g., the atmospheric volume description flags). In addition, several new optical parameters have been added (e.g., particulate depolarization ratio profiles), and range-resolved uncertainty estimates are now provided for all optical profile data.

The organization and structure of the version 3 profile products has also changed significantly. In particular, the aerosol profile products have been completely restructured, and now are reported on the same spatial grid as the cloud profile products. The horizontal resolution of the cloud and aerosol profile products is now identical to the horizontal resolution of the 5-km cloud and aerosol layer products, thus enabling a one-to-one match between the bulk optical properties of any layer (e.g., optical depth) and the profile data (e.g., extinction coefficients) from which the bulk properties were derived. The profile times and latitude/longitude coordinates are now reported identically in the 5-km profile products and the 5-km layer products.

Partly as a result of these many changes, and partly based on preliminary results from several validation studies, the data product maturity level of the profile products has been upgraded to provisional. Previous profile product releases were classified as beta.

In addition to the numerous algorithm updates, several new parameters have been added to the profile products. These include

The structure and content of the version 3 profile products is significantly different from previous releases. In particular, the aerosol profile products have been completely restructured, and now are reported on the same 5-km spatial grid as the cloud profile products. The horizontal resolution of the cloud and aerosol profile products is now identical to the horizontal resolution of the 5-km cloud and aerosol layer products, thus enabling a one-to-one match between the bulk optical properties of any layer (e.g., optical depth) and the profile data (e.g., extinction coefficients) from which the bulk properties were derived. The profile times and latitude/longitude coordinates are now reported identically in the 5-km profile products and the 5-km layer products. Uncertainty estimates are now provided for all profile data derived from the Level 1 lidar attenuated backscatter data, and several new diagnostic and quality assurance parameters have been added. The full extent of the changes made is identified and explained in the paragraphs below.

There are layers for which the optical depth can be reliably measured directly from the CALIOP backscatter signal. For these layers, the measured optical depths are reported in the layer products, and the layer two-way transmittance is used to constrain the extinction solution, so that an optimal estimate of the layer lidar ratio is retrieved. These constrained solutions are the most reliable retrievals, and can be identified by examining the corresponding profile of extinction QC flags. The version 3 retrieval analysis has been improved to increase the amount of constrained retrievals while still maintaining good data quality. Uncertainties of the other, unconstrained, retrievals are larger, and primarily depend on how closely the lidar ratio initially assumed by the algorithm agrees with the true lidar ratio, and on how well the attenuation of overlying layers has been estimated. Errors in the initial lidar ratio and attenuation correction propagate systematically and non-linearly into subsequent retrievals, so that extinction errors do not average out but instead produce biases.

In the case of opaque layers - i.e., where the lidar signal does not penetrate to the base of the layer - the reported optical depth refers only to the upper portion of the layer where there is measurable lidar signal. About 20% of cloud layers are identified as being opaque, as are a few very dense aerosol layers. In these cases CALIOP cannot measure the true layer base height and thus underestimates the true layer optical depth. The Opacity Flag in the Cloud and Aerosol Layer Products identifies layers which are opaque. In the profile products, opaque layers are identified by the extinction QC flags. About 90% of cloud retrievals and 99% of aerosol retrievals conclude successfully, in the sense that a physically possible solution is found (retrieved extinction is finite and non-negative for non-negative inputs). The results of the remaining retrievals can also be found in the data products, but are non-physical. For these cases, the extinction QC flags will enumerate the condition(s) that caused the failure. New in version 3.01, the layer Extinction QC Flags are reported on a profile-by-profile basis. Users are highly encouraged to make use of the extinction QC values when using the data products.

The extinction retrieval will, if necessary to avoid a non-physical solution, adjust the initial lidar ratio as required to produce a physically plausible solution. The most common reason for adjusting the lidar ratio is that the extinction solution diverges and the retrieved values tend toward infinity when the assumed lidar ratio is too large. For weakly scattering layers the lidar ratio is most often left unchanged, as a physical solution is usually obtained on the first iteration. However, if the initial lidar ratio is much larger than the true value, divergence can occur even for optically thin (optical depth < 0.5) layers. When divergence occurs the lidar ratio is automatically decreased and the retrieval is repeated until a convergent solution is obtained.

Retrievals in Aerosol Layers:

In the current implementation of the extinction retrieval algorithm, the lidar ratio adjustment scheme may result in unrealistic retrievals in aerosol layers in cases where the initial divergence is due to a misclassification of aerosol type. When divergence occurs, it can be because the true lidar ratio is much less than the lidar ratio selected by the algorithm, or that the attenuation of overlying layers was overestimated, or both. Aerosol retrievals where the final lidar ratio is not equal to the initial lidar ratio should be regarded carefully by the data user. The lidar ratio is adjusted in only about 5% of the aerosol retrievals.

Users should not be unduly pessimistic about the quality and usability of the CALIPSO optical depth estimates. Figure 1 (below) shows a preliminary comparison of CALIPSO version 3 aerosol optical depths with the optical depths derived from MODIS for all daytime measurements acquired during January 2007. The comparison is generally good, with MODIS appearing to slightly over-estimate values at the lower end of the optical depth range.

Figure 1: Comparison of CALIPSO aerosol optical depths to those derived from MODIS
(Preliminary - January 2007, daytime data, final lidar ratio = initial lidar ratio only)
Comparison of CALIPSO aerosol optical depths.

Retrievals in Cirrus Clouds:

A histogram of cirrus optical depths from the Cloud Layer Product shows a bimodal distribution (see Figure 2, left panel). The peak on the right, at an optical depth of ~2.5, is an artifact due to the behavior of the retrieval algorithm when the initial retrieval diverges and the lidar ratio is reduced to produce a convergent solution. This happens most often in totally attenuating (i.e., opaque) clouds, and when the true cirrus lidar ratio is significantly smaller than the initial value assumed by the algorithm. The second peak disappears when we consider only those solutions for which the lidar ratio was unchanged (i.e., the extinction QC flag = 0; see Figure 2, right panel). In any case, errors grow rapidly when the true cirrus lidar ratio is different from the assumed value. Errors in optical depth for thin cirrus (optical depth < 0.5) are on the order of 41% for unconstrained retrievals for clouds composed of random oriented crystals.

Figure 2: CALIPSO cirrus cloud optical depths
(v3.01 - August 2006, nighttime only)
August 2006: Ice Clouds, Night, Extintion QC = All. August 2006: Ice Clouds, Night, Extintion QC = 0.

Retrievals in Water Clouds:

Although the production code applies the extinction retrieval algorithm to all layers detected, the CALIOP extinction retrieval algorithm was developed for retrievals of aerosol and ice clouds, not water clouds. In version 2 the multiple scattering factor for water clouds was set to unity, resulting in large errors in retrievals of extinction and optical depth. In version 3, a value of η532 = 0.6 is used. Based on Monte Carlo simulations of multiple scattering, this value appears to be appropriate for semitransparent water clouds (τ < 1). (It is purely coincidental this is the same value used for ice clouds.) For denser water clouds (τ > 1) the multiply-scattered component of the signal becomes much larger than the single-scattered component, η532 becomes dependent on both cloud extinction and range into the cloud, and the retrieval becomes very sensitive to errors in the multiple scattering factor used. In these cases the multiple scattering cannot be properly accounted for in the current retrieval algorithm and retrieval results are unreliable.

Data Quality Statement for the release of the CALIPSO Lidar Level 2 Version 3.01, May 2010

The sections below highlight important changes to the layer detection, scene classification, and extinction algorithms that have implications for the overall quality of the Lidar Level 2 data products.

Version 3.01 of the Lidar Level 2 data products is a significant improvement over previous versions. Major code and algorithm improvements include

  • the elimination of a vicious, vile, and pernicious bug in the cloud clearing code that caused a substantial overestimate of low cloud fraction in earlier data releases (details given in Vaughan et al., 2010 (PDF));
  • enhancements to the cloud-aerosol discrimination algorithm that increase the number of diagnostic parameters used to make classification decisions (details given in Liu et al., 2010 (PDF));
  • improved daytime calibration procedures, resulting in more accurate estimates of layer spatial and optical properties (details given in Powell et al., 2010 (PDF)); and
  • an entirely new algorithm for assessing cloud thermodynamic phase (details given in Hu et al., 2009).

Layer Detection
As in previous versions, the layer boundaries reported in the Lidar Level 2 Cloud and Aerosol Layer Products appear to be quite accurate. Some false positives are still found beneath optically thick layers; these, however, can generally be identified by their very low CAD scores (e.g., |CAD score| ≤ 20).In opaque layers, the lowest altitude where signal is reliably observed is reported as the base. In actuality, this reported base may lie well above the true base. Opaque layers are denoted by an opacity flag. In this release, the layers which are reported represent a choice in favor of high reliability over maximum sensitivity. Weakly scattering layers sometimes will go unreported, in the interest of minimizing the number of false positives.

Cloud-Aerosol Discrimination
Figure 7 (below) compares the distributions of CAD scores derived from four months of version 3 test data to the corresponding version 2.01 data. The V3 curve shows a smoother distribution and generally has fewer low CAD values (i.e., values less than ~|95|), reflecting the better separation of clouds and aerosols when using the version 3 5-D PDFs as compared to the separation provided by 3-D PDFs in previous versions. One notable exception to this observation is the bump between -10 and 20 in the V3 test curve, which accounts for ~ 6% of the total features. The CAD scores in this region identify both outlier features whose optical/physical properties are not correctly measured or derived, and those features whose attributes fall within the overlap region between the cloud and aerosol PDFs. In contrast, these outliers are populated over the entire CAD span in the V2 release.

Figure 7: Histograms of CAD scores for Version 2 (red) and Version 3 (blue)
CAD score histogram comparison.
Figure 8 (below) presents the relationship between the CAD score and the layer IAB QA factor, which provides a measure of the integrated attenuated backscatter overlying a cloud or an aerosol layer. A layer IAB QA factor close to 1 indicates that the atmosphere above the layer under is clear. Decreasing values indicate the increasing likelihood of overlying layers that have attenuated the signal within the layer under consideration, and thus decreased the SNR of the measurement. A layer IAB QA factor of 0 would indicate total attenuation of the signal. As seen in the figure, the IAB QA is highest for high magnitude CAD scores and slopes down gradually for small CAD score magnitudes. This relationship reflects the fact that the presence of overlying features tends to add difficulty to the cloud-aerosol classification task, and therefore reduces the confidence of the classifications made. The dip between -10 and 20 represents features that are outliers in the 5-D CAD PDFs, and indicates that these outliers most often lie beneath other relatively dense features. The cloud layers with special CAD scores (103 and 104) have the smallest IAB QA values. The relatively big value at CAD = 0 corresponds to the features having zero CAD values at high altitudes where the probability of the presence of overlying features is low. At high altitudes the separation of clouds and aerosols is not as good as at low altitudes because of the presence of subvisible cirrus clouds.

Figure 8: Relation between CAD score and Layer IAB QA Factor
Relation between CAD score and Layer IAB QA Factor.
Overall, because of the better separation between clouds and aerosols in the 5D space, the 5D CAD algorithm significantly improves the reliability of the CAD scores. The improvements include:
  1. Dense aerosol layers (primarily very dense dust and smoke over and close to the source regions), which are sometimes labeled as cloud in the V2 release, are now correctly identified as aerosol, largely because of the addition of the integrated volume depolarization ratio to the diagnostic parameters used for cloud-aerosol discrimination. In addition, in the open oceans, dense aerosols that were previously classified as clouds are now frequently observed in the marine boundary layer. Improvements are also seen for these maritime aerosols. Note, however, dense dust/smoke layers found at single-shot (0.333 km) resolution will be classified as cloud by default. This issue will be revisited for post-V3 releases.

  2. Because the V2 CAD algorithm used a latitude-independent set of 3D PDFs, a class of optically thin clouds encountered in the polar regions that can extend from the surface to several kilometers were sometimes misclassified as aerosols. In version 3, these features are now correctly classified as cloud.

  3. Correct classification of heterogeneous layers is always difficult. An example of a heterogeneous layer would be an aerosol layer that is vertically adjacent to a cloud or contains an embedded cloud, but which is nonetheless detected by the feature finder as a single entity in the V2 release. By convention, heterogeneous layers should be classified as clouds. The version 3 feature finding algorithm has also been improved greatly, and can now much better separate the embedded or adjacent single-shot cloud layers from the surrounding aerosol. This improvement in layer detection contributes significantly to the improvement of the CAD performance.

  4. Some so-called features identified by the layer detection scheme are not legitimate layers, but instead are artifacts due to the noise in the signal, multiple scattering effects, or to artificial signal enhancements caused by non-ideal detector transient response or an over estimate of the attenuation due to overlying layers. These erroneous "pseudo-features" are neither cloud nor aerosol and are distributed outside of the cloud and aerosol clusters in the PDF space. The V3 CAD algorithm can better identify these outlier features by assigning a small CAD score (the bump between -10 and 20 in the V3 CAD histogram) and classify most of them as cloud by convention. A CAD threshold of 20 can effectively filter out these outliers.

Some misclassification may still occur with the 5D algorithm. For example, dust aerosols can be transported long distance to the Arctic. When moderately dense dust layers are occasionally transported to high latitudes, where cirrus clouds can present even in the low altitudes, they may be misclassified. This is also the case for moderately dense smoke aerosols occasionally transported to the high latitudes. Smoke can be mixed with ice particles during the long range transport, which makes the smoke identification even more difficult. When moderately dense dust and smoke are transported vertically to high altitudes, even at low latitudes, misclassifications can occur due to the presence of cirrus clouds. Volcanic aerosol that is newly injected into the high altitudes may have a large cross-polarized backscatter signal and thus may be misclassified as cloud.

Aerosol Type Identification
The main objective of the aerosol subtyping scheme is to estimate the appropriate value of the aerosol extinction-to-backscatter ratio (Sa) to within 30% of the true value. Sa is an important parameter used in the determination of the aerosol extinction and subsequently the optical depth from CALIOP backscatter measurements. Sa is an intensive aerosol property, i.e., a property that does not depend on the number density of the aerosol but rather on such physical and chemical properties as size distribution, shape and composition. These properties depend primarily on the source of the aerosol and such factors as mixing, transport, and in the case of hygroscopic aerosols, hydration.

The extinction products are produced by first identifying an aerosol type and then using the appropriate values of Sa and the multiple scattering factor, η(z). Note that multiple scattering corrections have not yet been implemented for the current data release, so that η(z) = 1 for all aerosol types. The accuracy of the Sa value used in the lidar inversions depends on the correct identification of the type of aerosol. In turn, the accuracy of the subsequent optical depth estimate depends on the accuracy of Sa.

The underlying paradigm of the type classification is that a variety of emission sources and atmospheric processes will act to produce air masses with a typical, identifiable aerosol 'type'. This is an idealization, but one that allows us to classify aerosols based on observations and location in a way to gain insight into the geographic distribution of aerosol types and constrain the possible values of Sa for use in aerosol extinction retrievals.

The aerosol subtype product is generated downstream of the cloud-aerosol discrimination (CAD) scheme and, therefore, depends on the cloud-aerosol classification scheme in a very fundamental way. If a cloud feature is misclassified as aerosol, the aerosol subtype algorithm will identify this 'aerosol' as one of the aerosol subtypes. The user must exercise caution where the aerosol subtype looks suspicious or unreasonable. Such situations can occur with some frequency in the southern oceans and the polar regions.

Cloud Ice/Water Phase Discrimination
The cloud phase algorithm used in Version 2 has been replaced with a new, completely different algorithm. The Version 3 algorithm classifies detected cloud layers as water, randomly-oriented ice (ROI), or horizontally-oriented ice (HOI) based on relations between depolarization, backscatter, and color ratio (Hu et al. 2009). These classifications have not yet been rigorously validated, which is difficult, but many of the obvious artifacts found in the Version 2 data have been eliminated.

The version 2 algorithm included a rudimentary ability to identify a specific subset of high confidence instances of HOI. These clouds were classified as ice clouds, and flagged with a 'special CAD score' of 102, indicating that they had been further classified as HOI. The new version 3 algorithm implements a much more sophisticated scheme for recognizing HOI that correctly identifies many more instances of these sorts of ice clouds. The special CAD score of 102 is no longer used to identify these layers. Instead, the "ice cloud" and "mixed phase cloud" classifications have been eliminated, and replaced as shown in the table below.

Value Version 2 Interpretation Version 3 Interpretation
0 unknown/not determined unknown/not determined
1 ice randomly oriented ice (ROI)
2 water water
3 mixed phase horizontally oriented ice (HOI)

The Ice/water Phase QA flags have also been redefined slightly for Version 3, as follows:

Value Version 2 Interpretation Version 3 Interpretation
0 no confidence no/low confidence
1 low confidence phase based on temperature only
2 medium confidence medium confidence
3 high confidence high confidence

A confidence flag of QA=1 indicates the phase classification is based on temperature. Initial classification tests are based on layer depolarization, layer-integrated backscatter, and layer-average color ratio. Layers classified as water with temperature less than -40 C are forced to ROI and given a confidence flag of QA=1. Layers classified as ROI or HOI with temperature greater than 0 C are forced to water and also given a confidence flag of QA=1. Clouds for which the phase is 'unknown/not determined' are assigned a confidence value of 0 (no/low confidence).

Layers classified as HOI based on anomalously high backscatter and low depolarization are assigned QA=3. These layer characteristics are rarely detected after the CALIOP viewing angle was changed to 3° in November 2007. The Version 3 algorithm computes the spatial correlation of depolarization and integrated backscatter, and uses this as an additional test of cloud phase. Layers classified as HOI using this test are assigned QA=2. The spatial correlation test is responsible for the majority of the layers classified as HOI. These layers typically have higher backscatter than ROI but similar depolarization, and are common even at a viewing angle of 3°. We interpret this as clouds with significant perpendicular backscatter from ROI but containing enough HOI to produce enhanced backscatter. These layers tend to be found at much colder temperatures than the high confidence HOI (see Hu et al. 2009).


Cloud and Aerosol Optical Depths
The reliability of cloud and aerosol optical depths reported in the version 3 data products is considerably improved over the version 2 release. Whereas the version 2 optical depths were designated as a beta quality product, and not yet suitable for use in scientific publications, the maturity level of the version 3 optical depths has been upgraded to provisional. Several algorithm improvements and bugs fixes factored into the decision to upgrade the maturity level. Among these were the addition of the aerosol layer base extension algorithm, which greatly improves AOD estimates in the planetary boundary layer (PBL), and several significant improvements to the code responsible for rescaling the attenuated backscatter coefficients in lower layers to compensate for the beam attenuation that occurs when traversing transparent upper layers.

PLEASE NOTE: Users of the CALIOP optical depths should read and thoroughly understand the information provided in the Profile Products Data Quality Summary. This summary contains an expanded description of the extinction retrieval process from which the layer optical depths are derived, and provides essential guidance in the appropriate use of all CALIOP extinction-related data products. Validation and improvements to the profile products QA are ongoing efforts, and additional data quality information will be included with future releases.




NASA
Last Updated: November 28, 2017
Curator: Charles R. Trepte
NASA Official: Charles R. Trepte

+ No Fear Acts
+ FOIA
+ Privacy

+ Office Of Inspector General
+ Agency Financial Reports

+ Contact NASA