NOTE: This page provides information supporting the version 3.01
data products. Please refer to the latest version of the Data Products Catalog that supports
the current versions of the data products -
CALIPSO Data Products Catalog (Release 4.80) (PDF).
We strongly encourage prospective and current CALIPSO data users read through all of the following page.
This will help guide you to use the data in the most appropriate manner and hopefully prevent
mistakes in your analyses. The following topics will be covered:
There is a plethora of information available about CALIPSO data and how it is generated.
The resources listed in this section should help you find it.
Data Quality Summaries
These documents provide a high-level quality assessment of all level 1 and level 2 products derived
from the CALIPSO lidar measurements, as described in section 2 of the
CALIPSO Data
Products Catalog (Version 3.6) (PDF).
As such, they represent the minimum information needed by
scientists and researchers for appropriate and successful use of these data products.
We strongly suggest that all authors, researchers, and reviewers of research papers
review this document periodically, and familiarize themselves with the latest status
before publishing any scientific papers using these data products.
Two examples of data quality flag usage and screening methods are
provided in the Examples and Tools section
section of this user guide.
The CALIPSO Data Products Catalog (DPC)
provides a complete enumeration of the contents of the data products that are used or produced by the CALIPSO
project Data Management System.
This is a very terse document that lists all of the scientific data sets (SDSs) available in each
data product. For each SDS, data type, and data size are given, as are the geophysical units and a
nominal range over which the data values can be expected to extend.
The DPC also provides some information about the source and format
of ancillary data, such as GMAO met data,
IGBP surface data types,
and DEM data sources (GTOPO30).
However, the DPC is not the document to go to for usage information about individual SDSs. To obtain this
kind of information please consult the data summaries
section of this user guide.
CALIPSO's payload includes a polarization-sensitive, two-wavelength lidar (CALIOP), a three-channel
Infrared Imaging Radiometer (IIR),
and a visible channel Wide Field Camera. The data from CALIOP is distributed as range-resolved
vertical profiles (level 1 and level 2); as tabulated sets of spatial and optical parameters
describing all layers detected (level 2); and as a high resolution bit-mapped vertical
feature mask (level 2). Level 1 and level 2 swath data from the IIR and WFC are available in
formats similar to those used by other passive remote sensors (i.e. MODIS, AVHRR, etc.).
All CALIPSO data products are available in the 'self-describing'
HDF4 data format.
This data product contains a half orbit (day or night) of calibrated and geolocated single-shot
(highest resolution*) lidar profiles, including 532 nm and 1064 nm attenuated backscatter and
depolarization ratio at 532 nm. The product released contains data from nominal science mode measurement.
The CALIOP Level 1B data product also contains additional parameters such as post processed
ephemeris data, celestial data, and converted payload status data.
The level 2 analyses of the CALIOP lidar backscatter data begins with an attempt to locate all coherent
"features" - i.e., clouds and aerosol layers - in each granule of the level 1 data. The results of
this search are reported in four different lidar level 2 layer products. Cloud layers are reported
at three different horizontal averaging resolutions: 1/3-km, 1-km, and 5-km. The 1/3-km data is
reported only in those regions where single shot information is available in the downlinked data;
that is, between ~8.2-km and -0.5-km. The 1-km and 5-km cloud layers are reported between
~20.2 and -0.5-km. Aerosol layers are reported between at a 5-km horizontal resolution between
~30.1-km and -0.5km. Layers detected in the stratosphere are recorded in the aerosol layer products,
and thus users seeking measurements of polar stratospheric clouds should order the 5-km aerosol layer
products. Note that the layers reported in the 5 km horizontal resolution data products may have
been required averaging to as much as 80 horizontally before being detected by the feature finder
algorithm. However, these fainter layers are still reported on the same uniform 5-km horizontal grid.
To effectively use the 5-km layer products, users should first thoroughly familiarize themselves
with the multi-gridded averaging scheme described here
and in the feature finder ATBD.
Each layer identified in the 5 km cloud and aerosol layer products is further analyzed to determine
the profiles of particulate extinction and backscatter within the layer. This profile data,
along with ancillary information (e.g., meteorological data from the GMAO) is reported in the
5 km cloud and aerosol layer products. As in the layer products, clouds and aerosols are
reported in separate data products, and the profile data for stratospheric features is
included in the aerosol profile products.
The feature mask data set describes the vertical and horizontal distribution of
cloud and aerosol layers observed by the CALIPSO lidar. Each range bin in the Lidar
Level 0 data is characterized by a single 16-bit integer, with the various bits in the
integer representing flags that describe some aspect of the data measured within the bin.
Instructions on how to decode these integer data are given in the
vertical feature mask summary page.
The data are recorded in nominal increments of 15 consecutive laser pulses, which is
nominally equivalent to a distance of 5-km along the laser ground-track.
IIR Data
The IIR instrument is a 3 channel imaging radiometer in the thermal infrared at 8.65,
10.6 and 12.05 microns. More information may be obtained from the CNES website:
http://smsc.cnes.fr/CALIPSO/GP_iir.htm
The following data products are distributed:
Emissivity and cloud particle data assigned to IIR pixels on a 1 km grid centered on the Lidar track.
WFC Data
The primary Wide Field Camera Level 1B data products are calibrated radiances and
bidirectional reflectances registered to an Earth-based grid centered on the lidar
ground track. During normal operations, the WFC acquires science data only
during the daylight portions of the CALIPSO orbits. For each daytime orbit segment, three different
data products are produced: 1 km Native Science grid, 125 m Native Science grid, and 1 km Registered Science grid.
In addition to radiance and reflectance grids, the WFC Level 1 data products
include two parameters that quantify the homogeneity of the cross track image frames:
swath homogeneity and track homogeneity.
On the identical grid as the CALIPSO IIR data and is produced to facilitate the use of the WFC data in the IIR retrievals.
Vertical and horizontal resolution of the lidar data
The CALIPSO lidar collects profile data at a nominal vertical resolution of 15 m (10 MHz sampling rate),
and at a laser pulse repetition rate of 20.16 Hz, which when accounting the the satellites orbital velocity,
translates into a horizontal profile spacing of ˜333 m. Within the lower atmosphere the footprint
of each profile is ˜90 m in diameter. Due to bandwidth limitations for downlinking
the data from the satellite, the data is averaged on-board the satellite in both the horizontal
and vertical dimensions.
The purpose of the on-board averaging scheme is to maximize spatial resolution and dynamic
range of the signal while minimizing the telemetry data volume. The resulting algorithm reduced
the required telemetry bandwidth by more than an order of magnitude relative to the raw data,
with minimum impact on the information content of the data. Unfortunately this comes at a price
in terms of complexity in the telemetry data.
The on-board averaging scheme provides the highest resolution in the lower troposphere where the
spatial variability of clouds and aerosols is the greatest and coarser resolutions higher in the atmosphere.
The degree of averaging varies with altitude, as detailed in the figure below.
The left panel above is a schematic of 15 consecutive full-resolution lidar profiles
(i.e., a 5-km horizontal distance), annotated to illustrate how CALIOP's on-board averaging scheme
varies vertically and horizontally as a function of altitude. The five different averaging regimes
are shown using five different shades of blue. The vertical lines within each blue band delineate
the individual profiles created by horizontally averaging the full resolution data. For example,
between 8.2-km and 20.2-km the data is averaged horizontally to a nominal spatial resolution of
1-km; i.e., the data from 3 full resolution (333 m) profiles are averaged to create each 1 km
averaged profile segment. Similarly, between 20.2 km and 30.1 km, 5 full resolution (333 m)
profiles are averaged to create 3 profiles that each spans a nominal horizontal extent of 5/3 km.
Processing in the vertical is done in a similar fashion, as indicated by the thin horizontal
black lines shown in the profile on the far right within each altitude regime.
The table in the middle gives the numerical values of the horizontal and vertical averaging for
all the altitude regions. The highest resolution data available is in the region between -0.5 and 8.5 km
which is provided at 333 m horizontally and 30 m in the vertical.
During level 1 processing the data is regridded to a uniform horizontal grid of 333 m
(see the right hand image above); however, the vertical (i.e. altitude) resolution remains identical
to that of the downlinked data. Thus even though the level 1B profile data is reported at a uniform
horizontal resolution of 333 m, in regions above 8.5 km it is over sampled. For example, to create
15 "pseudo single shot" profiles in the altitude region between 8.5 km to 20.1 km, the downlinked
1 km profiles in this region are replicated and grafted onto the top of three consecutive full
resolution (single shot) profiles.
The preceding commentary refers specifically to the 532 nm parallel and perpendicular channels only.
The 1064 nm channel is recorded at a vertical resolution of 60 m in the range be -0.5 and 8.2 km, also
no 1064 nm data is available above 20.1 km.
Detailed information about the altitude array and vertical and horizontal averaging
of the data users are encouraged to read
section 2.3 in the
Lidar Level 1 ATBD or section 3.3
in the mission overview document.
In all of the profile data sets the altitude array can be found in the metadata
field (VDATA) called Lidar_Data_Altitudes.
Multi-Gridded Averaging Scheme
The peak backscatter intensities of the features measured by space-borne lidar range over several
orders of magnitude. Strongly scattering features such as stratus and fair weather cumulus are
easily detected using a single laser pulse. For more tenuous features - e.g., thin cirrus clouds -
the average of several laser pulses may be required to obtain the signal-to-noise ratio necessary
to differentiate feature boundaries from the ambient scattering environment. The unambiguous
detection of the very weakest features - faint aerosol layers and subvisible cirrus - may require
averaging over a substantial number of pulses.
To identify all of the features within a given
scene at the maximum possible spatial resolution we employ a Selective Iterated Boundary
Location (SIBYL) scheme. The SIBYL algorithm makes multiple passes through a specified
scene, constructing profiles of attenuated scattering ratios at a series of increasingly coarse
spatial resolutions, nominally at 5, 20 and 80 km.
Immediately after construction, each profile is scanned for the presence of
clouds, aerosol layers, and/or surface returns using a profile scanner, (the 'Layer detection' box at
right).
The backscatter data from those regions
identified as containing a feature are removed from subsequent processing. As a consequence,
features found at high spatial resolutions (i.e., with less averaging) will not be included in the
profiles of attenuated scattering ratios scanned at coarser resolutions (i.e., more averaging).
(This encompasses the 'Remove layers...' and 'Average data to 20 or 80 km' boxes.)
Note that the layer detection at 1 km and 333 m horizontal resolution are only performed for those
regions where 5 km layers were detected. The boundary layer homogenization algorithm
(BLaH) will not be further discussed in this document.
Users are encouraged to read section 4 (section 4.4 is about BLaH) in the
feature detection ATBD.
Figure 1.
Real world example of the multi-gridded averaging scheme
The following series of slides serve as a guide to explain more details of the averaging scheme.
Browse image of the total 532 nm attenuated backscatter signal (the sum of the 532 nm parallel
and perpendicular return signals).
The signal strength has been color coded such that blues correspond to molecular scattering
and weak aerosol scattering, aerosols generally show up as yellow/red/orange.
Stronger cloud signals are plotted in gray scales, while weaker cloud returns
are similar in strength to strong aerosol returns and coded in yellows and reds.
Figure 2.
Zoomed in version of above image from the area between the black lines.
There is a transparent cirrus cloud located at ~17 km and an aerosol layer (most likely dust)
located from the surface or in some places ~ 1km up to an altitude of 4 km. The surface return is
seen as the white/red/yellow band near 0 km.
The horizontal line at 8 km is from the changing noise characteristics in the data due
to the change in horizontal and vertical averaging of the data. Data is plotted a 333 m horizontal
resolution.
Figure 3.
The following image shows the vertical and horizontal locations of all features that were
detected at the 5 km horizontal resolution. Only the data in the regions as identified by
the magenta color are used to compute the layer optical properties as would be found in the
5 km layer products.
The layer data in those regions will then be zeroed-out before the 5 km is averaged to 20 km.
Figure 4.
After the removal of features that were detected at the 5 km resolution, a search is
made for features at the 20 km resolution. The image at right shows features
that are found at the 20 km resolution in blue. The image (below-right)
shows both the 5 km features and the 20 km features.
Only the data in the regions shown in blue are used to calculate the layer optical
properties for those same regions. The regions in magenta are not included in the
averages for the features detected on the 20 km resolution. Layer
optical properties are repeated (i.e. reported in the data products)
for each 5 km column in which they appear.
Figure 5.
Corrected version of above image to show the true spatial contribution of data to the detected
layer, 20 km features are blue, 5 km features are magenta.
Figure 6.
The following image shows the vertical and horizontal locations of all features that were
detected at the 80 km horizontal resolution. Only the data in the regions as identified by
the white color are used to compute the layer optical properties as would be found in the
5 km layer products. Layer data are reported in every 5 km column the feature appeared in.
As in the 20 km averaging image, this one has plotted the vertical extent of each 80 km Feature
as it appears in the 5 km layer product file. However that area contribution of the underlying data
is actually a lot less, see the corrected image below.
Figure 7.
Corrected version of above image to show the true spatial contribution of data to the detected
layer.
Figure 8.
Final composite 'feature mask' image showing 5, 20 and 80 km horizontal resolutions.
Figure 9.
Another version of the 'feature mask' showing the same horizontal resolution as above-right but
including the 1 km and 333 m layer extents. The 1 km and 333 m profile data are only scanned for
layers in regions where a 5 km layer was found, therefore 1 km layers will only be
found in regions where a 5 km layers were found and subsequently 333 m are only found where 1 km
layers were found.
Figure 10.
This 'vertical feature mask' image shows the vertical locations of all layers detected by
the level 2 processing code, colors represent the type of layer as determined by the scene
classification algorithm. Cloud layers are identified by the cyan color, aerosols by green, dark
blue is 'clear air' where no layers were detected, etc.
Figure 11.
Improvements and highlights for Version 3.01
The third release of the CALIPSO data products features a comprehensive restructuring
and expansion of the Lidar Level 2 cloud and aerosol profile products; significant
enhancements to the Lidar Level 2 cloud and aerosol profile products; and the
implementation of an improved calibration technique for the Lidar Level 1 532 nm daytime calibration.
The primary contents of the CALIOP Level 1 product are calibrated profiles
of 532 nm perpendicular attenuated backscatter and total attenuated backscatter
at 532 nm and 1064 nm. The version 3 CALIOP Level 1 data are released with a
product maturity classification of Validated Stage 1, indicating that initial
validation of the CALIOP attenuated backscatter products has been successful.
The CALIOP Level 2 products consist of the full resolution vertical feature mask,
cloud and aerosol layer products reported at several different spatial resolutions, and
cloud and aerosol profile products reported at a uniform 5-km horizontal resolution.
Validation of the Level 2 products is an on-going process, and some products are better
characterized than others. Preliminary validation has been accomplished for the layer detection,
cloud-aerosol discrimination, and layer sub-typing algorithms. Data products derived solely
from these algorithms are designated as Validated Stage 1. Validation of the optical properties
algorithms that generate extinction profiles and layer optical depth estimates is a
considerably more challenging task. As a consequence, while all studies to date indicate
acceptable performance, the status of those data products derived from this second class
of algorithms remains Provisional. All obvious artifacts have been identified and
corrected in these data, but only limited comparisons with independent data sets are
currently available.
Detailed data quality statements for each data release may be found under the data
quality section of this user guide.
Reconstructed unprocessed instrument/payload data at full resolution; any and all communications artifacts (e.g., synchronization frames, communications headers) removed.
Level 1A
Reconstructed unprocessed instrument data at full resolution, time-referenced, and annotated with ancillary information, including radiometric and geometric calibration coefficients and geo-referencing parameters (i.e., platform ephemeris) computed and appended, but not applied, to the Level 0 data.
Level 1B
Level 1A data that have been processed to sensor units (not all instruments have a Level 1B equivalent).
Level 2
Derived geophysical variables at the same resolution and location as the Level 1 source data.
Level 3
Variables mapped on uniform space-time grid scales, usually with some completeness and consistency.
Level 4
Model output or results from analyses of lower level data (e.g., variables derived from multiple measurements).
Last Updated: October 20, 2020
Curator: Charles R. Trepte
NASA Official: Charles R. Trepte
