NASA - National Aeronautics and Space Administration
ABOUT CALIPSO PRODUCTS OUTREACH DOCUMENTS RESOURCES TOOLS CONTACTS        

CALIPSO Data User's Guide - Frequently Asked Questions


CALIPSO HOMECALIPSO’s User Guide HOME → FAQ

    General

  1. Where to find answers to key questions?
     
  2. Where do I get version 3.01, 3.02, and 3.30 CALIPSO level 1 and level 2 data?
     
  3. Where is the altitude array in the data products?
     
  4. Why can't I find lidar (CALIOP) data over a specific location? Is the order tool broken?
     
  5. How do I reproduce the CALIPSO color bar in my own plots?
     
  6. I'm having difficulty finding a data product that tells me boundary layer depth. Could you please direct me to this product?
     
  7. What are the fill values? Are they the same in every file/every variable?
     
  8. Why don't I see any negative values in the CAD score? I'm an IDL user.
     

  9. Data Quality

  10. What do I need to know about quality flags?
     
  11. How can I use the CAD score to identify possibly unreliable layers?
     
  12. Is CALIOP data quality homogeneous over the whole Earth?
     
  13. What are constrained cirrus cloud retrievals, how do I find them, and why should I bother to look for them?
     

  14. LIDAR Level 1B Product

  15. Why are there negative attenuated backscatter values in the L1 lidar data? (Aren't these values completely non-physical???) Are these negative values removed before calculating the Level 2 Column Integrated Backscatter?
     
  16. When I plot the lidar level 1B profile data, why are there discontinuities at ~8 km and again at ~20 km?
     

  17. LIDAR Level 2 Products

  18. In the layer products the data set Number_Layers_Found indicates that there are N (e.g. 3) layers in a column, does that mean there are N (e.g. 3) unique atmospheric layers in the column?
     
  19. I have discovered, at least in the 5 km cloud layer files, that the top of the second cloud layer is often higher than the bottom of the first (top) layer. Can you explain to me what this means?
     
  20. The file name of the 5 km layer products would seem to indicate that features are reported at 5 km, but in those files there are features reported at 20 and 80 km, what's up with that?
     

  21. LIDAR Level 2 Profile Products

  22. In the profile products what is the difference between 'attenuated backscatter' and 'backscatter'?
     
  23. What's the difference between volume depolarization ratios and particulate depolarization ratios?
     
  24. Are layer boundaries and layer type information provided in the CALIOP profile products?

  25. LIDAR Level 2 VFM Product

  26. Why does the surface occupy multiple range bins in the Vertical Feature Mask (VFM)?
     
  27. What information is provided by the Feature_Classification_Flag(s)?
     

  28. Miscellaneous

  29. How do I say CALIOP?
     



    General

  1. Where to find answers to key questions?
  2. Right here!

    Data users are highly encouraged to read the “Essential Reading” section as well as the Data Quality Statements (NASA Langley ASDC).

  3. Where do I get version 3.01, 3.02, and 3.30 CALIPSO level 1 and level 2 data?
  4. The CALIPSO level 1 and level 2 data are available from the NASA Langely Atmospheric Science Data Center (ASDC) at https://eosweb.larc.nasa.gov/.

  5. Where is the altitude array in the data products?
  6. The altitude array can be found in the metadata (Vdata) of each hdf data file. The field is called 'Lidar_Data_Altitudes'.

    Example Matlab code below, note that the returned 'metadata' is a cell array, the second line converts it to a normal array.

    
    metadata = hdfread('CAL_LID_L2_05kmCLay-Prov-V3-01.2010-02-02T01-16-31ZN.hdf', 
               '/metadata', 'Fields', 'Lidar_Data_Altitudes', 'FirstRecord',1 ,
               'NumRecords',1);
    altitudes=metadata{1};
    
    

    Sample IDL code for getting the altitudes, the output here is “altitudes”.

    	
    fid = hdf_open('CAL_LID_L2_05kmCLay-Prov-V3-01.2010-02-02T01-16-31ZN.hdf') 
    vds_id = hdf_vd_lone(fid) 
    vdata_id = hdf_vd_attach(fid,vds_id,/read) 
    nrec = hdf_vd_read(vdata_id,altitudes,fields='Lidar_Data_Altitudes')  
    hdf_vd_detach,vdata_id 
    hdf_close,fid
    
    
  7. Why can't I find lidar (CALIOP) data over a specific location? Is the order tool broken?
  8. CALIOP is a nadir-only lidar, therefore data is only collected along the ground track of the satellite. The CALIPSO satellite is in a sun-syncrhonous orbit with a afternoon equatorial crossing time, and the satellite orbit track repeats every 16 days. Maps of the orbit track can be found below. So, if you are unable to find lidar data over your specific location it is likely that the CALIPSO satellite does not pass over that location.

    16 day world. Example 1 day orbit track (daytime).
    16 day US. Example 1 day orbit track (nighttime).
    16 day world. 16 day orbit track for the globe.
    16 day US. 16 day orbit track for North America.
  9. How do I reproduce the CALIPSO color bar in my own plots?
  10. We've provided Matlab, and IDL code in the tools section of this user guide to help you plot lidar attenuated backscatter data with our 'standard' color scale.

  11. I'm having difficulty finding a data set that tells me boundary layer depth (boundary layer height). Could you please direct me to this product?
  12. Unfortunately we do not provide any boundary layer depth product in the current version (3.01) of the data sets. We have plans to provide such a product in future releases.

    Coming soon, generic mapping from attenuated backscatter to RGB values.

  13. What are the fill values? Are they the same in every file/every variable?
  14. The fill value in the data products depends on the data type of the individual data set. The following lists fill values you will encounter. Fill value signifies 'no data'.

    Float_32 -9999.0
    Int_8 see SDS description
    Int_16 -9999
    Int_32 -9999
    UInt_8 -127
    UInt_16 see SDS description
    UInt_32 see SDS description
    ExtinctionQC Fill Value 32768
    FeatureFinderQC No Features Found 32767
    FeatureFinderQC Fill Value 65535

  15. Why don't I see any negative values in the CAD score? I'm an IDL user.
  16. HDF has two one-byte data types, a signed 7-bit integer and an unsigned 8-bit integer. IDL has only one, and it's unsigned. If you don't add code to convert them, you will get numbers between 128 and 256 instead of negatives.


    Data Quality

  17. What do I need to know about quality flags?
  18. If you are using lidar level 1B profile data there are two quality flag data sets. QC_Flag and QC_Flag2 Both are bit fields that indicate various error, warning or information conditions such as a failure to geolocate a particular profile (Bit 4 in QC_Flag) or the '..single shot 532 laser energy is below data quality threshold' (Bit 15 in QC_Flag). Bits 4 and lower are considered error conditions and the L1 data should not be used. Please see the 1B profile data description for more detailed information.

    For users of level 2 lidar data, there are two data use examples that discuss recommended data quality screening flags and thresholds. A brief list of available data quality products are:

    CAD_Score
    Provides a numerical confidence level for the classification of layers by the CALIOP cloud-aerosol discrimination algorithm.
    Extinction_QC_532, Extinction_QC_1064
    Provides a bit-mapped set of flags that report the status of the extinction retrieval algorithm for each feature where a retieval was attempted.
    Layer_IAB_QA_Factor
    Provides a qualitative assessment of the confidence that users should assign to each layer reported.
    Feature_Classificaiton_Flags or Atmospheric Volume Description
    Reports (a) feature type (e.g., cloud vs. aerosol vs. stratospheric layer); (b) feature subtype; (c) layer ice-water phase (clouds only); and (d) the amount of horizontal averaging required for layer detection. Quality assessments are also provided for all layer classifications, such as the confidence of the ice-water phase discrimination and aerosol-typing.

    We've also provided uncertainty estimates for most data sets, they can further help you assess data quality.

  19. How can I use the CAD score to identify possibly unreliable layers?
  20. Assuming that you are interested in the optical properties of the aerosol layer and not just the layer boundaries then there are several other data products you should consider utilizing in your analysis:

    The CALIOP CAD_Score quantifies our confidence in the classification of any given layer as either a cloud or an aerosol. CAD_Score values range from -100 to 100: positive values indicate a cloud, negative an aerosol. Values close to zero indicate less confidence in the type classification. In general, layers for which the magnitude of the CAD score is less than 30 should be regarded with suspicion. We recommend excluding these features from analyses of layer optical properties. There are also several CAD values that fall outside the nominal range of -100 to 100. See the description here for information and usage advice about these "special" CAD_Score values.

    If you are interested in using the CAD_Score to evaluate layer optical properties, there are several other data products you should consider including in your analyses. Ideally, one would check the total optical depth above any layer to determine the quality of the it's measured properties. This is because the backscatter signal in layers with larger overlying optical depths will have relatively poorer signal-to- noise (SNR), which in turn reduces the confidence one should place in any derived layer optical properties. However, since elastic backscatter lidars such as CALIOP cannot measure optical depth directly, we have instead provided the Layer_IAB_QA_Factor as a qualitative proxy to the optical depth above the layer. The Layer_IAB_QA_Factor ranges between zero and one, with low values indicating that there is likely a high amount of attenuation above the feature and the quality of the extinction data and other layer descriptors are greatly reduced.

    The layer Extinction_QC_Flag_532 should also be examined, and if any of bit field indices greater than 5 (value > 31) are set, then the aerosol extinction data should probably not be used.

    If you are just interested in the layer geometric extent, then screening the CAD_Score and Layer_IAB_QA_Factor would be sufficient. Note that for completely attenuating features (i.e. opaque) the lidar is only measuring the apparent base not the true base of the feature. A layer Opacity_Flag of 1 flags a given feature as being opaque.

  21. Is CALIOP data quality homogeneous over the whole Earth?
  22. In theory, yes, as the same suite of algorithms is applied to the data uniformly around the entire globe. In practice, however, the data quality can and sometimes does vary from one point on the globe to another.

    For any global assessment data quality variability, the very first thing to realize is this: there is a substantial difference between data acquired when the satellite is in full darkness (i.e., nighttime data), and data acquired during the sunlit portions of the orbit (i.e., daytime data). The high solar background signal present during daylight measurements makes the daytime data much noisier than the nighttime data. As a consequence, the quality of data acquired in the lat/lon region will vary according to the local diurnal cycle.

    The additional noise in the daytime data especially prominent when CALIOP passes over bright clouds, where the noise magnitude above the stratus clouds is significantly larger than over the ocean surface. Retrievals of spatial and optical properties of layers lying above these clouds are typically much more uncertain than retrievals for layers lying above a dark ocean.

    The additional noise during daylight measurements also introduces additional uncertainty in the calibration of the lidar, and this is reflected in the greater uncertainty ascribed to optical parameters retrieved from daytime measurements relative to their nighttime counterparts.

    Another fundamental limitation on the quality of the CALIOP retrievals is the signal-to-noise ratio (SNR) of the measurement. For the uppermost layer in any profile, the SNR for nighttime data is largely a function of the lidar instrument state, and generally does not vary as a function of orbit location. However, for both nighttime and daytime measurements, the backscatter signal is attenuated and the SNR degrades as the lidar penetrates increasing amounts of optical depth. Thus measurements of an otherwise identical aerosol layer will have higher SNR for cloud-free conditions than (for example) on those occasions when there are cirrus clouds overhead.

    Some other conditions that may contribute to "inhomogeneous data quality" include the following:

    • In the South Atlantic Anomaly, excess radiation in the Van Allen Belts introduces additional noise into the backscatter measurement, thus reducing the quality of the nighttime calibration, which in turn can affect the accuracy of all subsequent parameters derived from this data.

    • Clouds and aerosols are distributed inhomogeneously around the globe, and therefore altitude and latitude dependent probability distribution functions (5D PDFs) are used in the cloud-aerosol discrimination (CAD). The CAD algorithm may perform differently from one point on the globe to another. In general, the discrimination between clouds and aerosols is worse at higher altitudes and latitudes where cirrus clouds and moderate dense nonspherical aerosol tend to be present simultaneously. In addition, the latitude resolution of the five-dimensional probability distribution functions (5D PDFs) used in the cloud-aerosol discrimination (CAD) algorithm is fairly coarse (10 degrees), and thus there are occasional CAD discontinuities at the boundaries.

    • There is a known flaw in the aerosol subtyping scheme that sometimes causes a misidentification of lofted smoke layers as marine layers. there is a geographical bias to the occurrence of this error, as it happens most frequently in those regions where dense smoke is transported out over the ocean.
  23. What are 'constrained cirrus cloud retrievals', how do I find them, and why should I bother to look for them?
  24. Constrained cirrus cloud retrievals are those cirrus cloud cases where it was possible to measure the backscatter molecular return below the cirrus cloud in a 'clear air' region and obtain a direct measurement of the cirrus cloud two-way transmittance or optical depth. This measured transmittance is then used to constrain the retrieval of the extinction profile of the cloud, thus reducing the uncertainty from using a default lidar ratio. These so-called constrained cirrus cloud retrievals are generally the highest quality retrievals of extinction provided by the lidar level 2 processing software.

    The Extinction_QC_Flag_532, and Extinction_QC_Flag_1064 data sets will have bit field 1 set for constrained retrievals. These data sets can be found the 5 km layer products and bin-by-bin in the 5 km profile products.


    LIDAR Level 1B Product

  25. Why are there negative attenuated backscatter values in the L1 lidar data? (Aren't these values completely non-physical???) Are these negative values removed before calculating the Level 2 Column Integrated Backscatter?
  26. The negative attenuated backscatter values arise from statistical variations in the CALIOP receiver photodectors, as explained in section 2.3 (page 11) CALIPSO lidar level ATBD.

    "The solar background signal can be significant - as large as the clear-sky atmospheric signal. The instrument measures the DC background of each profile from the signal acquired between 112 km and 97 km, where the laser backscatter signals are negligible. This DC signal is electronically subtracted from the analog profile before digitization to allow the dynamic range of the digitizer to be used most effectively. This subtraction will result in negative-going noise excursions if the laser backscatter signal is small."

    The negative values are not removed when computing column or layer integrated attenuated backscatters. In fact, doing so would introduce a bias into the calculation, as the lowest components of the random noise in the signal would be systematically removed, while the highest components would be retained.

  27. When I plot the lidar level 1B profile data, why are there discontinuities at ~8 km and again at ~20 km?
  28. Data are collected on board the satellite at a nominal vertical resolution of 15 m with a profile spacing of 333 m in the horizontal (along track) dimension. However do to bandwidth limitations the data is averaged on board before downlinking.

    The discontinuities that you may be observing at ~8 km and ~20 km correspond to the altitude where the horizontal averaging switches from 333 m to 1 km and then to 5/3 km ~(1.67). See the discussion in “Essential Reading” about the horizontal and vertical averaging.


    LIDAR Level 2 Layer Products

  29. In the layer products the data set Number_Layers_Found indicates that there are N (e.g. 3) layers in a column, does that mean there are N (e.g. 3) unique atmospheric layers in the column?
  30. Interpretation of the number of layers found parameter is straightforward for the 1-km and 1/3-km layer products: individual layers are always separated by regions of "clear air", and layer boundaries never overlap in the vertical dimension. However, this simplicity of interpretation does not always carry over into the 5-km cloud and aerosol layer products. CALIPSO uses a nested multi-grid feature finding algorithm, (also see the layer detection ATBD) , and thus the search for layer boundaries is conducted at multiple horizontal averaging resolutions. While the 1-km and 1/3-km layer products report only those features detected at, respectively, averaging resolutions of 1-km and 1/3-km, the 5-km products report layers detected at multiple averaging resolutions (5-km, 20-km, and 80-km in the version 3 products). Because the reporting resolution (5-km) is not always identical to the detection resolution, layers may appear to overlap in the vertical dimension.

  31. I have discovered, at least in the 5 km cloud layer files, that the top of the second cloud layer is often higher than the bottom of the first (top) layer. Can you explain to me what this means?
  32. The 5 km products report layers detected at multiple averaging resolutions (5 km, 20 km, and 80 km in the version 3 products). Because the reporting resolution (5-km) is not always identical to the detection resolution, layers may appear to overlap in the vertical dimension.

    Please read through the answer to question 6 (above), and read through the User Guide section in “Essential Reading” about the multi-grided averaging scheme.

  33. The file name of the 5 km layer products would seem to indicate that features are reported at 5 km, but in those files there are features reported at 20 and 80 km, what's up with that?
  34. We agree that there is a bit of a disconnect between the resolution of reported features and the file name, and that it may be confusing at first for those users familiar with remote sensing data from imaging sensors. The '5 km' is intended to indicate the highest resolution that the data is reported at, not necessarily the average resolution that was required to detect the feature.

    Do to the large variability in the measured attenuated backscatter that occurs in the atmosphere it was necessary to develop an algorithm that was capable of finding both weak and relatively homogenous layers (i.e. aerosols) and strong and highly variable features (i.e. water clouds). Please see our primer on the multi-grided averaging scheme for further information. Rather than report each horizontal averaging resolution in a separate file we decided to do you all a favor and report everything at the nominal 5 km horizontal resolution. The caveat is that 20 and 80 km features are only reported in the columns where it was found. Since high resolution data is replace by zeros before where ever features were found there are often 5 km columns and range bins where the higher resolution completely over-writes the 20 or 80 km feature.


    LIDAR Level 2 Profile Products

  35. In the profile products what is the difference between 'attenuated backscatter' and 'backscatter'?
  36. 'Attenuated backscatter' refers to the data products available in the lidar L1B profile product Total_Attenuated_Backscatter_532 or Attenuated_Backscatter_1064. These products are the geolocated, calibrated, and range-squared corrected raw lidar data. Note that these data are reported at a horizontal resolution of 333 m, however due to the satellite on-board averaging only data in the altitude range -0.5 to 8.2 km is available at that resolution. Data from higher alitudes are reported at the 333 m horizontal resolution but are over-sampled according to the figure shown here.

    'backscatter' refers to the Total_Backscatter_Coefficient_532 or Total_Backscatter_Coefficient_1064 available in the lidar level 2 5 km cloud or aerosol profile product. These products are the output from the lidar level 2 extinction retrieval algorithm (pdf). The cloud and aerosol backscatter and derivative products are reported at a uniform spatial resolution of 60-m vertically and 5-km horizontally, over a nominal altitude range from 20-km to -0.5-km for clouds, and over a broader altitude range of 30-km to -0.5-km for aerosols.

    Also note the following with regard to differences between the 'attenuated backscatter' reported in the Level 1 and Level 2 data products: At any altitude, the magnitudes of the measured backscatter coefficients (i.e., the level 1 attenuated backscatter coefficients) are lower than the corresponding magnitudes of the 'true' backscatter coefficients (i.e., the backscatter coefficients reported in the level 2 data products). This difference in magnitudes is due to the attenuation of the laser beam over the two-way path between the instrument and the altitude at which the measurement is made. One of the primary goals of the CALIOP level 2 processing is to determine how much attenuation has occurred, and then provide estimates of the true backscatter coefficients.

    Please consider reading the data quality summaries before using extinction data products, and don't forget to use QC flag information in your analyses.

  37. What's the difference between volume depolarization ratios and particulate depolarization ratios?
  38. The volume depolarization ratio refers to the depolarization ratio calculated from the attenuated backscatter data. Volume depolarization data sets are available in the lidar 5 km layer products as Integrated_Volume_Depolarization_Ratio, and Volume_Depolarization_Ratio_Statistics. The particulate depolarization ratio refers to the depolarization ratio calculated from the backscatter data, which is post-extinction data product. Particulate depolarization data sets are available in the lidar 5 km layer products as Integrated_Paticulate_Depolarization_Ratio, Particulate_Depolarization_Ratio_Statistics, and in the 5 km profile products as: Particulate_Depolarization_Ratio_Profile_532.

  39. Are layer boundaries and layer type information provided in the CALIOP profile products?
  40. Layer boundary information is NOT provided in the profile products, however layer top and base are provided the lidar level 2 layer products. While layer boundaries are not explicitly provided in the profile products it is possible to determine where boundaries are because valid data is only provide in the bins where layers have been found. All the other data bins are set to the FILL_VALUE for that particular data set.

    Users are also cautioned that the layer base of near-attenuating or fully attenuating features is only the apparent base of the feature, not the true base. The lidar is capable of penetrating to a level in the atmosphere with a one-way optical depth ~3.


    LIDAR Level 2 VFM Product

  41. Why does the surface occupy multiple range bins in the Vertical Feature Mask (VFM)?
  42. The CALIOP 532 nm channels do not recover instantaneously from very strong returns such as those from ocean and land surfaces. Instead, there is a decay which is approximately exponential.

    Due to this effect, there is often apparent lidar return from beneath the surface. This effect is particularly noticeable in Antarctica where frequent cloud-free atmospheres and a highly reflective surface result in surface signals which apparently extend significantly below the true surface.

    The lidar level 2 feature finding algorithm interprets and records the entire 'surface-spike' as a single feature, thus extending many range bins below the surface level.

    Users of Lidar Level 2 layer products can expect that the bases of strong scattering targets (i.e. optically dense water clouds) will be lower than expected. In most of these cases however, the observed layer is opaque to the lidar and the measurement of the true cloud base is not possible. The impact from the transient response to measurements of cirrus clouds and aerosol layers are considered negligible, please see the following document for further information.

  43. What information is provided by the Feature_Classification_Flag(s)?
  44. The Feature_Classification_Flag data set provides information on the type (e.g. aerosol, cloud, stratospheric), cloud phase, cloud or aerosol type, horizontal averaging, and qa flags, for all features detected.

    In the profile product, the Atmospheric_Volume_Description data set contains layer type information, that is found in the Feature_Classification_Flag data set. The Feature_Classification_Flag data set contains a bit field that indicates the feature type (e.g. cloud, aerosol, stratospheric, etc.).


    Miscellaneous

  45. How do I say CALIOP?
  46. CALIOP stands for Cloud-Aerosol LIdar with Orthogonal Polarization. It is the name of the lidar onboard the CALIPSO satellite, we like to pronounce it as the same as calliope, like the musical instrument (phonetically it is cal-eye-o-pee, not as cal-ee-op). See Merriam-Webster Dictionary for audio (use the left-hand speaker button for the correct pronunciation).




NASA
Last Updated: November 28, 2017
Curator: Charles R. Trepte
NASA Official: Charles R. Trepte

+ No Fear Acts
+ FOIA
+ Privacy

+ Office Of Inspector General
+ Agency Financial Reports

+ Contact NASA