What is a Fundamental Climate Data Record?

Fundamental climate data records (FCDRs) are a major focus in our project. An FCDR is, in general, the underlying dataset from which a climate data record (CDR) is generated. Most people are interested in the geophysical data -- the CDRs -- but the lower level data from which CDRs are retrieved are obviously an essential part of the story.

Can we define "FCDR" more precisely? This is important, because, ultimately, space agencies are responsible not only for running Earth observation missions, but also for curating the "raw" observations for the benefit of science and society. Having a clear definition of FCDR matters to how such agencies understand and approach this important responsibility.

I am aware of two published definitions:

• From the Global Climate Observing System (GCOS-154): The term “Fundamental Climate Data Record” (FCDR) denotes a well-characterized, long-term data record, usually involving a series of instruments, with potentially changing measurement approaches, but with overlaps and calibrations sufficient to allow the generation of products that are accurate and stable in both space and time to support climate applications. FCDRs are typically physical measurements such as calibrated radiances, backscatter of active instruments, or radio occultation bending angles. FCDRs also include the ancillary data used to calibrate them.
• From a US report NRC-2004 : Fundamental CDRs (FCDRs) are sensor data (e.g., calibrated radiances, brightness temperatures, radar backscatter) that have been improved and quality controlled over time, together with the ancillary data used to calibrate them.

Next week, at our FCDR users' workshop, I will be inviting participants to comment on how they would define an FCDR. Since these will be people to whom the question really matters, I hope to get some great input. Even if you are not there, you can comment using this page (comment box at bottom). Your feedback is welcome! Meanwhile, I will at the workshop propose a revised FCDR definition for discussion, which draws greatly on the two quotations above, but adds two perspectives on how "sensor data" should be "improved" when curating an FCDR:  harmonisation and uncertainty characterisation (two themes followers of this blog will recognise!).

My wording for the discussion:

Fundamental Climate Data Record

Definition: An FCDR consists of a continuous, harmonised record of calibrated, geolocated, uncertainty-quantified sensor observations in geophysical units (such as radiance), together with all ancillary and underlying data used to calibrate the observations and estimate uncertainty.

Explanation and further guidance:

1. This FCDR definition is a target. In the context of CDR derivation from the FCDR, this definition may exceed the minimum requirement (threshold) to derive societal and scientific benefits for some applications. In the following points, “should” is used to indicate conditions necessary to meet this target FCDR definition.
2. The FCDR should be continuous. During periods of overlaps of sensors in series, all overlapping data should be included in the FCDR, since overlaps can also be exploited when deriving CDRs to maximise stability.
3. The FCDR should be harmonised. Harmonisation is the recalibration of the sensor observations in geophysical units, using a stated reference and/or overlaps with other sensors in the series. The purpose of the recalibration is to bring consistency between sensors given the known (best estimate) differences in instrument characteristics (e.g., spectral response functions) between sensors. A harmonised FCDR should support more stable CDRs to be derived.
4. The FCDR comprises calibrated quantities in geophysical units together with the underlying data from which the calibrated quantities are derived (such as Earth view counts and calibration target counts). “Together with” does not necessarily mean “in the same file”, but it needs to be possible unambiguously to trace the underlying data from the calibrated data. This requirement enables both use of calibrated quantities to generate CDRs, and potential re-estimation of calibrated quantities should further research improve understanding of the instrument calibration.
5. The data in the FCDR should be located using a clearly defined co-ordinate system for geolocation and time, exploiting the best available estimates of orbital elements.
6. Quantitative uncertainty information should be provided with the FCDR observations. The form of uncertainty information should reflect the nature of the sensor and its error characteristics, as well as the requirements for quantifying uncertainty in derived CDRs. Examples of uncertainty information include: standard uncertainty, fractional uncertainty, and error covariance matrices. Uncertainty may be provided as data arrays or (where valid, in order to minimise data volumes) as parametric expressions to calculate uncertainty from other data in the FCDR. Uncertainty information should be provided per orbit/file, per scan or per datum as necessary to represent any significant variability of uncertainty, while taking account of data volume.
7. Similarly to comment 4 above, data underlying or ancillary to the uncertainty estimates should also be provided together with the uncertainty information in the FCDR.
8. Flags relating to assessment of data quality and instrument status should be made available in the FCDR together with the observations and their uncertainty. To increase usability, summary flags may be provided that give simple guidance to users about the quality status of pixels based on expert understanding of which of the flags and instrument conditions indicate questionable or invalid data.
9. The FCDR format should be consistent across the various sensors of the series. Versioning and informative metadata should be implemented, including links from calibrated observations to ancillary data (if not in the same files).

In a future blog, I will share a version revised after the workshop comments -- and any comments from you, Dear Reader.