The CDS and ADS have a regression testing system that can be used to verify that...

  1. The system as a whole is up and working
  2. Individual datasets/adaptors are up and working
  3. Requests deliver the expected files in the expected formats with the expected content. In particular, it can...
    1. Verify that a request returns the expected data (as described by a Python data structure) or
    2. Verify that two stacks return the same data, or
    3. Verify that two different datasets return the same data

Data comparisons are intelligent and, when data is not as expected, the system can specify exactly how it differs from expectation.

The system can be run either from the command line or as part of an ecFlow suite – useful for ad hoc and regular testing respectively. It is designed to bypass the caching system so data retrievals are properly tested.

All the code is in the cds-regression-testing repository.

Table of contents...