• Created by Unknown User (mak), last modified on Sep 14, 2016

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 2 Next »

ODB API

  File Modified
No files shared here yet.

Change history

0.14.0

  • First beta version of new Python API, see examples in subdirectory src/api/  The intention is to provide an API conforming with PEP 249 Python Database API Specification v2.0 https://www.python.org/dev/peps/pep-0249/  The new API, unlike the old one, allows for writing files (using INSERT, see examples).

0.13.0

  • Preliminary, beta version of new API for C/C++ and Fortran, see api subdirectory for examples. All comments, suggestions and ideas for improvement will be much appreciated!

  • New SQL syntaxes:
    • CREATE TABLE for associating SQL tables with files and setting metadata of new files
    • 'SELECT ALL *' for reading heterogeneous files (different columns on different parts of a file), available via the new API (ODBQL)
    • INSERT for writing files, available via ODBQL
    • MATCH '(' expression_list ')' IN QUERY '(' select_statement ')'
  • ECML, the embedded scripting language based on MARS language syntax, has been merged into ODB API
  • ODB API has been integrated with MarsKit, which allows direct retrieval of data from MARS or ODB Server
  • ODB Server's functionality for local files available as part of the library
  • A tarball with ODB API and all its dependencies is available, see file odb_api_bundle-0.13.0-Source.tar.gz. To build MarsKit it is necessary to pass -DENABLE_GRIBPP=OFF to cmake

0.10.5

  • Added file group.txt to source code's etc directory. The file is needed for unit testing. An up to date version of the file is in http://apps.ecmwf.int/odbgov/group/csv/ , it is also installed in /usr/local/apps/odb_api/codes on ECMWF systems, and is used by MARS client when archiving ODB data. ODB_API_CODES environment variable can be used to change default location.

  • Added cmake variable EXTRA_ODB_MIGRATOR_LIBS. It can be used to pass additional libraries needed for building odb_migrator, like gomp when building with gfortran.
  • eckit is not distributed with this version. It is available for download here: ecKit Releases

0.10.4

  • Use /usr/local/apps/odb_api/codes/group.txt ( http://data-portal.ecmwf.int/odbgov/csv/Group/ ) instead of deprecated obsgroups.txt (MARS client support code),

  • Fix odb_migrator for cases when SQL SELECT defining data to be converted contained  functions or expressions other than simple column names on the select list,
  • When writing output files create subdirectories if present in path name,
  • Put back changes reverted by accident in change 6389d281bfb (this regression was released with 0.10.3): ODB-75 (LIKE/RLIKE, atoi), ODB-85 (hex bitfields formatting), ODB-108 (WHERE 0).

0.10.3

  • Improve memory consumption of odb split tool (ODB-121)

0.10.2

  • Fix odb_migrator, it was crashing when invoked with no parameters (ODB-91)

  • Fix AIX build issues.

0.10.1

  • Updated to use eckit 0.6.0 and ecbuild 1.3.2
  • Table in the SELECT's INTO clause can be defined with a CREATE TABLE statement (ODB-67)

  • odb sql tool has new flags (ODB-85):

    [--binary]            Print bitfields in binary notation

    [--no_alignment]      Do not align columns

  • Python API for legacy ODB added (documentation to be written)
  • ...

0.9.32

  • "ORDER BY 1" was not working correctly (ODB-49)
  • oda2request tool broken in ODB API 0.9.31 (wrong default path to configuration) (ODB-70)
  • CSV import tool was not working correctly for some bitfield columns (ODB-76)
  • ODB_DOUBLE constant added to Fortran bindings (ODB-77)
  • Python API for accessing old ODB databases (ODB-74)
  • Added odb_migrator option to specify missing value indicators different than default ones (only for REAL and INTEGER currently, DOUBLE always the same as REAL, BITFIELD always 0) (ODB-68)

            Syntax: 
               -mdi <type1:MDI1,type2:MDI2,...>
            e.g.:
              -mdi REAL:2147483647,INTEGER:2147483647

0.9.31

  • Fixed a bug introduced in 0.9.30, where operator >= was behaving like =.
  • Now ORDER BY clause accepts also column numbers, for example: SELECT int(lat/10), int(lon/10), count( * ) ORDER BY 3 DESC
  • Support for #include directive

0.9.30

  • Public header oda.h renamed to odb_api.h
  • Added functions for reading bitfields' metadata: odb_read_iterator_get_bitfield  and odb_select_iterator_get_bitfield to Fortran API
  • Disabled again fix for codec short_real, (ODB-15). The new codec short_real2 will be used by default when all ODB API applications at ECMWF are upgraded to use an ODB API version >= 0.9.24
  • Added syntax for range expressions in SQL WHERE clause, e.g.: <value1> <=  <expression> <=  <value2>

0.9.29

  • Fixed encoding of files containing columns of type DOUBLE (no packing) only.

0.9.28

  • Improvements to the CSV import tool, now it can import bitfields
  • Fixed queries with shift operator (#) appearing as arguments of functions
  • Fixed option -T (do not print header with column names) of the sql tool

0.9.27

  • Upgraded to eclib 7.3.0 - now wrapped in namespace eclib
  • Fixed ODB-26: some expressions with mixed aggregated and non-aggregated functions, e.g.: sum(obsvalue)+rms(obsvalue), were calculated incorrectly

0.9.26

  • Introduced in-memory data tables

0.9.25

  •  Introduced row shift operator (#) (ODB-2)

0.9.24

  • Fix an issue in SELECT DISTINCT ... INTO (some rows contained NULLs instead of correct values)
  • Fix odb header tool so it works properly when string columns contain empty strings (ODB-19)
  • Asynchronous IO can be used optionally when writing files by setting ODB_API_USE_AIO
  • Fix the split tool (ODB-17)

  • Added new codec short_real2 which should replace codec short_real in the future. NOTE: files created with this codec cannot be decoded with versions 0.9.23 and earlier. The new codec short_real2 is not used for encoding by default (ODB-15)

  • CodecOptimizer can be configured by setting ODB_DEFAULT_CODEC

0.9.23

  • Fixed a memory leak

  •  Add options -offset and -length to the sql tool:

            [-offset <offset>]    Start processing file at a given offset

            [-length <length>]    Process only given number of bytes of data

  • Add option -offsets to odb header tool, for printing basic information about blocks of data on a file:

            offset length numberOfRows

  • Fix for ODB-8: Missing values were not handled correctly by SQL formulae if results of SELECT saved on file
  • Semantics of SQL multiplication was different than in the old ODB: now 0 * NULL == 0, not NULL (ODBSUP-13)
  • Result of most SQL functions is now double precision (ODB-11)
  • Improved odb compare tool
  • Added verification to the split tool

 

  • No labels