Page History

Who to contact: servicedesk@ecmwf.int

What information you will need to pass:

• Your login
• Your job id 
• The problem you are experiencing

Where to find this information:

• All this information can be found on your job list (see next paragraph) or
  
• In the service response (in case of a Web-API  script, look the log messages returned by the server)
• A lot of useful information can be also found on the MARS activity page

If you kill a Web-API request on your local environment (e.g. by CTRL+C), the corresponding job, on the web-API service level, is NOT cancelled but still running. In that case please keep in mind the following:

• Your request will continue to be visible in your job list
• If you wish to cancel it please use the corresponding option on your job list
• Once you have cancelled it, the request status will become aborted.
• Due to the "max active requests per user" limitation, killing or cancelling requests may affect the performance of your other submitted requests.

• Smaller Nr of tapes (a request accesses) means better request efficiency.
• Ideally requests should not access more than 1 tape.
• Requests for data which is online in general go faster
• All this information above, can be found in MARS activity

• If your request is "aborted" please check the log messages, find the "error"  and follow the troubleshooting instructions below.
• If the problem is not included  in the troubleshooting list below, please contact us.

'mars - ERROR  - 20160712.064620 - Request validation failed:',
'mars - ERROR  - 20160712.064620 - restricted access to valid data.',
'mars - ERROR  - 20160712.064620 - For more information, look at ',
'mars - ERROR  - 20160712.064620 - https://software.ecmwf.int/wiki/display/UDOC/MARS+access+restrictions#valid.',
'mars - ERROR  - 20160712.064620 - For any queries, please, contact User Support',
'mars - ERROR  - 20160712.064620 - Some errors reported',

• Your request will be  "aborted" if you are trying to access restricted data.
• For more information please have a look at MARS access restrictions

mars - ERROR  - 20160712.133205 - UserError: Cannot retrieve more 
than XXX.XXX fields in one request via the WebAPI. Please, split your 
request [marsode]
mars - ERROR  - 20160712.133205 - Error code is -2
mars - ERROR  - 20160712.133205 - Request failed
mars - INFO   - 20160712.133205 - Request time:  wall: 1 sec
mars - ERROR  - 20160712.133205 - Some errors reported

• On Web-API some limitations have been set to improve the service.  Check the current limits here: News feed 
• In this case, your job has reached the limit of XXX.XXX fields per request. Please split your request into smaller sub requests.

• On Web-API some limitations have been set to improve the service.  Check the current limits here: News feed 
• In this case your job has reached the limit of XXGB per request. Please split your request into smaller sub requests.

if you are using the Access ECMWF Public Datasets service and you get this error it means that the mandatory keyword "dataset" is not set. Review the request and add this keyword.

If you are an authorized user with MARS access you can use WebAPI FAQ#Can I use the Python requests to retrieve data using API MARS service?

The datasets interim_daily and interim_full do not exists any more. Please change your request to ask for the dataset interim instead. Please note that interim is now equivalent to the interim_full, the full resolution dataset. To get the equivalent to the interim_daily dataset, you will need to set "grid" to "1.5/1.5" in your request.

• You are not authorised to use service MARS.
• Make sure you have logged in https://apps.ecmwf.int/auth/login/ and then retrieve your key at https://api.ecmwf.int/v1/key/.
• For this, you will need to have an authorized account on ECMWF. See https://www.ecmwf.int/en/forecasts/accessing-forecasts/order-historical-datasets

WARNING: HTTP received 429
{
  "message": "API rate limit exceeded.",
  "type": "rate_limit_exceeded",
  "documentation_url": "https://software.ecmwf.int/wiki/display/WEBAPI/Web-API+Troubleshooting#Web-APITroubleshooting-HTTP429errors"
}
Error contacting the WebAPI, retrying in 60 seconds ...

Please check the ECMWF Service Status before reporting the problem

If the WEB-SERVICES status is green, it might be an Internal ECMWF server issue. Please report the problem

{
    "url"   : "https://api.ecmwf.int/v1",
    "key"   : "XXXXXXXXXXXXXXXXXXXXXX",
    "email" : "john.smith@example.com"
}

• Make sure you get a key from https://api.ecmwf.int/v1/key/.
• This key needs to be placed in ~/.ecmwfapirc.

>>> import socket
>>> socket.ssl
<function ssl at 0x4038b0>

Symptoms: A Python WebAPI script terminates with "Error contacting the WebAPI, retrying in 60 seconds ...".  The WebAPI retries 10 times, followed by "WARNING: httplib2.URLError received None <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed"

The problem may be that the SSL library used in this Python version doesn't look in the correct certificate path.

• Try from another environment.
• Check which version of Python you are using and how it was installed and linked to openssl.
• Check that ca-certificates are updated on your system. Important, the client, openssl on most systems, maintains a set of trusted root CAs that are used to validate the chain (intermediate cert > server cert) on a ssl connection, so they must be updated on the client.
  • Update OpenSSL, ca-certificates and Python.

• • The following are some references on how to update the ca-certificates package:

Mac OS X
    sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain ~/new-root-certificate.crt
    # Do not update OpenSSL through Homebrew, see https://stackoverflow.com/questions/15185661/update-openssl-on-os-x-with-homebrew

Windows
	# Type mmc.exe on search bar of the windows menu
	# Go to File > Add / Remove Snap In > Select Certificates > Click Add > Click OK
	# Expand Certificates - Current User
	# Right Click on Trusted Root Certification Authorities > All Tasks > Import
	# Follow the steps

	# alternatively
    certutil -addstore -f "ROOT" new-root-certificate.crt

Linux (Ubuntu, Debian)
    apt install ca-certificates
    # Copy your CA to dir /usr/local/share/ca-certificates/
    sudo cp foo.crt /usr/local/share/ca-certificates/foo.crt
    sudo update-ca-certificates

Linux (CentOs 6)
    yum install ca-certificates
    #Enable the dynamic CA configuration feature:
    update-ca-trust force-enable

    #Add it as a new file to /etc/pki/ca-trust/source/anchors/:
    cp foo.crt /etc/pki/ca-trust/source/anchors/

    update-ca-trust extract

Linux (CentOs 5)
    #Append your trusted certificate to file /etc/pki/tls/certs/ca-bundle.crt
    cat foo.crt >> /etc/pki/tls/certs/ca-bundle.crt

Anaconda
	conda update openssl

• • In case you need to add a specific Root Certificate, run the following command (The browser can be used to get the same information, if not on a *nix system, on view certificate):

user@local:~> openssl s_client -connect api.ecmwf.int:443                                                                                  
CONNECTED(00000003)
depth=2 C = BM, O = QuoVadis Limited, CN = QuoVadis Root CA 2 G3
...

Here we can see QuoVadis Root CA 2 G3 is the Root CA for api.ecmwf.int and can be downloaded from https://www.quovadisglobal.com/QVRepository/DownloadRootsAndCRL.aspx and installed locally (as per the commands specified above on the specific OS).

If you have the SSL error in the last step of your request (Transferring from...), you need to install the Root CA for "stream.ecmwf.int": QuoVadis Global SSL ICA G2

• The following is an hack for cases when updates are not possible for whatever reason:

  • Disable SSL Verification, this can be achieved by setting CURL_CA_BUNDLE="" before calling the python api:
    • CURL_CA_BUNDLE="" python main.py

• • Specify the Root CA directly, this can be achieved by setting REQUESTS_CA_BUNDLE="path to ROOT ca QuoVadis Root CA 2 G3" downloaded from the Quovadis Website (that your system cannot find somehow):
    • REQUESTS_CA_BUNDLE="/path_to_cert/QuoVadis_Root_CA_2_G3.pem" python main.py

• Connectivity network issues.
• Local disk space or file systems issues

Check if the results are still available in your job list

If not, you may wish to rerun the request

Check if you have installed the latest api client from our Web-API Downloads

export PERL_LWP_SSL_VERIFY_HOSTNAME=0

BEGIN { $ENV{"PERL_LWP_SSL_VERIFY_HOSTNAME"} = 0; }

Status: rejected

Reason: The total Nr of queued requests per user, allowed, on the API service has been exceeded

Current limit:  20 queued requests per user

Action: Please limit the Nr of your queued requests and try later

Status: rejected

Reason: The total Nr of queued requests in the API service has been exceeded. Presumably the API service is currently busy. Please accept our apologies for any inconvenience this may cause.

Current limit: 2000 requests in total

Action:  try  later you may also wish to check the  Web-API activity

Status: rejected

Reason: The total Nr of queued requests for the C3S ERA 5, in the API service has been exceeded.

Current limit:  4 queued requests per user for C3S ERA 5

Action:  ERA 5 is now available via the CDS, see how to migrate your scripts C3S ERA5: Web API to CDS API

Status: rejected

Reason: The Nr of fields per request  has been exceeded.

Current limit: 600.000 fields per request.

Action:  Split your request

Status: rejected

Reason:This dataset has been phased out and it is not any more available from our services.  It is likely that the dataset has been superseded by a new dataset which is currently available.

Action: See the corresponding link for more details or contact us.

Status: rejected

Reason: The dataset is not any more available via the ECMWF public datasets. It has been either migrated to another service or phased-out etc

Action:  see below

Status: rejected

Reason: The dataset has been phased out. In most cases a phased out datasets has been superseded by another dataset.

Action: We encourage users to have a look on the project documentation.  Contact servicedesk@ecmwf.int if you have any further questions.

For any kind of ecCodes issues please visit the  ecCodes Frequently Asked Questions

• If you are experiencing a MARS "error" please check also the General MARS Troubleshooting page.
• Some additional information on some specific issues has  also been added in the list  below.

• In this case the application is attempting to connect to one of the MARS  archive servers, which is down.
• Please check the ECMWF Service Status before reporting the problem

• MARS error
• Please report the problem

• Status: aborted
• Reason: The volume of your request is too large 
• Options: Split your request to smaller ones
  

• Status: aborted
• Reason: You are requesting too many fields
• Options: Split your request to smaller ones
  

• Status: aborted
• Reason: You try to access restricted data
• Options: Adapt your request
  

• Status: aborted
• Reason:  Retrieved Nr of fields != expected Nr of fields. Some of the fields that you have requested are not available. eg some S2S parameters are not available from all pressure levels.
• Options:
  • If you have access to the Archive Catalogue, you can check data available here: https://apps.ecmwf.int/archive-catalogue/
  • If you have access to the MARS Catalogue, you can check the data available here: http://apps.ecmwf.int/mars-catalogue/
  • For Public Datasets, check the availability of the dataset using the ECMWF Public Datasets web interface and adapt your request to reflect the availability.

• Status: aborted
• Reason:  This particular period of ERA 5 (1979-1999 )  will be made available only via the Copernicus Climate Data Store (CDS) shortly
  
• Options: Please migrate your scripts to the new service and wait until the data is made available.

For any other MARS problem you can visit the MARS Trouble-shooting

The problem is  that you repeat some MARS keyword values

 see also

https://confluence.ecmwf.int/display/UDOC/Retrieve#Retrieve-Assertionfailed