Problem

WARNING: httplib2.URLError received None <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed

Solution

The problem may be that the certifi package needs to be updated.

  • For Mac OS try the solution suggested on Stack Overflow: ssl-certificate verify failed with python3

    • Go to the folder where Python is installed, e.g., on Mac OS it is installed in the Applications folder with the folder name 'Python 3.x' where '3.x' is your Python version

    • Double click on 'Install Certificates.command'.

  • If you are not using Mac OS, or have a different setup and cannot find the 'Install Certificates.command' file, then try to update certif with pip

    pip install --upgrade certif

If this does not solve the problem, then it may be that the SSL library used in this Python version doesn't look in the correct certificate path.

  • Try from another environment.
  • Check which version of Python you are using and how it was installed and linked to openssl.
  • Check that ca-certificates are updated on your system. Important, the client, openssl on most systems, maintains a set of trusted root CAs that are used to validate the chain (intermediate cert > server cert) on a ssl connection, so they must be updated on the client.
    • Update OpenSSL, ca-certificates and Python.
    • The following are some references on how to update the ca-certificates package:
Mac OS X
    sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain ~/new-root-certificate.crt
    # Do not update OpenSSL through Homebrew, see https://stackoverflow.com/questions/15185661/update-openssl-on-os-x-with-homebrew
 
Windows
    # Type mmc.exe on search bar of the windows menu
    # Go to File > Add / Remove Snap In > Select Certificates > Click Add > Click OK
    # Expand Certificates - Current User
    # Right Click on Trusted Root Certification Authorities > All Tasks > Import
    # Follow the steps
 
    # alternatively
    certutil -addstore -f "ROOT" new-root-certificate.crt
 
Linux (Ubuntu, Debian)
    apt install ca-certificates
    # Copy your CA to dir /usr/local/share/ca-certificates/
    sudo cp foo.crt /usr/local/share/ca-certificates/foo.crt
    sudo update-ca-certificates
 
Linux (CentOs 6)
    yum install ca-certificates
    #Enable the dynamic CA configuration feature:
    update-ca-trust force-enable
 
    #Add it as a new file to /etc/pki/ca-trust/source/anchors/:
    cp foo.crt /etc/pki/ca-trust/source/anchors/
 
    update-ca-trust extract
 
Linux (CentOs 5)
    #Append your trusted certificate to file /etc/pki/tls/certs/ca-bundle.crt
    cat foo.crt >> /etc/pki/tls/certs/ca-bundle.crt
 
Anaconda
    conda update openssl
    • In case you need to add a specific Root Certificate, run the following command (The browser can be used to get the same information, if not on a *nix system, on view certificate):
user@local:~> openssl s_client -connect api.ecmwf.int:443                                                                                 
CONNECTED(00000003)
depth=2 C = BM, O = QuoVadis Limited, CN = QuoVadis Root CA 2 G3
...

Here we can see QuoVadis Root CA 2 G3 is the Root CA for api.ecmwf.int  and can be downloaded from https://www.quovadisglobal.com/download-roots-crl/ and installed locally (as per the commands specified above on the specific OS).

If you have the SSL error in the last step of your request (Transferring from...), you need to install the Root CA for "stream.ecmwf.int": QuoVadis Global SSL ICA G2

  • The following is a hack for cases when updates are not possible for whatever reason:

    • Disable SSL Verification, this can be achieved by setting CURL_CA_BUNDLE="" before calling the python api:
      • CURL_CA_BUNDLE="" python main.py
    • Specify the Root CA directly, this can be achieved by setting REQUESTS_CA_BUNDLE="path to ROOT ca QuoVadis Root CA 2 G3" downloaded from the Quovadis Website (that your system cannot find somehow):
      • REQUESTS_CA_BUNDLE="/path_to_cert/QuoVadis_Root_CA_2_G3.pem" python main.py