Teleport is the replacement for the ECaccess SSH service. Both services are currently available and can be widely used at this point.
However, once ECMWF's systems are fully operational in the Bologna Data Centre and the Reading-based hosts (the Cray HPCF, ECGATE and others) have been decommissioned, Teleport SSH will be the only way to connect using ssh to ECMWF's Bologna-based hosts (the Atos HPCF and ECGATE Services). The ECaccess service will not support interactive ssh connections to the Bologna-based hosts.
Teleport is software which provides an SSH Jump Host (or Bastion host) service in a secure, modern way, with support for role-based access control and single sign-on. At the moment there are two gateways available:
- jump.ecmwf.int based in Bologna Datacentre, best to access Bologna-based hosts (Atos HPCF)
- shell.ecmwf.int based in Reading Datacentre, best to access Reading-based hosts (Cray HPCF, ECGATE and others).
The Teleport gateway SSH Host Keys are currently:
jump.ecmwf.int SHA256:yAJzmK5nCp9R0TIxkfdaLjJt3uf3S0QkxjGX18Ws5EM
shell.ecmwf.int
SHA256:ST5P3QlRZdI88o79ozjPdp0+FWTczckLTKzGD2z3xmU
Please report any feedback or issues through the ECMWF Support Portal .
Overview
The Teleport service provides:
- Single SSH hop from client systems anywhere on the internet to servers inside ECMWF (ecGate, HPC, etc)
- Re-authentication required only every 12 hours (once per day)
- Integration with standard tools such as the OpenSSH ssh client, scp, and ssh-agent
- Web-SSH interface for in-browser terminal access, with scp
- X11 and Port forwarding
The single sign-on step is performed using an application called "tsh
", every 12 hours.
After that you use standard ssh or scp to connect to systems inside ECMWF.
Alternatively you can have simple terminal access in a web browser.
Downloading tsh
The tsh
application is required to perform user authentication once every 12 hours.
MacOS users can use Homebrew for installation: brew install teleport
.
Alternatively, and for Windows, download from the Gravitational website and place into your $PATH
.
The binary is available for Linux 32-bit, 64-bit, and ARM, as well as Windows 64-bit and a signed package for MacOS.
User Authentication
Once every 12 hours, you will need to refresh your tokens with the tsh
command, through your web browser.
SSH connections can remain active for longer than 12 hours, but new ones will require re-authentication.
First Time
Run tsh
, giving the location of our gateways. For Bologna hosts such as the Atos HPCF:
tsh login --proxy=jump.ecmwf.int
or for platforms based in Reading such as ECGATE or Cray HPCF:
tsh login --proxy=shell.ecmwf.int
Your default web browser will open and you should login with your email address, ECMWF password, and then HID (ActivID) Token code.
If you're already logged in to the ECMWF website, or have recently logged in to this service, the password prompt might be skipped.
Subsequent Occasions
if you only use one of the gateways, on subsequent occasions you may just do:
tsh login
If you use both, you will need to specify which one you want to login, otherwise it will just pick the last one used.
Using both gateways simultaneously
There is currently a limitation in the SSH agent that can only use one set of keys (the one for the last tsh login). If you wish to use both on the same session, you would need to tell ssh what keys need to be used explicitly. One way to do it is to define the following entry in your ~/.ssh/config (replace <email_address> by the actual value.
# Teleport gateways # Temporary workaround to allow concurrent usage of both gateways Host shell.ecmwf.int jump.ecmwf.int IdentityFile ~/.tsh/keys/%h/<email_address> CertificateFile ~/.tsh/keys/%h/<email_address>-ssh/%h-cert.pub HostKeyAlgorithms +ssh-rsa*,rsa-sha2-512 PubkeyAcceptedKeyTypes +ssh-rsa*
If you are not sure what email address you need to use, just run the tsh login once and run the following:
ls ~/.tsh/keys/*/*.pub
Connecting to hosts through the gateway
Windows users should skip to our Guide for Windows SSH to ECMWF.
OpenSSH 7.3 or later has a simple command line option to connect via our gateway to the destination-host
:
ssh -J username@jump.ecmwf.int username@destination-host ssh -J username@shell.ecmwf.int username@destination-host
For example, if your username is ab0
and you wish to connect to Atos HPCF:
ssh -J ab0@jump.ecmwf.int ab0@hpc-login
# For users without HPC access: ssh -J ab0@jump.ecmwf.int ab0@ecs-login
And if you wish to connect tp ecgate
:
ssh -J ab0@shell.ecmwf.int ab0@ecgate
The OpenSSH configuration setting for this is named ProxyJump,
e.g. add the following lines in the ~/.ssh/config
file on you client system:
Host jump.ecmwf.int shell.ecmwf.int HostKeyAlgorithms +ssh-rsa*,rsa-sha2-512 PubkeyAcceptedKeyTypes +ssh-rsa* User ab0 # For ecgate and Cray HPCF Host ecg* cc* HostKeyAlgorithms +ssh-rsa*,rsa-sha2-512 PubkeyAcceptedKeyTypes +ssh-rsa* User ab0 ProxyJump shell.ecmwf.int # For Atos HPCF Host a?-* a??-* hpc-* hpc2020-* ecs-* HostKeyAlgorithms +ssh-rsa*,rsa-sha2-512 PubkeyAcceptedKeyTypes +ssh-rsa* User ab0 ProxyJump jump.ecmwf.int
See the Legacy Configuration note below if your ssh client is older than 7.3.
If your connection fails after working for some time, it could be because your tokens have expired. You can check them:
$ tsh status > Profile URL: https://jump.ecmwf.int:443 Logged in as: firstname.lastname@ecmwf.int Cluster: jump.ecmwf.int Roles: * Logins: ab0 Valid until: 2020-06-22 23:26:30 +0100 BST [EXPIRED] Extensions: permit-X11-forwarding, permit-agent-forwarding, permit-port-forwarding, permit-pty
From OpenSSH 8.4 the client may refuse to connect with the cryptic message: "Connection closed by UNKNOWN port 65535".
This is because the Teleport system has to remain compatible with some old OpenSSH server versions. The fix is to add this extra lines to your OpenSSH configuration:
Host jump.ecmwf.int shell.ecmwf.int HostKeyAlgorithms +ssh-rsa*,rsa-sha2-512 PubkeyAcceptedKeyTypes +ssh-rsa* User ab0 Host ecg* cc* User ab0 HostKeyAlgorithms +ssh-rsa*,rsa-sha2-512 PubkeyAcceptedKeyTypes +ssh-rsa* Host a?-* a??-* hpc-* hpc2020-* ecs-* HostKeyAlgorithms +ssh-rsa*,rsa-sha2-512 PubkeyAcceptedKeyTypes +ssh-rsa* User ab0 ProxyJump jump.ecmwf.int
Destination Hosts available
The hosts directly available through the Bolgona Teleport gateway (jump.ecmwf.int):
- Atos HPCF, including ECS
For hosts based in Reading through the Reading Teleport gateway (shell.ecmwf.int) are:
- ECGATE (interactive node only)
- CCA and CCB login nodes
Configuring password-less login
This configuration enables single-hop ssh (using ProxyJump) to other ECMWF hosts.
Not required for ECGATE, CCA/CCB or Atos HPCF login nodes and Linux VDI.
Add the Teleport certificate authority to your ~/.ssh/authorized_keys
file, on the relevant system at ECMWF, e.g. ecgate, cca:
curl -fs https://nexus.ecmwf.int/repository/internal-teleport-configs/prod/teleport_user_ca.pub >> ~/.ssh/authorized_keys
On cca/ccb, you will need to load the curl module beforehand.
This configuration will allow access to any host which mounts the same $HOME
directory.
Terminal Access in a Web Browser
You can open a tabbed terminal in the web browser, with support for SCP upload and download.
Only available for ECGATE and CCA/CCB login nodes.
http://webshell.ecmwf.int/destination-host/username
(replacing destination-host and username with your selection).For example, if your username is ab0
and you wish to connect to ecgate
:
http://webshell.ecmwf.int/ecgate/ab0
The destination host should be just the short name, without "ecmwf.int".
If you open another tab inside the web terminal, use the QUICK LAUNCH box and enter "username@destination-host:22
", for example ab0@ecgate:22
.
The web terminal works very well to access tmux
or screen
running on the destination host.
Windows Clients
There are various ways to initiate SSH from Windows 10, so it depends on your system and your preferences.
We recommend using the Windows Subsystem for Linux if you can (on your own machine), downloading the Teleport Linux client for it, followed by starting the SSH Agent and then connecting as for Linux/MacOS systems.
Alternatively, you can use:
Other Notes
SSH Agent is required
If you have logged in but ssh fails to connect, it may be that your SSH agent is not running.
The Agent can be started and tokens refreshed this way:
eval $(ssh-agent -s) tsh logout tsh login --proxy=jump.ecmwf.int
And this will make sure the Agent continues to run in your environment:
echo 'eval $(ssh-agent -s)' >> ~/.bash_profile # or ~/.bashrc on WSL
(if you're using WSL creating a .bash_profile stops .bashrc being executed when you start a shell session, therefore use ~/.bashrc instead here)
SCP, X11, Agent, and Port Forwarding
SCP, Agent forwarding, X11 forwarding, and Port forwarding (including SOCKS proxy), all work through the Teleport gateway. A nice application of port forwarding with Teleport is the use of the ecflow_ui client on your local system to follow ecflow suites running at ECMWF. See Teleport - using local ecflow_ui for more details.
For scp
you can use the -o
option:
scp -o ProxyJump=ab0@jump.ecmwf.int ab0@hpc-login:/remote/file/path /local/file/path
Avoiding storing SSH Keys
You may wish to avoid storing SSH Keys to disk and always (and only) use the SSH Agent.
tsh
client versions 7.1.0 onwards have this feature:
-k, --add-keys-to-agent Controls how keys are handled. Valid values are [auto no yes only].
X11 under macOS
$ brew install --cask xquartz # start xquartz app $ export DISPLAY=:0 $ ssh -X ....
Legacy Configuration
For OpenSSH clients older than 7.3, the following configuration can be used:
# ~/.ssh/config file: Host a?-* a??-* ecs-* hpc-* hpc2020-* User ab0 ProxyCommand /usr/bin/ssh -q -W %h:%p ab0@jump.ecmwf.int
Similar setup can be configured to use jump.ecmwf.int for Atos HPCF and other Bologna-based hosts.
Difficult client environments
You might not be able to download and run tsh
, or access our web login service, or run the SSH Agent, from where you wish to use ssh.
Instead you can use (or copy) the identity files which tsh
stores in $HOME
:
# ~/.ssh/config file: Host ecgate User ab0 IdentityFile ~/.tsh/keys/shell.ecmwf.int/firstname.lastname@ecmwf.int ProxyCommand /usr/bin/ssh -q -i ~/.tsh/keys/shell.ecmwf.int/firstname.lastname@ecmwf.int -W %h:%p ab0@shell.ecmwf.int
This is a good way to access Teleport credentials via a shared file system from any host.
Similar setup can be configured to use jump.ecmwf.int for Atos HPCF and other Bologna-based hosts.
Even more difficult client environments
Recent Fedora Linux distributions (such as Fedora-33) using OpenSSH 8.4p1 no longer accept the "ssh-rsa" signature scheme using the SHA-1 hash algorithm in conjunction with the RSA public key algorithm.
As a workaround for this problem, you may need to add ssh-rsa (rsa-sha2-256 or rsa-sha2-512 can also be used) as a PubkeyAcceptedKeyTypes to your ~/.ssh/config file:
# ~/.ssh/config file: Host jump.ecmwf.int shell.ecmwf.int IdentityFile ~/.tsh/keys/%h/your@email.com CertificateFile ~/.tsh/keys/%h/your@email.com-ssh/%h-cert.pub HostKeyAlgorithms +ssh-rsa*,rsa-sha2-512 PubkeyAcceptedKeyTypes +ssh-rsa* User ab0 Host a?-* a??-* ecs-* hpc-* hpc2020-* User ab0 ProxyJump jump.ecmwf.int IdentityFile ~/.tsh/keys/jump.ecmwf.int/your@email.com CertificateFile ~/.tsh/keys/jump.ecmwf.int/your@email.com-ssh/jump.ecmwf.int-cert.pub
Note you would need to replace your your@email.com
by your registered email address at ECMWF and ab0
by your own ECMWF user id.
Network Requirements
The service is configured to use only standard ports 22, 80, and 443, to help with access wherever users are.
Additional configuration at the local user site may be required to allow connections out. The diagram below shows the TCP ports and destination hosts used.
Versions 8.3.5 and newer of the tsh
application support use of proxies (HTTP_PROXY
, etc). However, if the HTTPS_PROXY
, etc, environment variable is set, tsh
will also attempt to make an ssh connection on port 22 using the proxy. If your proxy does not support ssh connections to port 22 then set:
$ export no_proxy="shell.ecmwf.int:22,jump.ecmwf.int:22"
- The "
tsh login
" step uses ports 80 and 443 in order to log in to the service and obtain the client certificate.- initially it contacts shell.ecmwf.int at port 443 (user is able to see these steps by using tsh login --debug switch)
- then it opens a local http client on a high port 64xxx, produces a link on the localhost for the user to follow (like http://127.0.0.1:64068/da92794b-9d41-4008-ae6f-83fb77f64486) and waits for a callback from shell.ecmwf.int
- that localhost url then redirects user for OIDC authentication at https://accounts.ecmwf.int (port 443) involving Keycloak linked to user accounts on AD and HID token
- upon successful authentication, tsh receives a callback from shell.ecmwf.int and receives the client certificate completing it's login workflow
- from this point on, with the client certificate which is valid for 24 hours, user is authorised to access hosts behind the teleport proxy either via tsh ssh or OpenSSH workflows
- Your ssh client uses standard port 22 for server access.
- The web shell service also uses port 443 on the same host.