It is possible to connect local ecflow_ui running on your mac/pc to the remote servers running at the Centre via the Centre's Teleport gateway.Run ecflow_ui at home and interact with ecFlow suites as if you were at the Centre!

Setting up

...

SSH access

You As a prerequisite, you will need to install Teleport client on your laptop. SSH access to the Centre. ECMWF Teleport gateway offers relatively hassle-free SSH access service. Follow instructions on the Centre's User Documentation pages how to install and set up the Teleport client on your laptop.

You also Each time you connect to the Centre via ssh, you will need a valid Teleport SSH certificate on your laptop. Certificates expire after 12 hours. Fresh certificate is can be obtained from the Centre's Teleport service by executing "tsh login" command in a terminal. You may be asked  command on the laptop. These certificates are valid for 12 hours. "tsh login" may ask you for your ECMWF login credentials (security token). If you already have a valid certificate, the this authentication prompt will be skipped.

Connecting

...

to ecFlow servers at the Centre

...

Here we describe two methods of connecting ecflow_ui via Teleport gateway with ecFlow servers running at the Centre.

Method 1: Local Port Forwarding

Once you have installed and set up Teleport client, you can start the SSH local port forwarding session. The session will tunnel network traffic between your ecflow_ui and the Centre's servers:

• ecflow_ui ↔ ECMWF ecFlow servers
• ecflow_ui ↔ ECMWF logservers (but see the "Limitations" section)

via SSH

At home, you don't have direct access to the Centre's ecFlow servers. However, you can set up such access using SSH. The basic idea is that the SSH daemon running on your ECMWF workstation will "pretend" to be an ecflow_ui, forwarding network connections from your ecflow_ui at home to the Centre's ecFlow servers. There are two ways of doing it. Both methods rely on the Port Forwarding functionality built into OpenSSH software suite. 

Method #1: Local Port Forwarding

You will start SSH local port forwarding session on your laptop and connect your ecflow_ui to it. The local SSH session will forward network connections to the SSH daemon running on your ECMWF workstation and from there to the Centre's ecFlow servers.

Start SSH port forwarding session

Using 'user1' login  name as an example, please replace with your own username used at ECMWF. Here I wish to connect to ECMWF workstation hostname machine1 where I have a ecflow ecFlow server running on port 4141, also wish to connect to host machine2 where I have a ecflow ecFlow server running on port 3142 (replace "user1" with your ECMWF username and "machine1", "machine2" with real ECMWF hostnames). To start the SSH tunnel session on your laptop, execute:

ssh -J user1@shelluser1@jump.ecmwf.int user1@machine1 -C -N -L 4141:machine1:4141 -L 3142:machine2:3142

The ssh SSH session will keep running in the terminal, listening on local ports 4141 and 3142 and printing various log messages as it forwards your ecflow ecFlow network traffic to the Centre.

To access HPC job output via the log serverlogserver, you need to create establish a tunnel access for this as well. Here we use 'logserver' as an example (replace this with your an actual logserver). You also need to add '127.0.0.1 logserver' alias in /etc/hosts so that connections to 'logserver' actually go to localhost. Then use

ssh -J user1@shelluser1@jump.ecmwf.int user1@machine1 -C -N -L 4141:machine1:4141 -L 3142:machine2:3142 -L 9316:logserver:9316

Configure your ecflow_ui

Now you will connect your ecflow_ui to your local SSH port forwarding session. In ecflow_ui, edit ecFlow connection settings (Servers→ Manage Servers ...). Make sure to Next, configure the connection settings for the two ecflow servers in your ecflow_ui. Make sure you use "localhost" in the "Host" fields, not the actual ECMWF hostnames. 

Limitations of Local Port Forwarding:

...

 Redirect logserver requests to localhost

In Section 2.1.1 you have also established a tunnel for forwarding logserver traffic. In order to send the log requests from ecflow_ui to the local tunnel entrance, you need to add "127.0.0.1 logserver" alias in /etc/hosts on your laptop (replace "logserver" with an actual ECMWF logserver name).

In practice, using the Local Port Forwarding method we can only connect to a single

...

ECMWF logserver. This is because:

• different log servers logservers at the Centre use the same port number (9316).
• we cannot map all of them to a single 127.0.0.1:9316 local endpoint
• we could map each logserver:9316 to a different local port number, but we cannot reconfigure ecflow_ui to use these local port numbers (i.e. we cannot change ECF_LOGPORT=9316 defined on the ecFlow server without affecting other users).

Method

...

#2: Dynamic Port Forwarding

SSH also offers Dynamic Port Forwarding, ia.ek. it can act as a. SOCKS proxy functionality. This method has some advantages over Local Port Forwarding:

• There is no need to manually specify port number mapping for each ecflow server
• You can use original host names in the ecFlow servers connection settigssettings, instead of 'localhost'
• You are not limited to a single logserver; there is also no need to modify /etc/hosts file.

However, to use Dynamic Port Forwardingthe proxy, the client application must understand (ecflow_ui) must be able to speak SOCKS protocol. Here we We will show how to start SOCKS proxy and how to make enable it for ecflow_ui speak SOCKS. SOCKS proxy will tunnel all network traffic from ecflow_ui to the servers running inside the Centre:

• ecflow_ui ↔ ECMWF ecFlow servers
• ecflow_ui ↔ ECMWF ecFlow logservers
• ecflow_ui ↔ ECMWF DNS servers

.

NOTE: if you were using Local Port Forwarding method, you will now need to:

• Remove "127.0.0.1 logserver" from your /etc/hosts
• Use actual ECMWF hostnames in the "Host" fields of ecflow_ui connection settings, instead of "localhost".

Start SOCKS proxy session

In a terminal on your laptop, start the SOCK proxy (Dynamic Port Forwarding) session Start your SOCKS proxy in a terminal with:

% ssh -v -C -N -D9050D 9050 -J myecuser@shellmyecuser@jump.ecmwf.int myecuser@myecworkstation

(replace "myecuser" and "myecworkstation" with your real ECMWF username and workstation name).

The ssh session will keep running in the terminal, listening on local port 9050 and printing various log messages as it forwards your network traffic to the Centre.

...

Applications which want to use SOCKS proxy must speak SOCKS protocol. Some application applications can be " SOCKS-ified " using a tool called proxychains. The tool intercepts the application's network traffic, adds SOCKS a protocol layer and redirects traffic to the proxy. Luckily, the ecflow_ui.x executable can be SOCKS-ified this way.

First, install proxychains in your laptop. On MacOS, you can install proxychains tool do it with "brew install proxychains-ng". Many Some Linux distributions come with proxychains tool preinstalled.

By default, proxychains tool sends network traffic to localhost:9050. If your SOCKS proxy is listening on a different port, adjust /usr/local/etc/proxychains.conf configuration file (MacOS) accordingly.

Now edit the ecflow_ui launch script. Towards the end of the script, replace:

     "$exe"

with

     proxychains4 "$exe"

If you are using ecflow_ui version >= 5.7.0, you can start the ui with this command:

ecflow_ui -cmd proxychains4

However, if you are using an older version  you need to edit the ecflow_ui launch script and replace the "$exe" with proxychains4 "$exe" at the end of the script. On my machine, On my machine, the ecflow_ui launch script is installed as /opt/miniconda3/bin/ecflow_ui. On Linux, you may need to write "proxychains" instead of "proxychains4".

You should now be able to start ecflow_ui and interact with ecFlow servers and suites as if you were suites running at the Centre.

NOTE: If you were using Local Port Forwarding method up to now, you will also need to:

...

By default, proxychains tool sends network traffic to localhost:9050. If your SOCKS proxy is listening on a different port, adjust /usr/local/etc/proxychains.conf configuration file (MacOS) accordingly.

Accessing restricted ecFlow servers

...

• you can ask the ecFlow server administrator to add your laptop username to the server's whitelist, or
• you can create a new user account on your laptop, setting username to match the ECMWF username.

If you create a new user account for ecflow_ui but want to keep using your regular account, here is a handy script (this example for running ecflow_ui as another user. It uses Dynamic Port Forwarding method):, but can be easily adapted to use Local Port Forwarding.

#!/bin/bash
set -e
# --------------------------------------------------------------
# A script for starting SSH SOCKS proxy and ecFlow UI as another user.
# Prerequisites:
#   * User named "myecuser" must exist on the laptop
#   * "myecuser" has set up their Teleport client to access ECMWF.
# --------------------------------------------------------------

ECMWF_USER=myecuser              # your ECMWF username
ECMWF_HOST=myecworkstation       # your ECMWF workstation name

xhost + || :
sudo -i -u "$ECMWF_USER" -- sh << SUDO

    set -e

    # Starting ssh-agent for $ECMWF_USER.
    # Teleport service requires ssh-agent running.

    ssh-agent -- sh << SSH_AGENT

        set -e

        # "tsh login" will fetch SSH certificate from
        # shelljump.ecmwf.int and load it onto the ssh-agent.

        tsh login -d

        # "ssh -f" will start SSH tunnel in the background.
        # The trap will terminate the tunnel on exit.

        trap 'pkill -f ssh.*-f' 0 1 2 3 15
        ssh -f -N -v -C -D9050 -J "$ECMWF_USER@shellUSER@jump.ecmwf.int" "$ECMWF_USER@$ECMWF_HOST"

        # finally, start the local ecFlow UI
        /Volumes/Macintosh\ HD/opt/miniconda3/bin/ecflow_ui
SUDO
SSH_AGENT
SUDO

Servers with password-based access control

...

The password-based access control must be enabled on the ecFlow server side first. Discuss with your server administrator to set it up. 

...

Replace "host1" with a real ECMWF ecFlow server hostname and "myecflowuser" with your authorised ecFlow username.

Known issues

...

, for which you have the password.

...