It is possible to connect local ecflow_ui running on your mac/pc to the remote servers running at the Centre using teleport and SSH Local Port Forwarding functionality.

First make sure you connected to the Centre with teleport using your token id. On the MAC i had to use latest teleport version. (5.0.0) 

brew install teleport # *OR*   > brew upgrade teleport

...

the Centre via the Centre's Teleport gateway.

You will need to install Teleport client on your laptop. Follow instructions on the Centre's User Documentation pages how to install and set up the client.

You also need a valid Teleport SSH certificate on your laptop. Certificates expire after 12 hours. Fresh certificate is obtained from the Centre's Teleport service with: 

% tsh login

You may be asked for your ECMWF login credentials. If you already have a valid certificate, the authentication prompt will be skipped.

We will now discuss two methods of connecting ecflow_ui via Teleport gateway with ecFlow servers running at the Centre.

Method 1: Local Port Forwarding

Once you have installed and set up Teleport client, you can start the SSH local port forwarding session. The session will tunnel network traffic between your ecflow_ui and the Centre's servers:

• ecflow_ui ↔ ECMWF ecFlow servers
• ecflow_ui ↔ ECMWF logservers (but see the "Limitations" section)

Using 'user1' login  name as an example, please replace with your own username used at ECMWF. Here I wish to connect to ECMWF workstation hostname machine1 where I have a ecflow server running on port 4141, also wish to connect to host machine2 where I have a ecflow server running on port 3142

...

ssh -J user1@shell.ecmwf.int user1@machine1 -C -N -L 4141:machine1:4141 -L 3142:machine2:3142

...

The ssh session will keep running in the terminal, listening on ports 4141 and 3142 and printing various log messages as it forwards your ecflow network traffic to the Centre.

To access HPC job output via the log server, you need to create a tunnel access for this as well. Here we use 'logserver' as an example

...

(replace this with your actual logserver

...

). You also need to add '127.0.0.1 logserver' alias in /etc/hosts so that connections to 'logserver' actually go to localhost

...

. Then use

ssh -J user1@shell.ecmwf.int user1@machine1 -C -N -L 4141:machine1:4141 -L 3142:machine2:3142 -L 9316:logserver:9316

...

Next, configure the connection settings for the two ecflow servers in your ecflow_ui. Make sure you use "localhost" in "Host" fields, not the actual ECMWF hostnames. 

Image Added

Limitations of Local Port Forwarding

• In practice we can only connect to a single remote log server. This is because different log servers at the Centre use the same port number (9316).
• The timeline and server load functionality currently does not work.

Method 2: Dynamic Port Forwarding

SSH offers Dynamic Port Forwarding, i.e. it can act as a SOCKS proxy. This method has some advantages over Local Port Forwarding:

• There is no need to manually specify port number mapping for each ecflow server
• You can use original host names in the ecFlow servers connection settigs, instead of 'localhost'
• You are not limited to a single logserver; there is also no need to modify /etc/hosts file.

However, to use Dynamic Port Forwarding, the client application must understand SOCKS protocol. Here we will show how to start SOCKS proxy and how to make ecflow_ui speak SOCKS.

SOCKS proxy will tunnel all network traffic from ecflow_ui to the servers running inside the Centre:

• ecflow_ui ↔ ECMWF ecFlow servers
• ecflow_ui ↔ ECMWF ecFlow logservers
• ecflow_ui ↔ ECMWF DNS servers

Start your SOCKS proxy in a terminal with:

% ssh -v -C -N -D9050 -J myecuser@shell.ecmwf.int myecuser@myecworkstation

(replace "myecuser" and "myecworkstation" with your real ECMWF username and workstation name).

The ssh session will keep running in the terminal, listening on port 9050 and printing various log messages as it forwards your network traffic to the Centre.

Next we need to prepare ecflow_ui to use the proxy. Applications which want to use SOCKS proxy must speak SOCKS protocol. Some application can be "SOCKS-ified" using a tool called proxychains. The tool intercepts the application's network traffic, adds SOCKS protocol layer and redirects traffic to the proxy. Luckily, the ecflow_ui.x executable can be SOCKS-ified this way.

On MacOS, you can install proxychains tool with "brew install proxychains-ng". Many Linux distributions come with proxychains tool preinstalled.

By default, proxychains tool sends network traffic to localhost:9050. If your SOCKS proxy is listening on a different port, adjust /usr/local/etc/proxychains.conf configuration file (MacOS) accordingly.

Now edit the ecflow_ui launch script. Towards the end of the script, replace:

     "$exe"

with

     proxychains4 "$exe"

On my machine, the ecflow_ui launch script is installed as /opt/miniconda3/bin/ecflow_ui. On Linux, you may need to write "proxychains" instead of "proxychains4".

You should now be able to start ecflow_ui and interact with ecFlow servers and suites as if you were at the Centre.

NOTE: If you were using Local Port Forwarding method up to now, you will also need to:

• remove "127.0.01 localhost" alias from /etc/hosts
• reconfigure connection settings in ecflow_ui - use actual ECMWF hostnames in the "Host", not "localhost".

Limitations of Dynamic Port Forwarding

• The timeline and server load functionality currently doesn't work

Connecting to restricted ecFlow servers

Servers with username-based access control

Many ecFlow servers at the Centre use a whitelist to only allow authorized users in. This creates a problem when connecting remotely. Typically, your username on the laptop will be different from your ECMWF username and these servers will not be visible in your ecflow_ui.

There are two possible ways to access these servers:

• you can ask the ecFlow server administrator to add your laptop username to the whitelist, or
• you can create a new user account on your laptop, setting username to match the ECMWF username.

If you create new user account for ecflow_ui but want to keep using your regular account, here is a handy script (this example uses Dynamic Port Forwarding method):

#!/bin/bash
set -e
# --------------------------------------------------------------
# A script for starting SSH SOCKS proxy and ecFlow UI as another user.
# Prerequisites:
#   * User named "myecuser" must exist on the laptop
#   * "myecuser" has set up their Teleport client to access ECMWF.
# --------------------------------------------------------------

ECMWF_USER=myecuser              # your ECMWF username
ECMWF_HOST=myecworkstation       # your ECMWF workstation name

xhost + || :
sudo -i -u "$ECMWF_USER" -- sh << SUDO

    set -e

    # Starting ssh-agent for $ECMWF_USER.
    # Teleport service requires ssh-agent running.

    ssh-agent -- sh << SSH_AGENT

        set -e

        # "tsh login" will fetch SSH certificate from
        # shell.ecmwf.int and load it onto the ssh-agent.

        tsh login -d

        # "ssh -f" will start SSH tunnel in the background.
        # The trap will terminate the tunnel on exit.

        trap 'pkill -f ssh.*-f' 0 1 2 3 15
        ssh -f -N -v -C -D9050 -J "$ECMWF_USER@shell.ecmwf.int" "$ECMWF_USER@$ECMWF_HOST"

        # finally, start the local ecFlow UI
        /Volumes/Macintosh\ HD/opt/miniconda3/bin/ecflow_ui
SUDO
SSH_AGENT

Servers with password-based access control

As an alternative to username-based access control, ecFlow offers password-based access control. See: Security(custom user).

The password-based access control must be enabled on the ecFlow server side first. Discuss with your server administrator to set it up. 

When the server-side setup is done by the administrator, configure the connection settings in your ecflow_ui: 

Image Added

Replace "myecworkstation" with a real ecFlow server hostname and "myecflowuser" with your authorised ecFlow username.

...

The next steps vary dependent on whether your username at ECMWF matches your local username on your MAC/PC and if ecflow server has a whitelist file. The whitelist file use to control access to the ecflow server based on the user name.

Remote ecflow Server with NO whitelist file:

Where there is NO whitelist file you are connecting using your local username on the MAC. This can be seen by viewing the log output file

Remote ecflow Server with whitelist file:

There are 3 options:

1/ Create a login name on your local machine which matches your user name at ECMWF.

2/ Ask your ecflow server administrator to add your local username to the ecflow server whitelist file  

3/ Ask your ecflow server administrator to setup a CUSTOM user, See: Security(custom user) this requires both client/server setup password based access. When this is done invoke ecflow_ui and use: 

...

Image Removed

Limitations

Currently we can only connect to one log server. Typically log servers use the same port name on different HPC. Hence this does pose a problem with port forwarding.

The timeline and server load functionality will not work.