...

At home, you don't have direct access to the Centre's ecFlow servers. However, you can use SSH to connect ecflow_ui running on your laptop with ecFlow servers at ECMWFset up such access using SSH. The basic idea is that the SSH server daemon running on your ECMWF workstation will "pretend" to be an ecflow_ui, forwarding network connections from your home ecflow_ui at home to the Centre's ecFlow servers. There are two ways of doing it. Both methods rely on the Port Forwarding functionality built into OpenSSH software suite. 

...

You will start SSH local port forwarding session on your laptop and connect your ecflow_ui to it. The local SSH session will forward network connections to the SSH server daemon running on your ECMWF workstation and from there to the Centre's ecFlow servers.

...

ssh -J user1@shelluser1@jump.ecmwf.int user1@machine1 -C -N -L 4141:machine1:4141 -L 3142:machine2:3142

...

To access HPC job output via the logserver, you need to establish a tunnel for this as well. Here we use 'logserver' as an example (replace this with an actual logserver).

ssh -J user1@shelluser1@jump.ecmwf.int user1@machine1 -C -N -L 4141:machine1:4141 -L 3142:machine2:3142 -L 9316:logserver:9316

...

Now you will connect your ecflow_ui to your local SSH port forwarding session. In ecflow_ui, edit ecFlow connection settings (Servers→ Manage Servers ...). Make sure to use "localhost" in the "Host" fields, not the actual ECMWF hostnames.

 Redirect logserver requests to localhost

In Section 2.1.1 we you have also established a tunnel for forwarding logserver traffic. In order to send the the log requests from ecflow_ui to the local tunnel entrance, you need to add "127.0.0.1 logserver" alias in /etc/hosts on your laptop (replace "logserver" with an actual ECMWF logserver name).

In practice, using the Local Port Forwarding method we can only connect to a single ECMWF logserver. This is because:

...

SSH also offers Dynamic Port Forwarding, a.k.a. SOCKS proxy functionality. This method has some advantages over Local Port Forwarding:

• There is no need to manually specify port number mapping for each ecflow server
• You can use original host names in the ecFlow servers connection settigssettings, instead of 'localhost'
• You are not limited to a single logserver; there is also no need to modify /etc/hosts file.

The SOCKS proxy can tunnel all network traffic from your ecflow_ui to the servers running inside the Centre:

• ecflow_ui ↔ ECMWF ecFlow servers
• ecflow_ui ↔ ECMWF ecFlow logservers
• ecflow_ui ↔ ECMWF DNS servers

However, to connect with SOCKS use the proxy, the client application (ecflow_ui) must be able to speak SOCKS protocol. We will show how to enable it for ecflow_ui.

NOTE: if you were using Local Port Forwarding method, you will now need to:

• remove Remove "127.0.0.1 logserver" from your /etc/hosts
• use Use actual ECMWF hostnames in the "Host" fields of ecflow_ui connection settings, instead of "localhost".

...

% ssh -v -C -N -D 9050 -J myecuser@shellmyecuser@jump.ecmwf.int myecuser@myecworkstation

...

Applications which want to use SOCKS proxy must speak SOCKS protocol. Some applications can be SOCKS-ified using a tool called proxychains. The tool intercepts the application's network traffic, adds SOCKS a protocol layer and redirects traffic to the proxy. Luckily, the ecflow_ui.x executable can be SOCKS-ified this way.

First you need to , install proxychains in your laptop. On MacOS, you can install do it with "brew install proxychains-ng". Some Linux distributions come with proxychains tool preinstalled.

If you are using ecflow_ui version >= 5.7.0, you can start the ui with this command:

ecflow_ui -cmd proxychains4

However, if you are using an older version  you need to Now comes the key part - edit the ecflow_ui lauch launch script and replace the "$exe" with proxychains4 "$exe" at the end of the script. On my machine, ecflow_ui launch script is installed as /opt/miniconda3/bin/ecflow_ui.

...

By default, proxychains tool sends network traffic to localhost:9050. If your SOCKS proxy is listening on a different port, adjust /usr/local/etc/proxychains.conf configuration file (MacOS) accordingly.

Accessing restricted ecFlow servers

Servers with username-based access control

Many ecFlow servers at the Centre use a whitelist to only allow authorized users in. This creates a problem when connecting remotely - typically your username on the laptop will be different from your authorized ECMWF username.

...

• you can ask the ecFlow server administrator to add your laptop username to the server's whitelist, or
• you can create a new user account on your laptop, setting username to match the ECMWF username.

If you create a new user account for ecflow_ui but want to keep using your regular account, here is a handy script for running ecflow_ui as another user. It uses Dynamic Port Forwarding method, but can be easily adapted to use Local Port Forwarding.

#!/bin/bash
set -e
# --------------------------------------------------------------
# A script for starting SSH SOCKS proxy and ecFlow UI as another user.
# Prerequisites:
#   * User named "myecuser" must exist on the laptop
#   * "myecuser" has set up their Teleport client to access ECMWF.
# --------------------------------------------------------------

ECMWF_USER=myecuser              # your ECMWF username
ECMWF_HOST=myecworkstation       # your ECMWF workstation name

xhost + || :
sudo -i -u "$ECMWF_USER" -- sh << SUDO

    set -e

    # Starting ssh-agent for $ECMWF_USER.
    # Teleport service requires ssh-agent running.

    ssh-agent -- sh << SSH_AGENT

        set -e

        # "tsh login" will fetch SSH certificate from
        # shelljump.ecmwf.int and load it onto the ssh-agent.

        tsh login -d

        # "ssh -f" will start SSH tunnel in the background.
        # The trap will terminate the tunnel on exit.

        trap 'pkill -f ssh.*-f' 0 1 2 3 15
        ssh -f -N -v -C -D9050 -J "$ECMWF_USER@shellUSER@jump.ecmwf.int" "$ECMWF_USER@$ECMWF_HOST"

        # finally, start the local ecFlow UI
        /Volumes/Macintosh\ HD/opt/miniconda3/bin/ecflow_ui
SSH_AGENT
SUDO

Servers with password-based access control

As an alternative to username-based access control, ecFlow offers password-based access control. See: Security(custom user).

...

Replace "host1" with a real ECMWF ecFlow server hostname and "myecflowuser" with your authorised ecFlow username, for which you have the password.

Known issues

...

...