Tip | ||
---|---|---|
| ||
Go to to https://jupyterhub.ecmwf.int and log in with your ECMWF credentials. You You will require a second factor authentication Using Time-based One-Time Passwords |
ECMWF offers the possibility of using a JupyterLab environment to authorised users. The options available to you will depend on the privileges granted to your user account.
Table of Contents
JupyterLab on HPCF or ECS
Tip | ||
---|---|---|
| ||
You may run a JupyterLab instance on HPCF or ECS if you have access to those services. |
JupyterHub offers a novel way to access HPCF or ECS services, connecting you to an interactive JupyterLab session where you can run your Jupyter notebooks, manage your files, and much more.
JupyterLab would effectively run within a job on the Atos HPCF or ECS with dedicated resources on an allocated node for the job, and would allow you to conveniently interact with it from your browser without the need of any other connection to the HPCF using other methods.
Note | ||
---|---|---|
| ||
All JupyterLab sessions running on this service are time limited. When the time is up, the instance will be killed automatically along with any active processing that may be taking place. You can only have one JupyterLab instance running. If you left one running, JupyterHub will connect you straight back into it. |
First use: enabling access to your account from JupyterHub
Tip |
---|
This is only needed once |
Before you can start using this service, you need to enable the access from JupyterHub into your account on the target platform.
A unique SSH key pair is created for this purpose, and you will need to append the generated public SSH key into your ~/.ssh/authorized_keys
in your ECMWF HOME space.
You will need to access Atos HPCF, ECS or VDI services via other means. From a terminal in one of those systems, you can then:
No Format |
---|
$ grep -q $USER@jupyter ~/.ssh/authorized_keys || echo "ssh-ed25519 AAAA...." >> ~/.ssh/authorized_keys |
Note you should replace "ssh-ed25519 AAAA...."
by the actual value of your SSH key that appears in the JupyterHub portal.
Configuring your JupyterLab session
Select the desired profile in the profile dropdown:
Configure the resources and the time allocated for your session:
Configure the project account where your session will be accounted for:
...
title | Debug output |
---|
By default, the session standard output will not be saved. If you run into problems starting JupyterLab, or something does not work as you would expect once within JupyterLab, you may enable the debug option to generate the job output in your HOME directory, which will be directed to a file called userid-jupyterhub-jobid.out.
JupyterHub allows accessing ECMWF resources in an interactive and modern way. With JupyterHub, authorised users can spin up Jupyter sessions on multiple backends including High Performance Computing Facility (HPCF) and the ECGATE Service (ECS), leveraging the computational capacity and data resources available at ECMWF without leaving their browser.
Note | ||
---|---|---|
| ||
JupyterHub is currently available to all registered users with access to ECMWF HPCF and ECS. In order to use those services, non-registered users need to apply for access. People who work for a national meteorological service or scientists/researchers who work for a research organisation within one of our Member States or Co-operating States may follow the standard guidelines to access our computing facilities. |
Children Display | ||
---|---|---|
|
When you are ready, click Start
You may need to wait between 30 seconds and several minutes for the system to schedule your new job and connect to your new JupyterLab session
After a few minutes, your session will be up and running. Take note of the finishing time at the top right corner of the page. Your session will be terminated automatically once that time is reached.
Stopping your JupyterLab
Note |
---|
Closing your browser will not stop JupyterLab. It will continue to run in the background until it is actively stopped or it reaches its time limit. |
To stop your JupyterLab session, you may go to the top bar menu, and then click on File - Hub Control Panel
This will bring you back to your JupyterHub Home, from which you will be able to Stop your server (red button) or connect back to it (blue button)
JupyterHub and ecinteractive Jupyter sessions
Since it is also possible to start Jupyter sessions with ecinteractive, you should avoid running two JupyterLabs at the same time with each tool. They run on the same platform using the same HOME filesystem, so you may run into unexpected behaviour.
Accessing other filesystems beyond your HOME
Your Jupyter instance will always start with HOME as your top level directory. In order to access any other of your storage spaces in the Atos HPCF or ECS, you may create soft links to those spaces from your HOME directory. You can do that from a terminal:
No Format |
---|
for space in perm hpcperm scratch; do ln -snf /$space/$USER ~/$space; done |
You may also add those and other frequently accessed places to your favourites:
Leverage the ECMWF software stack: using modules
Note | ||
---|---|---|
| ||
Make sure you load the necessary modules BEFORE starting your kernels so they have the right environment. Loading modules after will not have any effect on existing kernels. |
You can leverage the existing software stack on the systems with modules with the Lmod extension in Jupyterlab:
...