This service is available as best effort in the pilot phase. Be aware when trying this service in your tenancy. In order to get access, get in touch with the admin users of EWC on EUMETSAT side, the pre-requisite for using this pilot service is to enroll in IAM for your tenancy. This service is required for the batch processing system.

EWC Pilot container registry is based on Harbor which is an open source registry that secures artifacts with policies and role-based access control, ensures images are scanned and free from vulnerabilities, and signs images as trusted. Harbor, a CNCF Graduated project, delivers compliance, performance, and interoperability to help you consistently and securely manage artifacts across cloud native compute platforms like Kubernetes and Docker.

URL Service: https://harbor-pilot.eumetsat.europeanweather.cloud/

Login credentials: EWC IAM credentials

EWC Container Registry has an IP allowlist ranges enabled for all operations. Currently for users it is possible only to push and pull from EWC machines.

Project

A project in Harbor contains all repositories of an application. Images cannot be pushed to Harbor before a project is created. Role-Based Access Control (RBAC) is applied to projects, so that only users with the appropriate roles can perform certain operations.

Every tenancy gets by default one private project! Only users who are members of the project can see the private project.

Login

There are two ways to login to the container registry to pull/push images:

CLI Secret → use for short lived docker work and manual docker login
Robot Account → use for longer lived work or in scripts to automate pull/push

CLI secret

Dashboard is currently accessible only through EWC.

Every user that wants to do simple manual docker login and short lived docker work can you your username and the CLI secret.

Go to User Profile
Take the CLI Secret and use it after running podman login or docker login, as shown in the section below Container Registry Login.

Robot account

Every user with admin credentials can create a robot account to run operations that can be automated (push/pull) in their tenancy projects. Robot Accounts cannot log in to the Container Registry interface.

Go to your Project → Use tab Robot Accounts → Click on + New Robot Account
Fill in Name and expiration time (days or never)
Give permissions to your robot account
Once you saved remember to store your secret somewhere, because you are not going to be able to retrieve it after

How to use Harbor

Pre-requisite

Using a container registry requires installation of a container platform:

podman
- https://podman.io/docs/installation
docker
- https://docs.docker.com/engine/install/

Containerize your application

Example from EWC Batch Processing service in Part 1: Containerize your project: https://gitlab.eumetsat.int/EWC/ewc-examples/-/tree/main/htcondor-container-job#part-1-containerize-your-project

Container registry login

You can login into your project on EWC container registry using your login credentials (CLI secret or robot account) and the following commands:

using podman

podman login https://harbor-pilot.eumetsat.europeanweather.cloud

using docker

docker login https://harbor-pilot.eumetsat.europeanweather.cloud

Push to your project

After login, using the following command you can push an image to the registry to the repository you want. The command will automatically create the repository. This step assumes that you already built your image and you have it locally (see example here: https://gitlab.eumetsat.int/EWC/ewc-examples/-/tree/main/htcondor-container-job#build-the-container)

using podman

podman push IMAGE_ID harbor-pilot.eumetsat.europeanweather.cloud/YOUR_PROJECT/REPOSITORY[:TAG]

where:

IMAGE_ID is the image ID you can find locally after building your image as shown below

[murdaca@fm-dev htcondor-container-job]# podman image ls
REPOSITORY                     TAG                 IMAGE ID       CREATED          SIZE
test-conda-env                 latest              29fe72839e1e   49 seconds ago   1.1GB

using docker

docker tag SOURCE_IMAGE[:TAG] harbor-pilot.eumetsat.europeanweather.cloud/YOUR_PROJECT/REPOSITORY[:TAG]

docker push harbor-pilot.eumetsat.europeanweather.cloud/YOUR_PROJECT/REPOSITORY[:TAG]

It’s a good practice to push with different tags, in order to not overwrite existing images.

where:

SOURCE_IMAGE is the name of the repository in your local machine
TAG is the tag of the image (you can choose a different one and it’s a good practice to do it)

[murdaca@fm-dev htcondor-container-job]# docker image ls
REPOSITORY                     TAG                 IMAGE ID       CREATED          SIZE
test-conda-env                 latest              29fe72839e1e   49 seconds ago   1.1GB

Container registry logout

using podman

podman logout https://harbor-pilot.eumetsat.europeanweather.cloud

using docker

docker logout https:/harbor-pilot.eumetsat.europeanweather.cloud

How to use Harbor in HTCondor Batch Processing

EWC Batch processing can pull images also from private registry from https://harbor-pilot.eumetsat.europeanweather.cloud, you don’t have to push your images to a public cloud if you can’t!

If you want to use the pilot registry into HTCBatchprocessing, you can follow the example here: https://gitlab.eumetsat.int/EWC/ewc-examples/-/blob/main/htcondor-container-job/dockerextcontreg.sub.