We have made available in Github a series of templates for automating a number of aspects of provisioning and configuring your instances to tailor them to your needs.

Ansible templates

Ansible is a very popular automation tool that can be used to configure and customise your instances and other operations. We have published a number of Ansible Playbooks and Roles to carry out specific operations and customisations on the EWC that you can mix and match.

Ansible playbook repositories

  • ewc-flavours: A set of flavours and instance types such as the ECMWF Data flavour or the local Jupyterhub. Although these are integrated in the Morpheus Portal, these playbooks allow you to apply them yourself to standard instances either manually or through your IaC pipelines.
  • ewc-ecmwf-ai-stacks: A collection of playbooks to for setting up ML/AI stacks. See EWC ECMWF AI Stacks for more details on how to use them.

Generic usage instructions

Each of those repositories will have specific instructions on how to use them, but here are the fundamental steps to get started if you want to use them directly on the command line or CI/CD pipeline:

You may run Ansible on the same instance or anywhere else from where you can connect to your instance via SSH, such as another instance in the same private network, or from your own computer if your instance can be reached over SSH. We will refer to this as your seed platform. You will need at least git and python available to follow these steps.

  1. On your seed platform, clone the repository and cd into the directory:
    git clone <repository-url>
cd <repository>
  2. If you don't have Ansible installed, you may install it with pip:
    pip install --user -r requirements.txt
    or if you prefer to do it in a virtual environment:
    python3 -m venv ansible-venv
source ansible-venv/bin/activate
pip install -r requirements.txt
  3. Install the necessary Ansible roles that are going to be used by the playbooks:
    ansible-galaxy role install -r requirements.yml roles/
  4. If you don't have it already, define your Ansible inventory. The simplest approach would be to create a file called inventory, on the same directory where the plabooks are, containing the fully qualified domain name (FQDN) or IP address used to connect to the instance from your seed platform. If running on the same instance, you may use localhost.
  5. Apply the desired playbook with ansible-playbook:
    ansible-playbook -i inventory playbookname.yml
    You may pass additional options to ansible-playbook, such as:
    • -v for verbose output
    • -K for asking sudo password, if your user does not have password-less sudo privileges on the target instance.
    • -u yourremoteuser if Ansible needs to use a specific user account to connect to the target instance.
    • -e var=value for add-hoc customisation of playbook variables to customise your installation.

Further customisation

Check each specific role documentation in the URLs found in requirements.yml to see all the variables you may customise when running the automation. For example, for the Anemoi role: https://github.com/ewcloud/ewc-ansible-role-anemoi

Ansible role repositories

These are the basic building blocks upon which the provided playbooks are built, so you can create your own Ansible playbooks combining them as you see fit. All these repositories are clearly named with the prefix "ewc-ansible-role-":

Advanced: How to use your custom Ansible playbook using provided Ansible roles

For advanced users with existing Ansible playbooks and greater customisation needs, it is possible to just pick and choose roles you are interested and include them into your own playbooks, instead of running separate playbooks. You can find the URLs for each of the dependant roles in the requirements.yml file. 

  1. Open your playbook file and include the roles that you are interested in. For example, here is a playbook that includes 3 roles and customises some variable:
    myplaybook.yml
    ---
- hosts: all
  become: yes
  vars:
    conda_prefix: /opt/conda
  tasks:
    - name: Mars client
      ansible.builtin.include_role:
        name: ewc-ansible-role-mars-client
 
    - name: ML basic stack
      ansible.builtin.include_role:
        name: ewc-ansible-role-ml-basic
 
    - name: Anemoi
      ansible.builtin.include_role:
        name: ewc-ansible-role-ecmwf-anemoi

    See the official Ansible documentation for more information on roles and how to include them. 
  2. Make sure the necessary roles are available before you run the playbook. For that you may install them with ansible-galaxy either individually or writing your own requirements.yml. For the example above, this could be a requirements.yaml file:
    requirements.yml
    ---
# Ansible Requirements
roles:
  - name: ewc-ansible-role-mars-client
    src: https://github.com/ewcloud/ewc-ansible-role-mars-client.git
    version: main

  - name: ewc-ansible-role-conda
    src: https://github.com/ewcloud/ewc-ansible-role-conda.git
    version: main

  - name: ewc-ansible-role-ml-basic
    src: https://github.com/ewcloud/ewc-ansible-role-ml-basic.git
    version: main

  - name: ewc-ansible-role-anemoi
    src: https://github.com/ewcloud/ewc-ansible-role-anemoi.git
    version: main

    See the official Ansible documentation for more details on how to create such file.
  3. Install the roles with ansible-galaxy:
    ansible-galaxy role install -r requirements.yml roles/
  4. Run the playbook
    $ ansible-playbook -i inventory myplaybook.yml

Further customisation

Check each specific role documentation in the URLs above to see all the variables you may customise when running the automation. 


  • No labels