Page History

...

You should avoid using conda through the Anaconda or Miniconda distributions, as you may run into licensing issues. Access to Anaconda’s public repository of packages is only free to individuals and small organisations (<200 employees). A paid license is required for larger organisations and anyone embedding or mirroring Anaconda’s repository. See Anaconda's Terms of Service for details.

The use of Miniforge is recommended instead, which provides the minimal installers for Conda and Mamba specific to conda-forge, with the following features pre-configured:

...

• Use tykky, which uses Miniforge internally to create a containerised environment. It may give you the best performance results. 
• Use provided conda module, which is based on Miniforge. Resulting environments on shared filesystems such PERM may suffer from slow performance.
• Install Miniforge on your own space (discouraged in favour of above alternatives, to be used only if those solutions cannot satisfy your requirements).

If working on personal laptops of or any other ECMWF computing platform, including the European Weather Cloud:

• Install Miniforge , if not already installed and migrate your environments and remove the previous anaconda/miniconda installation.

Check below for instructions on what to do to detect and migrate existing environments to alternative solutions.

How to check if any of my installations or environments are

...

using Anaconda-licensed packages

• On ECMWF's Atos HPCF or ECS:
  

  No Format
  /usr/local/apps/conda/conda_licensed_envs

• On any other platform: < add here link to a downloadable copy of UNIX-based platform (Linux, Mac or Windows Subsystem for Linux - WSL) platform, you can download conda_licensed_envs > and run it in your terminal.
• On Windows, you can download conda_licensed_envs.ps1 and run it in your Windows Terminal or PowerShell.In order for it to work, these are the requirements (Any Linux or Mac platform should work out of the box):
• bash
• cat
grep

Checking manually

• if using your own installation, check LICENSE or LICENSE.txt file at the root of the installation. If they contain the Anaconda Terms of Service, then you are using Anaconda's distribution and are subject to those Terms of Service.
• For existing environments run:
  

  No Format
  conda list --explicit -n <environment_name>

  If any of the packages come from https://repo.anaconda.com/pkgs then the environment is using Anaconda's own channels.
• If  "defaults" is in the list of channels of the environment or in your ~/.condarc condarc, then packages subject to Anaconda's Terms of Service might be used.

How to migrate anaconda/miniconda environments

...

• On ECMWF's Atos HPCF or ECS:
  

  No Format
  /usr/local/apps/conda/conda_export my_env > my_env.yml

• On any other platform: < add here link to a downloadable copy of conda_export >, you can download conda_exporter and run it in your terminal.
  • In order for it to work, these are the requirements:
    • Python 3.6 or above
    • Pyyaml python package, which can usually be installed with pip install pyyaml if not available.
    • a working conda installation.

Create a new containerised environment with Tykky on Atos HPCF or ECS

See HPC2020: Containerised Standard conda environments may perform poorly when installed on shared filesystems like the ones on the HPCF, and may create significant pressure on the filesystem infrastructure as a whole, and ultimately causing an overall service degradation. For that reason, the use of containerised software installations with Tykky is recommended.

In a nutshell, once you have the environment description in a YAML file,  you can create an environment and use it like this:

ml tykky
conda-containerize new --mamba --prefix=$TYKKY_PATH/my_env my_env.yml
tykky activate my_env

Create a new conda environment with Miniforge

...

on Atos HPCF or ECS

If the above method does not work for you on Atos HPCF or ECS you may use the provided conda module which is based on Miniforge. See See HPC2020: Conda at ECMWF

ml conda
conda env create -f my_env.yml -n myenv
conda activate my_env

ml conda
mamba env create -f my_env.yml -n myenv
mamba activate my_env

Create a new conda environment with Miniforge on any other platform

You may install Miniforge on your own platforms following the instructions in Getting started with Conda. 

Once installed and initialised, you just need to create the new environment and activate it

conda env create -f my_env.yml -n myenv
conda activate my_env

mamba env create -f my_env.yml -n myenv
mamba activate my_env