Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Knowledge bases for each of the new data services can be found through this landing page: https://eumetsatspace.atlassian.net/wiki/spaces/EUM/overview

These services are currently in their pilot phase, and are provided on a best-effort basis. Even now the services are open, it will still take time to fully populate them. As a workaround for this intermediate period, virtual machines on the EUMETSAT cloud are able to use object storage data buckets holding selected datasets.In addition, some selected datasets can be accessed from EWC VMs from the EWC object storage. Please contact us, if you need an additional data which is not provided in the Data Store. 

Option 1: Using EUMETSAT data services

...

  1. Make sure you have latest rclone installed

    Code Block
    curl https://rclone.org/install.sh | sudo bash


  2. Create mount directory, i.e. for ascat:

    Code Block
    sudo mkdir -p /eumetsatdata/ascat && sudo chown -R $USER:$USER /eumetsatdata/ascat && chmod 755 /eumetsatdata/ascat


  3. Put rclone like listed below config into a config file. For example, into /etc/eum_bucket_creds (keys without brackets) 

    Code Block
    sudo sh -c "cat >>/etc/eum_bucket_creds <<-EOF
    [eumetsat-ascat]
    type = s3
    provider = Ceph
    env_auth = false
    access_key_id={{ aws_access_key_id }}
    secret_access_key={{ aws_secret_access_key }}
    endpoint = https://s3.waw3-1.cloudferro.com
    EOF


  4. Now you can already mount the data with command:

    Code Block
    sudo rclone --config /etc/eum_bucket_creds mount --allow-other --daemon eumetsat-ascat:ascat.metop-abc.level-1b.native /eumetsatdata/ascat


...

  1. Link rclone

    Code Block
    sudo ln -s /usr/bin/rclone /sbin/mount.rclone


  2. Create systemd service:, for example for ascat:

    Code Block
    sudo sh -c "cat >>/etc/systemd/system/eum-data-ascat.service" <<-EOF
    [Unit]
    Description=Mount EUMETSAT data bucket ascat
    After=network-online.target
    
    [Mount]
    Type=rclone
    What=eumetsat-ascat:ascat.metop-abc.level-1b.native
    Where=/eumetsatdata/ascat
    Options=ro,allow_other,args2env,vfs-cache-mode=writes,config=/etc/eum_bucket_creds,cache-dir=/var/rclone,vfs-cache-max-size=512Mi,vfs-cache-poll-interval=30m'
    EOF


  3. Add corresponding line into /etc/fstab:

    Code Block
    echo "eumetsat-ascat:847632567fbf45408555318627075a26:ascat.metop-abc.level-1b.native /eumetsatdata/ascat rclone ro,noauto,nofail,x-systemd.automount,args2env,allow_other,vfs_cache_mode=writes,config=/etc/eum_bucket_creds,cache_dir=/var/cache/rclone,vfs-cache-max-size=512Mi,vfs-cache-poll-interval=30m 0 0" | sudo tee -a /etc/fstab


  4. Mount the folder:

    Code Block
    sudo mount /eumetsatdata/ascat


...