Versions Compared

Old Version 5

changes.mady.by.user Francesco Murdaca

Saved on

compared with

New Version Current

changes.mady.by.user Roberto Cuccu

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents


Pre-requisites

Info

EWC EUMETSAT S3 endpoint URL is: https://s3.waw3-1.cloudferro.com

EWC ECMWF S3 endpoint URLs: 


Info

For private buckets you need the access keys to your S3 bucket ready. Typically you'll find your access key and secret key in Morpheus under Tools -> Cypher. 

CLI libraries

Some of the CLI libraries that can be used to access s3 buckets are described below:

s3cmd 

Warning

It works only for private buckets

Run the following command with your 'bucket_name' after configuring the remote to use (s3cmd --configure and follow the interactive configuration for a new remote)

Code Block
s3cmd ls s3://bucket_name/

...

Run the following command with your 'bucket_name' after configuring a remote s3 with the endpoint (rclone configure and follow the interactive configuration for a new remote)

Code Block
rclone ls s3://bucket_name/


  • Object storage: How to use rclone → Rclone is a command-line program to move/synchronise files from one storage location to another - for those familiar with rsync, rclone is rsync for cloud and more.  Additionally, Rclone mounts any local, cloud or virtual filesystem as a disk on Windows, macOS, linux and FreeBSD, and also serves these over SFTP, HTTP, WebDAV, FTP and DLNA. Rclone is mature, open-source software originally inspired by rsync and written in Go.
  • s3cmd → S3cmd is afree command line tool and client for uploading, retrieving and managing data in cloud storage service providers that use the S3 protocol. It is best suited for power users who are familiar with command line programs. It is also ideal for batch scripts and automated backup to S3, triggered from cron, etc.
  • Object storage: How to use awscli → command-line for s3 buckets.


You can find more details using the following pages:

Children Display

awscli

Run the following command after configuring the remote for a private bucket (aws configure  and follow the interactive configuration for a new remote):

Code Block
aws s3 ls --endpoint-url s3_endpoint_url s3://bucket_name/

Run the following command for a public bucket:

...