Google Cloud Platform

For Google Cloud Platform, we will need the following information:

• Bucket Name (mandatory)
• Client Email (mandatory)
• clientId (mandatory)
• prefix (optional)
• privateKey (mandatory)
• privateKeyId (mandatory)
• projectId (mandatory)
• scheme = e.g. https

These can be found/extracted from the Google service account credentials file.

We need the following permissions: storage.objects.get, storage.objects.list, storage.objects.create, storage.objects.delete, storage.buckets.get.

Why do we need extra permissions?

• storage.objects.create: to upload new files.
• storage.objects.get: to read object metadata (e.g. size, checksum) after upload.
• storage.objects.delete: to remove or replace corrupted files and re-upload them.
• storage.objects.list: to check if an object already exists in the bucket.
• storage.buckets.get: to confirm that the target bucket exists and is reachable.

 Microsoft Azure Platform

We will need the following information:

• SAS (SharedAccessSignature) URL with access to service, container and objects. (The URL should contain parameter srt=sco).

• Blob container name

We need the following permissions: blob.upload(), blob.exists(), blob.delete(), blob.upload(), blob.download() and blob.getProperties().

Why do we need extra permissions?

• blob.upload(): to upload new files into the container.

• blob.exists(): to check if an object is already present.

• blob.delete(): to remove or replace corrupted files and re-upload them.

• blob.download() / blob.getProperties(): to read object metadata (e.g. size, checksum) after upload.

Amazon AWS Platform

We will need the following information:

• aws_access_key_id 
• aws_secret_access_key
• Bucket name

We need the following permissions: s3:DeleteObject, s3:PutObject, s3:GetObject, s3:ListBuckets.

For Amazon S3 transfers, ECMWF can also enable Amazon S3 Transfer Acceleration and Amazon S3 Dual-Stack Endpoints. Please let us know if you wish to use these features.

Why do we need extra permissions?

• s3:PutObject: to upload new files. Uploads are atomic; objects only appear once the full transfer has succeeded.

• s3:GetObject: to read object metadata (e.g. size, checksum) after upload.

• s3:DeleteObject: to remove or replace corrupted files and re-upload them.

• s3:ListBuckets: to check that the target bucket exists.

S3 Compatible Third-Party Storage Systems

We will need the following information:

• aws_access_key_id 
• aws_secret_access_key
• Bucket name

We need the following permissions: s3:DeleteObject, s3:PutObject, s3:GetObject, s3:ListBuckets.

We can configure S3-compatible third-party storage systems (for example, Google Cloud Storage via the S3 API or Cloudflare R2). Please ensure the provided endpoint and credentials follow the standard S3 format.

Why do we need extra permissions?

• s3:PutObject: to upload new files. Uploads are atomic; objects only appear once the full transfer has succeeded.

• s3:GetObject: to read object metadata (e.g. size, checksum) after upload.

• s3:DeleteObject: to remove or replace corrupted files and re-upload them.

• s3:ListBuckets: to check that the target bucket exists.

FTP/SFTP Connection

 We will need the following information:

• host name or IP
• protocol (ftp/sftp)
• username and password(or SSH-Key*) with permission to read-write
• receiving directory

*Alternatively, for SFTP, we support SSH key-based authentication. In this setup, we will provide our public SSH key, which should be added to the appropriate user's list of authorised keys on your server (typically in the .ssh/authorized_keys file within the user's home directory). Once configured, we will initiate connections using the corresponding private key.

Please note that for ftp we support the following ports:
21 (standard)
2121 (non standard)

For sftp we support:
22 (standard)
2222 (non standard)

Why do we need extra permissions?

• read: to verify uploaded files and check directory contents.
• write: to upload files.
• delete/rename: to replace corrupted files and to finalise transfers (optional: temporary .tmp files are renamed only after a successful upload).