Page History

Intro

With S3 buckets, accessing data is even easier than before. For those who want to use Python (encouraged), it is easy as pie.

Make sure you have both Python 3 and the access keys to your S3 bucket ready. Typically you'll find your access key and secret key in Morpheus under Tools -> Cypher. 

Pre-requisites

python3 -m pip install boto3

Access private buckets that require s3 credentials

Take a look at this code segment, which allows you to access the bucket, list its objects and upload/download files from it.

Before running any Python code, install the boto3 library:

...

Start by declaring some initial values for boto3 to know where your bucket is located at. Feel free to copy paste this segment and fill in with your own values.

...

https://towardsdatascience.com/introduction-to-pythons-boto3-c5ac2a86bb63

Access public buckets (no credentials required)

import os
import io
import boto3

from botocore import UNSIGNED
from botocore.config import Config

# Initializing some values
bucketname = 'seviri.meteosat-10.level-15.native'  #Fill this in
endpoint = 'https://s3.waw3-1.cloudferro.com'  #Fill this in

# Lets start by initializing the S3 client endpoint:
# Initialize the S3 client
s3 = boto3.client('s3', endpoint_url=endpoint,
        config=Config(signature_version=UNSIGNED))

# As a first step, and to confirm we have successfully connected, lets list the objects inside our bucket (up to a 1.000 objects).
# List the objects in our bucket
response = s3.list_objects(Bucket=bucketname)
for item in response['Contents']:
    print(item['Key'])

Related articles

...