With S3 buckets, accessing data is even easier than before. For those who want to use Python (encouraged), it is easy as pie.
If you're connecting to EUMETSAT buckets, take note of (1).
Make sure you have these ready:
...
Code Block | ||
---|---|---|
| ||
import os import io import boto3 #Initializing some values project_id = '123' #Fill this in bucketname = 'MyFancyBucket123' #Fill this in access_key = '123asdf' #Fill this in secret_access_key = '123asdf111' #Fill this in endpoint = 'https://my-s3-endpoint.com' #Fill this in |
(1) If you aren't connecting to EUMETSAT buckets, or a bucket someone shared with you, you can probably skip this next code block. Now, if the bucket is located at another project (e.g. someone is sharing their bucket with you), you probably need to denote the bucket name with project_id:bucketname. For boto3 to be able to use that kind of naming convention, we need to use a small trick.
Code Block | ||
---|---|---|
| ||
from botocore.session import Session
from botocore.handlers import validate_bucket_name
#This is a neat trick that allows us to specify our bucket name in terms of ProjectID:bucketname
bucketname = project_id + ':' + bucketname
botocore_session = Session()
botocore_session.unregister('before-parameter-build.s3', validate_bucket_name)
boto3.setup_default_session(botocore_session = botocore_session) |
Now on to the general tutorial; lets Lets start by initializing the S3 client with our access keys and endpoint:
...