Introduction
S3 (Simple Storage Service) is a highly scalable, object-based cloud storage service used for storing and retrieving data.
There are different way of performing actions on files (e.g. upload, download, read, delete) or bucket specifically (e.g. create bucket) on s3 buckets:
- general packages (s3cmd, rclone, awscli): s3 from command line
- specific clients' packages (e.g. boto3 or s3fs Python libraries): s3 using Python libraries.
With S3 buckets, accessing data is even easier than before. For those who want to use Python (encouraged), it is easy as pie.
Make sure you have these ready:
- Python 3
- Project ID of the bucket you want to mount
- Access key and secret access key for the bucket
Take a look at this code segment, which allows you to access the bucket, list its objects and upload/download files from it. Feel free to copy paste this segment and fill in with your own values.
| Code Block | ||
|---|---|---|
| ||
import os
import boto3
from botocore.session import Session
from botocore.handlers import validate_bucket_name
#Initializing some values
project_id = 'myprojectID' #Fill in your own
bucketname = 'mybucket123' #Fill in your own
bucketname = project_id + ':' + bucketname
#This is a neat trick to allow us to specify our bucket name in terms of ProjectID:bucketname
botocore_session = Session()
botocore_session.unregister('before-parameter-build.s3', validate_bucket_name)
boto3.setup_default_session(botocore_session = botocore_session)
#Initialize the S3 client
s3 = boto3.client('s3', endpoint_url='https://s3.waw3-1.cloudferro.com',
aws_access_key_id = 'MyAccessKey',
aws_secret_access_key = 'MySecretAccessKey')
#List all the objects in the bucket
response = s3.list_objects(Bucket=bucketname)
for item in response['Contents']:
print(item['Key'])
#Downloading a file from the bucket
with open('myfile.txt', 'wb') as f:
s3.download_fileobj(bucketname, 'myfile.txt', f)
#Uploading a file to the bucket (make sure you have write access)
response = s3.upload_file('myfile.txt', bucketname, 'myfile.txt')
|
...
Related articles
| Content by Label | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
...