...
Object storage (S3) can be configured to send notifications when objects are created , modified or listeddeleted. The notifications can be sent to
...
Topic can be created following:
| Code Block | ||
|---|---|---|
| ||
#!/usr/bin/python3
import boto3
access_key_id = 'xx' # stored in Cypher
secret_access_key = 'xx' # stored in Cypher
ceph_endpoint = 'https://s3.waw3-1.cloudferro.com'
region_name = 'waw3-1' # required by boto3, any value works
endpoint = "http://vm-name.tenancy-name.s/f.ewcloud.host" # your own http endpoint
topic_name = 'example-topic'
s3 = boto3.client('sns',
region_name=region_name,
endpoint_url= ceph_endpoint,
aws_access_key_id=access_key_id,
aws_secret_access_key=secret_access_key)
s3.create_topic(Name=topic_name, Attributes={'persistent': 'True', 'push-endpoint': endpoint}) |
...
| Code Block | ||
|---|---|---|
| ||
#!/usr/bin/python3
import boto3
access_key_id='xx' # from Cypher
secret_access_key='xx' # from Cypher
ceph_endpoint = "https://s3.waw3-1.cloudferro.com"
region = 'waw3-1' # required by boto3, any value works
bucket_name = "reflectivity.composite.opera.hdf5"
s3 = boto3.client('s3',
endpoint_url=ceph_endpoint ,
aws_access_key_id=access_key_id,
aws_secret_access_key=secret_access_key)
# Create notification configuration
response = s3.put_bucket_notification_configuration(
Bucket=bucket_name,
NotificationConfiguration={
'TopicConfigurations': [
{
'Id': 'test-http-topic',
'TopicArn': 'arn:aws:sns:waw3-1::example-topic',
'Events': ['s3:ObjectCreated:*']
}
]
}
)
|
TopicArn follows pattern arn:aws:sns:{region}::{topic-name}
Most typical event types are:
- s3:ObjectCreated:*
- s3:ObjectRemoved:*
Please consult S3 Bucket Notifications Compatibility — Ceph Documentation for details.
Note, that one can also add filters for objects, e.g.:
| Code Block |
|---|
#!/usr/bin/python3 import boto3 access_key_id='xx' # from Cypher secret_access_key='xx' # from Cypher ceph_endpoint = "https://s3.waw3-1.cloudferro.com" region = 'waw3-1' # required by boto3, any value works bucket_name = "reflectivity.composite.opera.hdf5" s3 = boto3.client('s3', endpoint_url=ceph_endpoint , aws_access_key_id=access_key_id, aws_secret_access_key=secret_access_key) # Create notification configuration response = s3.put_bucket_notification_configuration( Bucket=bucket_name, NotificationConfiguration={ 'TopicConfigurations': [ { 'Id': 'test-http-topic', 'TopicArn': 'arn:aws:sns:waw3-1::example-topic', 'Events': ['s3:ObjectCreated:*'], 'Filter': { 'Key': { 'FilterRules': [ { 'Name': 'suffix', 'Value': 'reflectivity-composite-opera.h5' } ] } } } ] } ) |
See Bucket Operations — Ceph Documentation for more detailed documentation.
List topics
Available topics can be listed following:
| Code Block | ||
|---|---|---|
| ||
#!/usr/bin/python3
import boto3
access_key_id = 'xx' # stored in Cypher
secret_access_key = 'xx' # stored in Cypher
ceph_endpoint = 'https://s3.waw3-1.cloudferro.com'
region_name = 'waw3-1' # required by boto3, any value works
s3 = boto3.client('sns',
region_name=region_name,
endpoint_url= ceph_endpoint,
aws_access_key_id=access_key_id,
aws_secret_access_key=secret_access_key)
response = s3.list_topics()
# Print the Topic ARNs
print('All topics:')
for topic in response['Topics']:
print(' -'+topic['TopicArn'])
|
Delete topics
Topics can be deleted following:
| Code Block | ||||
|---|---|---|---|---|
| ||||
#!/usr/bin/python3
import boto3
access_key_id='xx' # from Cypher
secret_access_key='xx' # from Cypher
ceph_endpoint = "https://s3.waw3-1.cloudferro.com"
region = 'waw3-1' # required by boto3, any value works
arn = 'arn:aws:sns:waw3-1::example-topic'
sns = boto3.client('sns',
region_name=region,
endpoint_url=ceph_endpoint ,
aws_access_key_id=access_key_id,
aws_secret_access_key=secret_access_key)
# Delete the SNS topic
response = sns.delete_topic(TopicArn=arn)
|