gridscale Quick Start Guide: S3-Compatible Object Storage
Our S3-compatible Object Storage offering is developed with data sovereignty (EU GDPR) in mind, designed to be accessed from anywhere including websites, mobile applications or IoT devices. When it comes to using the Object Storage offering, there are a few things to consider which this post aims to cover.
Navigate to the Object Storage section within the Panel to get an overview of the offering.
The interface allows you to manage your buckets as well as the access keys for the same. It is possible to manage buckets directly via the interface, and several functionalities are available in the interface within the Bucket Settings, namely:
- Bucket Creation and Deletion
- Bucket Website: Allows you to host a static public website
- Bucket Versioning: An extra security layer for easy recovery from potential failures and user errors
- Bucket Lifecycle Configuration: Set bucket-level rules to automatically delete objects based on criteria that you set
However, for maximum flexibility and for larger buckets, we recommend using Object Storage tools such as Cyberduck (Mac, Windows) and S3cmd (Linux, Mac). In general, the Object Storage is compatible with all common S3 tools and applications that allow you to connect to S3. In the following sections we’ll introduce you to the two tools we mentioned earlier and show you how to connect to the gridscale Object Storage.
Along with your Access Key and Secret, the following endpoint parameters are required depending on which Object Storage location you’re referring to:
- de/fra2: gos3.io
- ch/app1: bc01.gos3.io
- nl/ams1: ce21.gos3.io
Cyberduck is a libre Server and Cloud Storage browser with an easy-to-use graphical user interface, available for Mac and Windows. Cyberduck is very suitable for easy management of your buckets.
Creating a new connection is quite simple. Open Cyberduck and click on “Open Connection”. In the window for the new connection select Amazon S3, add the relevant endpoint under Server and enter your Access Key as well as the Secret.
After the connection is established, you can easily browse through your buckets. You can upload/download files, synchronize paths, browse buckets and much more. Cyberduck offers many features and a good usability, feel free to explore the program a bit and you will quickly find your way around.
S3cmd is a free command line tool without a graphical user interface and available for Linux and Mac. s3cmd requires Python 2.6 or newer for installation, in version 2.x it is also compatible with Python 3.x.
S3cmd can be installed via RPM Package, PIP and Homebrew.
If you already have Homebrew installed, this is probably the fastest way to install it.
brew install s3cmd
Are you ready to start?
Or do you still have questions? Create your free account now, or get a personal consultation.
Once you’ve installed the S3cmd tool, you can configure it to access the Object Storage by running the following:
This creates a new default configuration. In case you wish to create multiple configurations, you can also write your specified configuration to a separate file.
s3cmd --configure --config=gos3cfg
The above command creates a config file named gos3cfg.
After running either command, s3cmd interactively prompts you for some setup parameters.
Taking the de/fra2 location as an example, you can specify the following parameters to make the connection to the Object Storage work:
Access Key: Your Key
Secret Key: Your Secret
Default Region: eu-central-1
S3 Endpoint: gos3.io
DNS-style bucket+hostname:port … : %(bucket)s.gos3.io
For the rest of the configurations, accept the default settings by hitting the Enter key. Finally, for the Save Settings prompt confirm by entering “y”.
If you wish to learn what else you can do with the s3cmd tool, run the following command to access the help manual.
The above command returns the following help manual:
Commands: Make bucket s3cmd mb s3://BUCKET Remove bucket s3cmd rb s3://BUCKET List objects or buckets s3cmd ls [s3://BUCKET[/PREFIX]] List all object in all buckets s3cmd la Put file into bucket s3cmd put FILE [FILE...] s3://BUCKET[/PREFIX] Get file from bucket s3cmd get s3://BUCKET/OBJECT LOCAL_FILE Delete file from bucket s3cmd del s3://BUCKET/OBJECT Delete file from bucket (alias for del) s3cmd rm s3://BUCKET/OBJECT Restore file from Glacier storage s3cmd restore s3://BUCKET/OBJECT Synchronize a directory tree to S3 (checks files freshness using size and md5 checksum, unless overridden by options, see below) s3cmd sync LOCAL_DIR s3://BUCKET[/PREFIX] or s3://BUCKET[/PREFIX] LOCAL_DIR Disk usage by buckets s3cmd du [s3://BUCKET[/PREFIX]] Get various information about Buckets or Files s3cmd info s3://BUCKET[/OBJECT] Copy object s3cmd cp s3://BUCKET1/OBJECT1 s3://BUCKET2[/OBJECT2] Modify object metadata s3cmd modify s3://BUCKET1/OBJECT Move object s3cmd mv s3://BUCKET1/OBJECT1 s3://BUCKET2[/OBJECT2] Modify Access control list for Bucket or Files s3cmd setacl s3://BUCKET[/OBJECT] Modify Bucket Policy s3cmd setpolicy FILE s3://BUCKET Delete Bucket Policy s3cmd delpolicy s3://BUCKET Modify Bucket CORS s3cmd setcors FILE s3://BUCKET Delete Bucket CORS s3cmd delcors s3://BUCKET Modify Bucket Requester Pays policy s3cmd payer s3://BUCKET Show multipart uploads s3cmd multipart s3://BUCKET [Id] Abort a multipart upload s3cmd abortmp s3://BUCKET/OBJECT Id List parts of a multipart upload s3cmd listmp s3://BUCKET/OBJECT Id Enable/disable bucket access logging s3cmd accesslog s3://BUCKET Sign arbitrary string using the secret key s3cmd sign STRING-TO-SIGN Sign an S3 URL to provide limited public access with expiry s3cmd signurl s3://BUCKET/OBJECT <expiry_epoch|+expiry_offset> Fix invalid file names in a bucket s3cmd fixbucket s3://BUCKET[/PREFIX] Create Website from bucket s3cmd ws-create s3://BUCKET Delete Website s3cmd ws-delete s3://BUCKET Info about Website s3cmd ws-info s3://BUCKET Set or delete expiration rule for the bucket s3cmd expire s3://BUCKET Upload a lifecycle policy for the bucket s3cmd setlifecycle FILE s3://BUCKET Get a lifecycle policy for the bucket s3cmd getlifecycle s3://BUCKET Remove a lifecycle policy for the bucket s3cmd dellifecycle s3://BUCKET List CloudFront distribution points s3cmd cflist Display CloudFront distribution point parameters s3cmd cfinfo [cf://DIST_ID] Create CloudFront distribution point s3cmd cfcreate s3://BUCKET Delete CloudFront distribution point s3cmd cfdelete cf://DIST_ID Change CloudFront distribution point parameters s3cmd cfmodify cf://DIST_ID Display CloudFront invalidation request(s) status s3cmd cfinvalinfo cf://DIST_ID[/INVAL_ID]
If you’re interested in reading more about Object Storage and all of our other offerings, do feel free to visit our Product Documentation.