/ Tutorials / By

gridscale Quick Start Guide: S3-Compatible Object Storage

Our S3-compatible Object Storage offering is developed with data sovereignty (EU GDPR) in mind, designed to be accessed from anywhere including websites, mobile applications or IoT devices. When it comes to using the Object Storage offering, there are a few things to consider which this post aims to cover. 

Navigate to the Object Storage section within the Panel to get an overview of the offering. 

The interface allows you to manage your buckets as well as the access keys for the same. It is possible to manage buckets directly via the interface, and several functionalities are available in the interface within the Bucket Settings, namely:

  • Bucket Creation and Deletion
  • Bucket Website: Allows you to host a static public website
  • Bucket Versioning: An extra security layer for easy recovery from potential failures and user errors
  • Bucket Lifecycle Configuration: Set bucket-level rules to automatically delete objects based on criteria that you set 
Table Of Contents
  1. Cyberduck
  2. S3cmd

However, for maximum flexibility and for larger buckets, we recommend using Object Storage tools such as Cyberduck (Mac, Windows) and S3cmd (Linux, Mac). In general, the Object Storage is compatible with all common S3 tools and applications that allow you to connect to S3. In the following sections we’ll introduce you to the two tools we mentioned earlier and show you how to connect to the gridscale Object Storage.

Along with your Access Key and Secret, the following endpoint parameters are required depending on which Object Storage location you’re referring to:

  • de/fra2: gos3.io
  • ch/app1: bc01.gos3.io
  • nl/ams1: ce21.gos3.io

Cyberduck

Cyberduck is a libre Server and Cloud Storage browser with an easy-to-use graphical user interface, available for Mac and Windows. Cyberduck is very suitable for easy management of your buckets.

Creating a new connection is quite simple. Open Cyberduck and click on “Open Connection”. In the window for the new connection select Amazon S3, add the relevant endpoint under Server and enter your Access Key as well as the Secret.

Cyberduck UI Object Storage

Using Cyberduck

After the connection is established, you can easily browse through your buckets. You can upload/download files, synchronize paths, browse buckets and much more. Cyberduck offers many features and a good usability, feel free to explore the program a bit and you will quickly find your way around. 

S3cmd

S3cmd is a free command line tool without a graphical user interface and available for Linux and Mac. s3cmd requires Python 2.6 or newer for installation, in version 2.x it is also compatible with Python 3.x.

S3cmd can be installed via RPM Package, PIP and Homebrew.

If you already have Homebrew installed, this is probably the fastest way to install it.

brew install s3cmd

Bist du bereit zu starten?

Oder hast du noch Fragen? Lasse dir jetzt dein Konto erstellen oder dich in einem persönlichen Gespräch beraten.

Start now
Beraten lassen

Configure S3cmd

Once you’ve installed the S3cmd tool, you can configure it to access the Object Storage by running the following:

s3cmd --configure

This creates a new default configuration. In case you wish to create multiple configurations, you can also write your specified configuration to a separate file.

s3cmd --configure --config=gos3cfg

The above command creates a config file named gos3cfg.

After running either command, s3cmd interactively prompts you for some setup parameters.

Taking the de/fra2 location as an example, you can specify the following parameters to make the connection to the Object Storage work:

Access Key: Your Key
Secret Key: Your Secret
Default Region: eu-central-1
S3 Endpoint: gos3.io
DNS-style bucket+hostname:port … : %(bucket)s.gos3.io

s3cmd Object Storage

For the rest of the configurations, accept the default settings by hitting the Enter key. Finally, for the Save Settings prompt confirm by entering “y”.

If you wish to learn what else you can do with the s3cmd tool, run the following command to access the help manual.

s3cmd --help

The above command returns the following help manual:

Commands:
Make bucket
s3cmd mb s3://BUCKET
Remove bucket
s3cmd rb s3://BUCKET
List objects or buckets
s3cmd ls [s3://BUCKET[/PREFIX]]
List all object in all buckets
s3cmd la
Put file into bucket
s3cmd put FILE [FILE...] s3://BUCKET[/PREFIX]
Get file from bucket
s3cmd get s3://BUCKET/OBJECT LOCAL_FILE
Delete file from bucket
s3cmd del s3://BUCKET/OBJECT
Delete file from bucket (alias for del)
s3cmd rm s3://BUCKET/OBJECT
Restore file from Glacier storage
s3cmd restore s3://BUCKET/OBJECT
Synchronize a directory tree to S3 (checks files freshness using size and md5 checksum, unless overridden by options, see below)
s3cmd sync LOCAL_DIR s3://BUCKET[/PREFIX] or s3://BUCKET[/PREFIX] LOCAL_DIR
Disk usage by buckets
s3cmd du [s3://BUCKET[/PREFIX]]
Get various information about Buckets or Files
s3cmd info s3://BUCKET[/OBJECT]
Copy object
s3cmd cp s3://BUCKET1/OBJECT1 s3://BUCKET2[/OBJECT2]
Modify object metadata
s3cmd modify s3://BUCKET1/OBJECT
Move object
s3cmd mv s3://BUCKET1/OBJECT1 s3://BUCKET2[/OBJECT2]
Modify Access control list for Bucket or Files
s3cmd setacl s3://BUCKET[/OBJECT]
Modify Bucket Policy
s3cmd setpolicy FILE s3://BUCKET
Delete Bucket Policy
s3cmd delpolicy s3://BUCKET
Modify Bucket CORS
s3cmd setcors FILE s3://BUCKET
Delete Bucket CORS
s3cmd delcors s3://BUCKET
Modify Bucket Requester Pays policy
s3cmd payer s3://BUCKET
Show multipart uploads
s3cmd multipart s3://BUCKET [Id]
Abort a multipart upload
s3cmd abortmp s3://BUCKET/OBJECT Id
List parts of a multipart upload
s3cmd listmp s3://BUCKET/OBJECT Id
Enable/disable bucket access logging
s3cmd accesslog s3://BUCKET
Sign arbitrary string using the secret key
s3cmd sign STRING-TO-SIGN
Sign an S3 URL to provide limited public access with expiry
s3cmd signurl s3://BUCKET/OBJECT <expiry_epoch|+expiry_offset>
Fix invalid file names in a bucket
s3cmd fixbucket s3://BUCKET[/PREFIX]
Create Website from bucket
s3cmd ws-create s3://BUCKET
Delete Website
s3cmd ws-delete s3://BUCKET
Info about Website
s3cmd ws-info s3://BUCKET
Set or delete expiration rule for the bucket
s3cmd expire s3://BUCKET
Upload a lifecycle policy for the bucket
s3cmd setlifecycle FILE s3://BUCKET
Get a lifecycle policy for the bucket
s3cmd getlifecycle s3://BUCKET
Remove a lifecycle policy for the bucket
s3cmd dellifecycle s3://BUCKET
List CloudFront distribution points
s3cmd cflist
Display CloudFront distribution point parameters
s3cmd cfinfo [cf://DIST_ID]
Create CloudFront distribution point
s3cmd cfcreate s3://BUCKET
Delete CloudFront distribution point
s3cmd cfdelete cf://DIST_ID
Change CloudFront distribution point parameters
s3cmd cfmodify cf://DIST_ID
Display CloudFront invalidation request(s) status
s3cmd cfinvalinfo cf://DIST_ID[/INVAL_ID]

If you’re interested in reading more about Object Storage and all of our other offerings, do feel free to visit our Product Documentation.