S3

Getting started⁠

S3 API is an Amazon S3 API based API for working with object storage resources. With S3 API you can:

View information about the number and volume of containers and objects within an account;
Create and delete containers;
upload, view, copy, move, download, and delete objects in containers;
manage segmented loading of objects;
manage CORS, etc.

API access⁠

Only authorized access to the S3 API is possible.

To access the container via the S3 API, the user must have a role with access to the object store, see the Manage Object Store Access instructions for details.

The address (URL) when using Path-Style addressing (default) can be viewed in the URL list.

If Virtual Hosted addressing is enabled, you can address the container at <container_name>.<s3_domain>.

Authentication⁠

Authentication in the S3 API is accomplished by signing requests. We support AWS Signature Version 4 and AWS Signature Version 2.

Signature generation will require access keys, which can be obtained by issuing an S3 key to the user:

Access Key ID — value of the Access key field from the S3 key;
Secret Access Key — value of the Secret key field from the S3 key.

You can sign the request using one of the methods:

via the Authorization HTTP header, see the Authenticating Requests: Using the Authorization Header (AWS Signature Version 4) in Amazon's documentation for more information;
using query parameters or a Presigned URL. With this method, you can specify all query parameters in the URL, see Authenticating Requests: Using Query Parameters (AWS Signature Version 4) in the Amazon documentation for more information.

The lifetime of a request signature is 15 minutes.

Compatibility⁠

The S3 API is compatible with the following Amazon S3 API features:

Method	Compatibility
Bucket CRUD.	✓
Bucket Acl	✗
Bucket CORS	✓
Bucket Encryption	✗ Client-side encryption supported (AWS SDK)
Bucket Lifecycle	✗
Bucket Location	✓
Bucket Logging	✗
Bucket Metrics Configuration	✗
Bucket Notification	✗
Bucket Ownership Controls	✗
Bucket Policy	✓
Bucket Replication	✗
Bucket Request Payment	✗
Bucket Tagging	✗
Bucket Versioning	✓
Bucket Website	✗
Object CRUD	✓
Object Copy	✓
Object Acl	✗ Get ACL works
Object Content	✗
Object Lock Configuration	✗
Object Response	✗
Object Retention	✗
Object Tagging	✗
Object Torrent	✗
Object Versions	✓
Multipart Upload	✓
Public Access Block	✗ All queries are Private by default

AWS SDK⁠

Python⁠

Boto⁠

boto3 — are development kits (SDKs) for Python 3.x programming languages. SDKs are designed to work with AWS services.

Install boto, more details in the boto3 documentation.
In your home directory, create a configuration file ~/.aws/credentials:

[default]
    aws_access_key_id = <access_key>
    aws_secret_access_key = <secret_key>

Specify:

<access_key> — field value Access key from S3 key;
<secret_key — field value secret key from the S3 key.

In your home directory, create a configuration file ~/.aws/config:

[default]
    region=<pool>

Examples of operations⁠

#!/usr/bin/env python
#-*- coding: utf-8 -*-
import boto3

# Authorization
s3 = boto3.client(
   service_name='s3',
   endpoint_url='<url>'
)

# Upload object from string
s3.put_object(Bucket="BucketName", Key="ObjectName1", Body="Test")

# Upload object from file
s3.upload_file("data.docx", "BucketName", "ObjectName2")

# Get list of objects in the container
for key in s3.list_objects(Bucket="BucketName")["Contents"]:
   print(key["Key"])

# Download object
get_object_response = s3.get_object(Bucket="BucketName", Key="ObjectName2")
print(get_object_response["Body"].read())

# Delete multiple objects
objects_to_delete = [{"Key": "ObjectName1"}, {"Key": "ObjectName2"}]
s3.delete_objects(Bucket="BucketName", Delete={"Objects": objects_to_delete})

Example for boto3⁠

Create an s3 client by providing an endpoint_url, a pool, and an S3 key (EC2 key) issued to the user:

s3 = boto3.client("s3", endpoint_url="<url>", region_name="<pool>", aws_access_key_id="access_key", aws_secret_access_key="secret_key")

Call the generate_presigned_post() method, passing the name of the bucket to which the upload will be performed, and a key name that can contain a wildcard ${filename} for the filename provided by the user at the time of upload. This call will return a dictionary with the URL to which the form should be submitted, and a fields dictionary with all the required populated fields for that form (X-Amz-Algorithm, X-Amz-Credential, X-Amz-Date, X-Amz-Signature, Policy).

post = s3.generate_presigned_post(bucket, key)

If additional fields need to be added to the object (and form), such as Content-Type header, pass a dictionary with these fields with the Fields argument. Also, according to the Policy specification, you need to describe each additional field in the conditions array (argument Conditions in boto).

post = s3.generate_presigned_post(bucket, key,
                                  Fields={"Content-Type": "image/webp"},
                                  Conditions=[["eq", "$content-type", "image/webp"]])

From the received data, you can compose an HTML form or reproduce the request via requests:

requests.post(post["url"], data=post["fields"], files=[("file", ("filename", b"body_data"))])

When generating an HTML form, it is worth considering that the file field with the data of the uploaded file should be at the end of the form.

PHP⁠

AWS SDK for PHP — is a development kit for working with AWS services.

The SDK is a modern open source PHP library that simplifies the integration of a PHP application with the S3 object store.

Follow these steps to connect the library using the Composer dependency management tool:

Install Composer:

curl -sS 'https://getcomposer.org/installer' | php

Run the Composer command to install the latest stable version of the SDK:

php composer.phar require aws/aws-sdk-php

Include aws-sdk-php in your script. Access Key and Secret Key values from S3-key are required for authorization.

Examples of operations⁠

<?php
require "vendor/autoload.php";

use Aws\S3\S3Client;
// Create client
$s3Client = new S3Client([
   "version" 	=> "latest",
   "region"  	=> "<pool>",
   "use_path_style_endpoint" => true,
   "credentials" => [
   	"key"	=> "<access_key>",
   	"secret" => "<secret_key>",
   ],
   "endpoint" => "<url>"
]);

// Upload object
$s3Client->putObject([
   "Bucket" => "BucketName",
   "Key"	=> "ObjectName",
   "Body"   => "Test"
]);

// Download object
$result = $s3Client->getObject([
   "Bucket" => "BucketName",
   "Key"	=> "ObjectName"
]);

echo $result["Body"];

Specify:

<access_key> — field value Access key from S3 key;
<secret_key — field value secret key from the S3 key.

JavaScript⁠

AWS SDK for Node.js — is a development kit for running JavaScript with AWS services in the Node.js environment.

Example of work for Node.js⁠

var S3 = require("aws-sdk/clients/s3");

var s3 = new S3({
    credentials: {
      accessKeyId: "<access_key>",
      secretAccessKey: "<secret_key>"
    },
    endpoint: "<url>",
    s3ForcePathStyle: true,
    region: "<pool>",
    apiVersion: "latest"
});

// Upload object

var params = {
    Bucket: "BucketName",
    Key: "ObjectName",
    Body: "Test"
};

s3.upload(params, (err, data) => {
    if (err) {
        console.log(err, err.stack);
    } else {
        console.log(data);
    }
    /*
    data = {
        ETag: "0cbc6611f5540bd0809a388dc95a615b",
        Location: "<url>/BucketName/ObjectName",
        key: "ObjectName",
        Key: "ObjectName",
        Bucket: "BucketName"
    }
    */

});

// Get object metadata

var params = {
    Bucket: "BucketName",
    Key: "ObjectName"
};

s3.headObject(params, (err, data) => {
    if (err) {
        console.log(err, err.stack);
    } else {
        console.log(data);
    }
    /*
    data = {
        AcceptRanges: "bytes",
        LastModified: 2019-12-03T17:29:15.000Z,
        ContentLength: 4,
        ETag: "0cbc6611f5540bd0809a388dc95a615b",
        ContentType: "application/octet-stream",
        Metadata: {}
    }
    */
});

// Get object

var params = {
    Bucket: "BucketName",
    Key: "ObjectName"
};

s3.getObject(params, (err, data) => {
    if (err) {
        console.log(err, err.stack);
    } else {
        console.log(data);
    }
    /*
    data = {
        AcceptRanges: "bytes",
        LastModified: 2019-12-03T17:29:15.000Z,
        ContentLength: 4,
        ETag: "0cbc6611f5540bd0809a388dc95a615b",
        ContentType: "application/octet-stream",
        Metadata: {},
        Body: <Buffer 54 65 73 74>
    }
    */
});


// Delete object

var params = {
    Bucket: "BucketName",
    Key: "ObjectName"
};

s3.deleteObject(params, (err, data) => {
    if (err) {
        console.log(err, err.stack);
    } else {
        console.log(data);
    }
    /*
    data = {
    }
    */
});

Specify:

<access_key> — field value Access key from S3 key;
<secret_key — field value secret key from the S3 key.

Java⁠

// Configure S3 client connection
AWSCredentials credentials = new BasicAWSCredentials(
        "<access_key>",
        "<secret_key>"
);

EndpointConfiguration endpoint =
        new EndpointConfiguration("<url>", "<pool>");

AmazonS3 s3client = AmazonS3ClientBuilder
        .standard()
        .withCredentials(new AWSStaticCredentialsProvider(credentials))
        .withPathStyleAccessEnabled(true)
        .withEndpointConfiguration(endpoint)
        .build();

// Create bucket
String bucketName = "s3bucket";
String objectName = "s3object";

if(!s3client.doesBucketExistV2(bucketName)) {
        s3client.createBucket(bucketName);
}

// Upload object
s3client.putObject(
        bucketName,
        objectName,
        "sample-data"
);

// Download object
S3Object s3object = s3client.getObject(bucketName, objectName);
S3ObjectInputStream inputStream = s3object.getObjectContent();

inputStream.transferTo(new FileOutputStream("downloaded-object"));

// Delete object
s3client.deleteObject(bucketName, objectName);

// Delete bucket
s3client.deleteBucket(bucketName);

Specify:

<access_key> — field value Access key from S3 key;
<secret_key — field value secret key from the S3 key.

Getting started⁠​

API access⁠​

Authentication⁠​

Compatibility⁠​

AWS SDK⁠​

Python⁠​

Boto⁠​

Examples of operations⁠​

Example for boto3⁠​

PHP⁠​

Examples of operations⁠​

JavaScript⁠​

Example of work for Node.js⁠​

Java⁠​

Getting started⁠

API access⁠

Authentication⁠

Compatibility⁠

AWS SDK⁠

Python⁠

Boto⁠

Examples of operations⁠

Example for boto3⁠

PHP⁠

Examples of operations⁠

JavaScript⁠

Example of work for Node.js⁠

Java⁠