Skip to content

Working with Amazon S3 Bucket Service

Ballerina is an open-source programming language that empowers developers to integrate their system easily with the support of connectors. In this guide, we are mainly focusing on connecting to the Amazon Simple Storage Service API to create, store, download, and use data with other services.

The wso2/amazons3 module allows you to perform the following operations.

  • Create Bucket
  • List Buckets
  • Delete Bucket
  • Create Object
  • List Objects
  • Get Object
  • Delete Object

This example explains how to use the S3 client to connect with the Amazon S3 instance and to create an Amazon S3 bucket, list buckets and delete buckets.

You can find other integrations modules from wso2-ballerina GitHub organization.

What you'll build

This application connects with the Amazon S3 API and creates a new bucket on Amazon S3 instance with the provided name, gets the available buckets and deletes the specified bucket.

working with Amazon S3 Object service

Before you begin

Get the code

Pull the module from Ballerina Central using the following command.

ballerina pull wso2/integration_with_amazon_s3_bucket

Alternately, you can download the ZIP file and extract the contents to get the code.

Download ZIP

Implementation

A Ballerina project is created for the integration use case explained above. Please follow the steps given below to create the project and modules. You can learn about the Ballerina project and modules in this guide.

1. Create a new project.

$ ballerina new working-with-bucket-service

2. Create a module.

$ ballerina add integration_with_amazon_s3_bucket

The project structure is created as indicated below.

working-with-bucket-service
    ├── Ballerina.toml
    └── src
        └── integration_with_amazon_s3_bucket
            ├── Module.md
            ├── main.bal
            ├── resources
            └── tests
                └── resources

3. Set up credentials for accessing Amazon S3

  • Visit Amazon S3 and create an Amazon S3 account.

  • Create a new access key, which includes a new secret access key. - To create a new secret access key for your root account, use the security credentials page. Expand the Access Keys section, and then click Create New Root Key.

  • To create a new secret access key for an IAM user, open the IAM console. Click Users in the Details pane, click the appropriate IAM user, and then click Create Access Key on the Security Credentials tab.

  • Download the newly created credentials, when prompted to do so in the key creation wizard.

4. Add project configurations file

Add the project configuration file by creating a ballerina.conf file under the root path of the project structure. This file should have following configurations. Add the obtained Amazon S3 configurations to the file.

ACCESS_KEY_ID="<Amazon S3 key ID>"
SECRET_ACCESS_KEY="<Amazon S3 secret key>"
REGION="<Amazon S3 region>"
BUCKET_NAME="<Amazon S3 bucket name>"
TRUST_STORE_PATH="<Truststore file location>"
TRUST_STORE_PASSWORD="<Truststore password>"

5. Write the integration

Open the project with VS Code. The integration implementation is written in the src/integration_with_amazon_s3_bucket/main.bal file.

import ballerina/config;
import ballerina/http;
import ballerina/log;

import wso2/amazons3;

// Constants for error codes and messages.
const string RESPOND_ERROR_MSG = "Error in responding to client.";
const string CLIENT_CREATION_ERROR_MSG = "Error while creating the AmazonS3 client.";
const string BUCKET_CREATION_ERROR_MSG = "Error while creating bucket on Amazon S3.";
const string BUCKETS_RETRIEVING_ERROR_MSG = "Error while listing buckets on Amazon S3";
const string PAYLOAD_CONVERTION_ERROR_MSG = "Error occured while converting bucket list to json";
const string BUCKET_DELETION_ERROR_MSG = "Error while deleting bucket from Amazon S3.";

// Create Amazons3 client configuration with the above accesskey and secretKey values.
amazons3:ClientConfiguration amazonS3Config = {
    accessKeyId: config:getAsString("ACCESS_KEY_ID"),
    secretAccessKey: config:getAsString("SECRET_ACCESS_KEY"),
    region: config:getAsString("REGION"),
    clientConfig: {
        http1Settings: {chunking: http:CHUNKING_NEVER}
    }
};

// Create AmazonS3 client with the above amazonS3Config.
amazons3:AmazonS3Client|amazons3:ConnectorError amazonS3Client = new(amazonS3Config);

@http:ServiceConfig {
    basePath: "/amazons3"
}
service amazonS3Service on new http:Listener(9091) {
    @http:ResourceConfig {
        methods: ["POST"],
        path: "/imageStore/{bucketName}"
    }
    // Function to create a new bucket.
    resource function createBucket(http:Caller caller, http:Request request, string bucketName) {
        // Assign amazonS3Client global variable to a local variable
        amazons3:AmazonS3Client|amazons3:ConnectorError s3Client = amazonS3Client;
        if (s3Client is amazons3:AmazonS3Client) {
            // Define new response.
            http:Response backendResponse = new();
            // Invoke createBucket remote function from amazonS3Client.
            amazons3:ConnectorError? response = s3Client->createBucket(<@untainted> bucketName);
            if (response is amazons3:ConnectorError) {
                // Send the error response.
                createAndSendErrorResponse(caller, <@untainted> <string>response.detail()?.message,
                                BUCKET_CREATION_ERROR_MSG);
            } else {
                // If there is no error, then bucket created successfully. Send the success response.
                string textPayload = bucketName + " created on Amazon S3.";
                backendResponse.setTextPayload(textPayload, contentType = "text/plain");
                respondAndHandleError(caller, backendResponse, RESPOND_ERROR_MSG);
            }
        } else {
            // Send the error response.
            createAndSendErrorResponse(caller, <string>s3Client.detail()?.message, 
                                       CLIENT_CREATION_ERROR_MSG);
        }
    }

    @http:ResourceConfig {
        methods: ["GET"],
        path: "/imageStore"
    }
    // Function to list buckets.
    resource function listBuckets(http:Caller caller, http:Request request) {
        // Assign amazonS3Client global variable to a local variable
        amazons3:AmazonS3Client|amazons3:ConnectorError s3Client = amazonS3Client;
        if (s3Client is amazons3:AmazonS3Client) {
            // Define new response.
            http:Response backendResponse = new();
            // Invoke listBuckets remote function from amazonS3Client.
            var response = s3Client->listBuckets();
            if (response is amazons3:ConnectorError) {
                // Send the error response.
                createAndSendErrorResponse(caller, <@untainted> <string>response.detail()?.message,
                                BUCKETS_RETRIEVING_ERROR_MSG);
            } else {
                // If there is no error, then bucket list retrieved successfully. Send the bucket list.
                var list = json.constructFrom(response);
                if (list is json) {
                    backendResponse.setJsonPayload(<@untainted> list, contentType = "application/json");
                    respondAndHandleError(caller, backendResponse, RESPOND_ERROR_MSG);
                } else {
                    createAndSendErrorResponse(caller, <@untainted> <string>list.detail()?.message,
                                               PAYLOAD_CONVERTION_ERROR_MSG);
                }
            }
        } else {
            // Send the error response.
            createAndSendErrorResponse(caller, <string>s3Client.detail()?.message, CLIENT_CREATION_ERROR_MSG);
        }
    }

    @http:ResourceConfig {
        methods: ["DELETE"],
        path: "/imageStore/{bucketName}"
    }

// Function to delete bucket.
    resource function deleteBucket(http:Caller caller, http:Request request, string bucketName) {
        // Assign amazonS3Client global variable to a local variable
        amazons3:AmazonS3Client|amazons3:ConnectorError s3Client = amazonS3Client;
        if (s3Client is amazons3:AmazonS3Client) {
            // Define new response.
            http:Response backendResponse = new();
            // Invoke deleteBucket remote function from amazonS3Client.
            amazons3:ConnectorError? response = s3Client->deleteBucket(<@untainted> bucketName);
            if (response is amazons3:ConnectorError) {
                // Send the error response.
                createAndSendErrorResponse(caller, <@untainted> <string>response.detail()?.message,
                                        BUCKET_DELETION_ERROR_MSG);
            } else {
                // If there is no error, then bucket deleted successfully. Send the success response.
                backendResponse.setTextPayload(<@untainted> string `${bucketName} deleted from Amazon S3.`,
                                        contentType = "text/plain");
                respondAndHandleError(caller, backendResponse, RESPOND_ERROR_MSG);
            }
        } else {
            // Send the error response.
            createAndSendErrorResponse(caller, <string>s3Client.detail()?.message, CLIENT_CREATION_ERROR_MSG);
        }
    }
}

// Function to create the error response.
function createAndSendErrorResponse(http:Caller caller, string errorMessage, string respondErrorMsg) {
    http:Response response = new;
    //Set 500 status code.
    response.statusCode = 500;
    //Set the error message to the error response payload.
    response.setPayload(<string> errorMessage);
    respondAndHandleError(caller, response, respondErrorMsg);
}

// Function to send the response back to the client and handle the error.
function respondAndHandleError(http:Caller caller, http:Response response, string respondErrorMsg) {
    // Send response to the caller.
    var respond = caller->respond(response);
    if (respond is error) {
        log:printError(respondErrorMsg, err = respond);
    }
}

Testing

First let’s build the module. Navigate to the project root directory and execute the following command.

$ ballerina build integration_with_amazon_s3_bucket

This creates the executables. Now run the integration_with_amazon_s3_bucket.jar file created in the above step.

$ java -jar target/bin/integration_with_amazon_s3_bucket.jar

You will see the following service log after successfully invoking the service.

[ballerina/http] started HTTP/WS listener 0.0.0.0:9091

1. Testing the create bucket service

  • Invoke the following curl request to create a new bucket.
    curl -v -X POST http://localhost:9091/amazons3/imageStore/firstbalbucket
    You see the response as follows after successfully creating the Amazon S3 bucket.
    firstbalbucket created on Amazon S3.

2. Testing the list bucket service

  • Invoke the following curl request to list buckets.
    curl -X GET http://localhost:9091/amazons3/imageStore
    
 {"name":"firstbalbucket", "creationDate":"2019-10-04T11:04:30.000Z"}

3. Testing delete Bucket service

  • Invoke the following curl request to delete the above bucket.
    curl -v -X DELETE http://localhost:9091/amazons3/imageStore/firstbalbucket
    You see the response as follows:
    firstbalbucket deleted from Amazon S3
Top