Skip to content

AmazonSQS Connector Example

Amazon Simple Queue Service (SQS) is a fully managed message queuing service that allows you to run business applications and services so that the messaging is not dependent on the IT infrastructure itself. This means the messages can run and fail independently of each other in a way that does not cause slowdowns, system-wide faults, or a disturbance within the application. By using Amazon SQS, you can move data between distributed components of your applications that perform different tasks without losing messages or requiring each component to be always available.

The WSO2 Amazon SQS connector allows you to access the exposed API through the WSO2 EI. Through this connector, you can perform CRUD operations for queues in Amazon SQS instance, update permissions and can work with messages. For further reference please refer to Amazon SQS API reference.

What you'll build

This example explains how to use Amazon SQS Connector to create a queue in the Amazon SQS, send a message to the queue, forward it to Simple Stock Quote Service Backend and send the response to the user. It has a single HTTP API resource, which is sendToQueue.

AmazonSQS-Connector

If you do not want to configure this yourself, you can simply get the project and run it.

Setting up the environment

  1. Please follow the steps mentioned in the Setting up the Amazon S3 Environment document in order to create a Amazon account and obtain access key id and secret access key. Keep them saved to be used in the next steps.

  2. In this example we will be using XPath 2.0 which needs to be enabled in the product as shown below before starting the WSO2 EI service.

    If you are using EI7, you need to enable this property by adding the following to the PRODUCT-HOME/conf/deployment.toml file. You can further refer to the Product Configurations.

      [mediation]
      synapse.enable_xpath_dom_failover="true"

    If you are using EI 6, you can enable this property by uncommenting synapse.xpath.dom.failover.enabled=true property in PRODUCT-HOME/conf/synapse.properties file.

  3. In this example we use SimpleStockQuote Service backend. Therefore SimpleStockQuote service needs to be started.

Configure the connector in WSO2 Integration Studio

Follow these steps to set up the Integration Project and the Connector Exporter Project.

  1. Open WSO2 Integration Studio and create an Integration Project. Creating a new Integration Project

  2. Right click on the project that you created and click on Add or Remove Connector -> Add Connector. You will get directed to the WSO2 Connector Store.

  3. Search for the specific connector and download it to the workspace. In this case, Salesforce is used as an example, but you can do this for the connector you require for your integration scenario.
    Search Connector in the Connector Store

  4. Click Finish, and your Integration Project is ready. The downloaded connector is displayed on the left side palette with its operations.

  5. You can drag and drop the operations to the design canvas and build your integration logic. Drag connector operations

  6. First let's create the following sequences, which are buildMessage, createQueue, sendMessage and ReceiveAndForwardMessage. Right click on the created Integration Project and select, -> New -> Sequence to create the Sequence. Adding a Sequence

  7. Provide the Sequence name as buildMessage. You can go to the source view of the XML configuration file of the API and copy the following configuration. In this sequence we are taking the user's input companyName and we build the message using a Payload Factory Mediator.

      <?xml version="1.0" encoding="UTF-8"?>
      <sequence name="buildMessage" trace="disable" xmlns="http://ws.apache.org/ns/synapse">
          <property expression="json-eval($.companyName)" name="companyName" scope="default" type="STRING"/>
          <payloadFactory media-type="xml">
              <format>
                  <m0:getQuote xmlns:m0="http://services.samples">
                      <m0:request>
                          <m0:symbol>$1</m0:symbol>
                      </m0:request>
                  </m0:getQuote>
              </format>
              <args>
                  <arg evaluator="xml" expression="get-property('companyName')"/>
              </args>
          </payloadFactory>
          <header name="Action" scope="default" value="urn:getQuote"/>
          <enrich>
              <source clone="true" type="body"/>
              <target property="target_property" type="property"/>
          </enrich>
      </sequence>

  8. Create the createQueue sequence as shown below. In this sequence, we create a queue in the Amazon SQS instance.

      <?xml version="1.0" encoding="UTF-8"?>
      <sequence name="createQueue" trace="disable" xmlns="http://ws.apache.org/ns/synapse">
          <amazonsqs.init>
              <accessKeyId>AKIAJRM3ROHOPXQ4V6QA</accessKeyId>
              <secretAccessKey>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</secretAccessKey>
              <version>2009-02-01</version>
              <region>us-east-2</region>
          </amazonsqs.init>
          <amazonsqs.createQueue>
              <queueName>{$ctx:queueName}</queueName>
          </amazonsqs.createQueue>
          <property expression="json-eval($.CreateQueueResponse.CreateQueueResult.QueueUrl)" name="queueURL" scope="default" type="STRING"/>
          <log level="custom">
              <property expression="$ctx:queueURL" name="queueURL"/>
          </log>
          <property expression="fn:substring($ctx:queueURL,39,12)" name="queueId" scope="default" type="STRING" xmlns:fn="http://www.w3.org/2005/xpath-functions"/>
          <log level="custom">
              <property expression="$ctx:queueId" name="queueId"/>
          </log>
      </sequence>

  9. Create sendMessage sequence as shown below. In this sequence, we send the message that we built in step 1 to the Amazon SQS Queue.

      <?xml version="1.0" encoding="UTF-8"?>
      <sequence name="sendMessage" trace="disable" xmlns="http://ws.apache.org/ns/synapse">
          <amazonsqs.init>
              <accessKeyId>AKIAJRM3ROJKJJXQ4V6QA</accessKeyId>
              <secretAccessKey>r7hfmtqVjdwieILi4mq7MJ2kkUZ2GlcCkBNg</secretAccessKey>
              <version>2009-02-01</version>
              <region>us-east-2</region>
          </amazonsqs.init>
          <amazonsqs.sendMessage>
              <queueId>{$ctx:queueId}</queueId>
              <queueName>{$ctx:queueName}</queueName>
              <messageBody>{$ctx:target_property}</messageBody>
          </amazonsqs.sendMessage>
      </sequence>

  10. Create the ReceiveAndForwardMessage sequence as shown below. In this sequence, we will receive the message from the Amazon SQS queue and forward it into the StockQuote Endpoint.

      <?xml version="1.0" encoding="UTF-8"?>
      <sequence name="ReceiveAndForwardMessage" trace="disable" xmlns="http://ws.apache.org/ns/synapse">
          <amazonsqs.init>
              <accessKeyId>AKIAJRM3ROJKJJXQ4V6QA</accessKeyId>
              <secretAccessKey>r7hfmtqVjdwieILi4mq7MJ2kkUZ2GlcCkBNg</secretAccessKey>
              <version>2009-02-01</version>
              <region>us-east-2</region>
          </amazonsqs.init>
          <amazonsqs.receiveMessage>
              <maxNumberOfMessages>5</maxNumberOfMessages>
              <queueId>{$ctx:queueId}</queueId>
              <queueName>{$ctx:queueName}</queueName>
          </amazonsqs.receiveMessage>
          <property expression="json-eval($.ReceiveMessageResponse.ReceiveMessageResult.Message.Body)" name="messageBody" scope="default" type="STRING"/>
          <payloadFactory media-type="xml">
              <format>
                  <soapenv:Envelope xmlns:soapenv="http://www.w3.org/2003/05/soap-envelope">
                      <soapenv:Body>$1</soapenv:Body>
                  </soapenv:Envelope>
              </format>
              <args>
                  <arg evaluator="xml" expression="$ctx:messageBody"/>
              </args>
          </payloadFactory>
          <header name="Action" scope="default" value="urn:getQuote"/>
          <call>
              <endpoint key="SimpleStockQuote"/>
          </call>
      </sequence>

  11. Now right click on the created Integration Project and select New -> Rest API to create the REST API.

  12. Provide the API name as SQSAPI and the API context as /sqs. You can go to the source view of the XML configuration file of the API and copy the following configuration.

      <?xml version="1.0" encoding="UTF-8"?>
      <api context="/sqs" name="SQSAPI" xmlns="http://ws.apache.org/ns/synapse">
          <resource methods="POST" uri-template="/sendToQueue">
              <inSequence>
                  <property expression="json-eval($.queueName)" name="queueName" scope="default" type="STRING"/>
                  <sequence key="buildMessage"/>
                  <sequence key="createQueue"/>
                  <sequence key="sendMessage"/>
                  <sequence key="ReceiveAndForwardMessage"/>
                  <respond/>
              </inSequence>
              <outSequence/>
              <faultSequence/>
          </resource>
      </api>

Exporting Integration Logic as a CApp

CApp (Carbon Application) is the deployable artifact on the Enterprise Integrator runtime. Let us see how we can export integration logic we developed into a CApp along with the connector.

Creating Connector Exporter Project

In order to bundle Connector into a CApp a Connector Exporter Project is needed.

  1. Navigate to File -> New -> Other -> WSO2 -> Extensions -> Project Types -> Connector Exporter Project.
    Add Connector Exporter Project

  2. Enter a name for the Connector Exporter Project.

  3. In the next screen select, Specify the parent from workspace and select the specific Integration Project you created from the dropdown. Naming Connector Exporter Project

  4. Now you need to add the Connector to Connector Exporter Project that you just created. Right click on the Connector Exporter Project and select, New -> Add Remove Connectors -> Add Connector -> Add from Workspace -> Connector

  5. Once you are directed to the workspace, it displays all the connectors that exist in the workspace. You can select the relevant connector and click Ok. Selecting Connector from Workspace

Creating a Composite Application Project

To export the Solution Project as a CApp, a Composite Application Project needs to be created. Usually, when a solution project is created, this project is automatically created by Integration Studio. If not, you can specifically create it by navigating to File -> New -> Other -> WSO2 -> Distribution -> Composite Application Project.

Exporting the Composite Application Project

  1. Right click on Composite Application Project and click on Export Composite Application Project.
    Export as a Carbon Application

  2. Select an Export Destination where you want to save the .car file.

  3. In the next Create a deployable CAR file screen, select both the created Integration Project and the Connector Exporter Project to save and click Finish. The CApp will get created at the specified location provided at the previous step. Create a deployable CAR file

Get the project

You can download the ZIP file and extract the contents to get the project code.

Download ZIP

Deployment

Follow these steps to deploy the exported CApp in the Enterprise Integrator Runtime.

Deploying on WSO2 Enterprise Integrator 7

You can copy the composite application to the /repository/deployment/server/carbonapps folder and start the server. Micro Integrator will be started and the composite application will be deployed. You can further refer to the application deployed through the CLI tool. You can download the CLI tool from here from the Other Resources section. Make sure you first export the PATH as below.

  $ export PATH=/path/to/mi/cli/directory/bin:$PATH
  1. Log in to Micro Integrator using the following command.

    ./mi remote login
  2. Provide default credentials admin for both username and password.

  3. In order to view the APIs deployed, execute the following command.

    ./mi api show
Click here for instructions on deploying on WSO2 Enterprise Integrator 6
  1. You can copy the composite application to the /repository/deployment/server/carbonapps folder and start the server.

  2. WSO2 EI server starts and you can login to the Management Console https://localhost:9443/carbon/ URL. Provide login credentials. The default credentials will be admin/admin.

  3. You can see that the API is deployed under the API section.

Testing

  1. Create a file called data.json with the following payload.
    {
      "companyName":"WSO2",
      "queueName":"Queue1"
    }
  2. Invoke the API as shown below using the curl command. Curl Application can be downloaded from [here] (https://curl.haxx.se/download.html).
    curl -H "Content-Type: application/json" --request POST --data @body.json http://localhost:8290/sqs/sendToQueue
    Expected Response: You should get the following response with the 'sys_id' and keep it saved.
        <ns:getQuoteResponse xmlns:ns="http://services.samples">
          <ns:return xmlns:ax21="http://services.samples/xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="ax21:GetQuoteResponse">
              <ax21:change>4.233604086603518</ax21:change>
              <ax21:earnings>-8.707965767387106</ax21:earnings>
              <ax21:high>-150.5908765590026</ax21:high>
              <ax21:last>153.98353327622493</ax21:last>
              <ax21:lastTradeTimestamp>Wed Apr 08 10:38:56 IST 2020</ax21:lastTradeTimestamp>
              <ax21:low>158.9975778178183</ax21:low>
              <ax21:marketCap>-565228.6001002677</ax21:marketCap>
              <ax21:name>WSO2 Company</ax21:name>
              <ax21:open>-151.38099715271312</ax21:open>
              <ax21:peRatio>23.761940918708092</ax21:peRatio>
              <ax21:percentageChange>-2.8310759126772127</ax21:percentageChange>
              <ax21:prevClose>-149.5404650806414</ax21:prevClose>
              <ax21:symbol>WSO2</ax21:symbol>
              <ax21:volume>9834</ax21:volume>
          </ns:return>
      </ns:getQuoteResponse>

What's Next

Top