Publishing Events to a GCS Bucket¶
Purpose¶
This example shows how to publish events to a GCS bucket via the siddhi-io-gcs sink extension. The Siddhi application receives events by consuming them from the http://localhost:8006/inputStream
endpoint. These events are then sent to the GCS
sink that aggregates and commits the events to a Google Cloud Storage bucket.
Before you begin:
- Create an account in Google Cloud.
- Download the credential file that is generated through the GCP console and save it in a directory of your choice. For more information, see Google Cloud Authentication Documentation.
- Save the sample Siddhi application in Streaming Integrator Tooling.
Executing the Sample¶
To execute the sample, follow the steps below:
-
Enter values for the
credential.path
and thebucket.name
parameters of the GCS Sink definition in the sample Siddhi application.Info
You can either set the
credential.path
parameter in the sink configuration as shown below or can set a system variable withGOOGLE_APPLICATION_CREDENTIALS
as the name with the path to the credential file. -
Start the sample by clicking the Start button (shown below) or by clicking Run => Run.
If the Siddhi application starts successfully, the following message appears in the console.
GCSSinkSample.siddhi - Started Successfully!
Testing the Sample¶
To test the sample Siddhi application, open a terminal and issue the following curl command
curl -H "Content-Type: application/json" -d '{"event":{"key":"testFile","payload":"message", "test":"wjson"}}' http://localhost:8006/inputStream
Click here to view the sample Siddhi application.
@App:name("GCSSinkSample")
@App:description("Publish Events to a GCS bucket using a http-source")
@Source(type = 'http',
receiver.url='http://localhost:8006/inputStream', @map(type='json'))
define stream inputStream(key string, payload string, suffix string);
@sink(type='google-cloud-storage', credential.path='<credential.path>', bucket.name='<bucket.name>',
object.name='test-object-{{suffix}}',
@map(type='text'))
define stream outputStream(key string, payload string, suffix string);
from inputStream
select *
insert into outputStream;