Performance Analysis Results¶
Note
These performance statistics were taken when the load average was below 3.8 in the 4 core instance.
Consume events using Kafka source¶
Specifications of EC2 Instances¶
- Stream Processor : c5.xLarge
- Kafka server : c5.xLarge
- Kafka publisher : c5.xLarge
Siddhi Application¶
@App:name("HelloKafka")
@App:description('Consume events from a Kafka Topic and publish to a different Kafka Topic')
@source(type='kafka',
topic.list='kafka_topic',
partition.no.list='0',
threading.option='single.thread',
group.id="group",
bootstrap.servers='172.31.0.135:9092',
@map(type='json'))
define stream SweetProductionStream (name string, amount double);
@sink(type='log')
define stream KafkaSourceThroughputStream(count long);
from SweetProductionStream#window.timeBatch(5 sec)
select count(*)/5 as count
insert into KafkaSourceThroughputStream;
Results¶
- Average Publishing TPS to Kafka : 1.1M
- Average Consuming TPS from Kafka: 180K
Consuming messages from an HTTP Source¶
Specifications of EC2 Instances¶
- Stream Processor : c5.xLarge
- JMeter: c5.xLarge
Siddhi Application¶
@App:name("HttpSource")
@App:description('Consume events from http clients')
@source(type='http', worker.count='20', receiver.url='http://172.31.2.99:8081/service',
@map(type='json'))
define stream SweetProductionStream (name string, amount double);
@sink(type='log')
define stream HttpSourceThroughputStream(count long);
from SweetProductionStream#window.timeBatch(5 sec)
select count(*)/5 as count
insert into HttpSourceThroughputStream;
Results¶
- Average Publishing TPS to Http Source : 30K
- Average Consuming TPS from Http Source: 30K
Sending HTTP requests and consuming the responses¶
Specifications of EC2 Instances¶
- Stream Processor : c5.xLarge
- JMeter: c5.xLarge
- Web server : c5.xLarge
Siddhi Application¶
@App:name("HttpRequestResponse")
@App:description('Consume events from an HTTP source, ')
@source(type='http', worker.count='20', receiver.url='http://172.31.2.99:8081/service',
@map(type='json'))
define stream SweetProductionStream (name string, amount double);
@sink(type='http-request', l, sink.id='production-request', publisher.url='http://172.17.0.1:8688//netty_echo_server', @map(type='json'))
define stream HttpRequestStream (batchNumber double, lowTotal double);
@source(type='http-response' , sink.id='production-request', http.status.code='200',
@map(type='json'))
define stream HttpResponseStream(batchNumber double, lowTotal double);
@sink(type='log')
define stream FinalThroughputStream(count long);
@sink(type='log')
define stream InputThroughputStream(count long);
from SweetProductionStream
select 1D as batchNumber, 1200D as lowTotal
insert into HttpRequestStream;
from SweetProductionStream#window.timeBatch(5 sec)
select count(*)/5 as count
insert into InputThroughputStream;
from HttpResponseStream#window.timeBatch(5 sec)
select count(*)/5 as count
insert into FinalThroughputStream;
Results¶
- Average Publishing TPS to HTTP Source : 29K
- Average Publishing TPS from HTTP request sink: 29K
- Average Consuming TPS from HTTP response source: 29K