Configuring Database and File System State Persistence

This section explains how to prevent the loss of data that can result from a system failure by persisting the state of WSO2 SI periodically either into a database system or into the file system.

Prerequisites

Before configuring RDBMS database persistence, the following prerequisites must be completed.

  • One or more Siddhi Applications must be running in the WSO2 SI server.
  • A working RDBMS instance that can be used for data persistence must exist.
  • The requirements of the datasource must be already defined.

  • Database persistence involves updating the databases connected to WSO2 Streaming Integrator with the latest information relating to the events that are being processed by WSO2 SI at a given time.

Configuring database system persistence

The supported databases are H2, MySQL, Postgres, MSSQL and Oracle. The relevant jdbc driver jar should be downloaded and added to the <SI_HOME>/lib directory to prior to using database system persistence.

To configure periodic data persistence, update the <SI_HOME>/conf/server/deployment.yaml file under state.persistence as follows:

Parameter Purpose Required Value
enabled This enables data persistence. true
intervalInMin The time interval in minutes that defines the interval in which state of Siddhi applications should be persisted 1
revisionsToKeep The number of revisions to keep in the system. When a new persist takes place, the old revisions are removed. 3
persistenceStore The persistence store . org.wso2.carbon.streaming.integrator.core.persistence.DBPersistenceStore
config > datasource The datasource to be used in persisting the state. The provided datasource should be properly defined in the deployment.yaml. For detailed instructions of how to configure a datasource, see Configuring Datasources .
WSO2_PERSISTENCE_DB (Datasource with this name should be defined in wso2.datasources)
config > table The table that should be created and used for the persisting of the state.
PERSISTENCE_TABLE

The following is a sample segment of the required configurations in the <SI_HOME>/conf/server/deployment.yaml file to configure file system persistence.

Sample deployment.yaml segment

    state.persistence:
      enabled: true
      intervalInMin: 1
      revisionsToKeep: 3
      persistenceStore: org.wso2.carbon.streaming.integrator.core.persistence.DBPersistenceStore
      config:
        datasource: <DATASOURCE NAME>   # A datasource with this name should be defined in wso2.datasources namespace
        table: <TABLE NAME>

Configuring file system persistence

This section explains how to persist the states of Siddhi applications during a required time interval in the file system in order to maintain back-ups. To configure state persistence, update the <SI_HOME>/conf/server/deployment.yaml file under state.persistence as follows:

Parameter Purpose Required Value
enabled This enables data persistence. true
intervalInMin The time interval in minutes that defines the interval in which state of Siddhi applications should be persisted
1
revisionsToKeep The number of revisions to keep in the system. When a new persist takes place, the old revisions are removed. 3
persistenceStore The persistence store.
org.wso2.carbon.streaming.integrator.core.persistence.FileSystemPersistenceStore
config > location A fully qualified folder location to where the revision files should be persisted.
siddhi-app-persistence

The following is a sample segment of the required configurations in the <SI_HOME>/conf/server/deployment.yaml file to configure file system persistence.

Sample deployment.yaml segment

    state.persistence:
      enabled: true
      intervalInMin: 1
      revisionsToKeep: 2
      persistenceStore: org.wso2.carbon.streaming.integrator.core.persistence.FileSystemPersistenceStore
      config:
        location: siddhi-app-persistence
Top