How to Set Up and Start the Synchronization Engine
Introduction
It is very easy to set up and start the SymmetricDS engine. The SetupSymmetricListener does it automatically, based on the contents of the symmetric-ds.properties file and the list of tables to be synchronized (see how to configure the synchronization of a table).
Note: for symmetric ds to work correctly it is important that the source.path property in the Openbravo.properties should be set correctly.
Install the module
First install the module by cloning it like this in the modules directory of your development project:
hg clone https://code.openbravo.com/erp/pmods/org.openbravo.replication.symmetricds
The module will be offered on the Openbravo forge in a later stadium.
Note: If you are on OB Commerce 16Q1 development then you need to apply a changeset to get the right dbsourcemanager. This is described in the README which is present in the replication symmetric ds module.
It most of the time makes sense to install an additional module which contains the sync definition (for syncing masterdata from main server to store server):
These modules have to be inserted both in the central and in the store servers.
Then rebuild the system using ant smartbuild -Dlocal=no.
Installing SymmetricDS in the Main Server
To install SymmetricDS in the main server, the install.symmetric.ds ant task needs to be invoked:
ant -f modules/org.openbravo.replication.symmetricds/build.xml install.symmetric.ds.
That task will:
- Create the SymmetricDS database objects
- Create triggers in the Openbravo tables that should be synchronized
- Create the symmetric-ds.properties file and place it in the config/ folder of the synchronization module. That file can be updated to customize the synchronization properties, if needed.
Once the task has been invoked, the symmetric-ds.properties file needs to be copied to the WEB-INF folder, for instance by running the ant smartbuild task.
Installing SymmetricDS in a Store Server
There is no need to install SymmetricDS manually in the store servers, apart from cloning the modules as explained in the previous section. When Tomcat is started in a store server, the synchronization properties file will be automatically created based on the configuration of the mobile servers, and those properties will be used to set up and start the synchronization engine. As part of the start up procedure of the synchronization engine, it will issue a registration request to the main server. Once the store server has been successfully registered, it will load all the non node specific synchronization properties from the main server.
Uninstalling SymmetricDS
The SymmetricDS database objects can be uninstalled at any moment by running the ant uninstall.symmetric.ds task in the org.openbravo.replication.symmetricds module. This process must be executed also in the store servers, even though the install.symmetric.ds task is not run in those servers.
Installing/Uninstalling the SymmetricDS triggers from the Openbravo tables
The install.symmetric.ds.triggers.in.ob.tables and uninstall.symmetric.ds.triggers.from.ob.tables ant tasks install and uninstall respectively from the database the triggers that were created by SymmetricDS in the Openbravo tables that should be synchronized.
The symmetric-ds.properties file
The symmetric-ds.properties file defines some properties about the server that will be used by Symmetric DS to configure the synchronization engine. These properties are:
- group.id: Defines the type of server, its possible values are MainServer and StoreServer
- external.id: The Mobile Server Key of the server.
- engine.name: The name of the server. It can be any string, as long as it is unique. It is used for logging purposes.
- db.driver: The driver that should be used to connect to the database
- db.url: The url that will be used to connect to the database
- db.user: The user that should be used to connect to the database
- db.password: The password that will be used to connect to the database
- auto.registration: If the auto.registration SymmetricDS parameter is set to true, then when a node attempts to register, the node will automatically be registered. If not, it will have to be manually pre-registered
- registration.url: The URL where this server will connect for registration and to receive its configuration. It will be empty for the main server. For the store server it will point to the context/org.openbravo.replication.symmetricds/sync URL of the main server (see example below).
- sync.url: The URL where this node can be contacted for synchronization. It will point to the context/org.openbravo.replication.symmetricds/sync URL of the server being configured.
- reload.servlet.user and reload.servlet.password: The table reloads are scheduled using a servlet in the Main Server. The store servers need the credentials of a user that can execute servlets in the main server. These two properties are only required in the properties file of the store servers.
This symmetric-ds.properties file is created automatically both in the main and in the store servers.
- In the main server it will be created when the install.symmetric.ds task is invoked, and will be placed in the config/ folder of the synchronization module. It can be updated to customize the synchronization properties manually.
- In the store servers it will be created in the java temporary folder. There is no need to update it manually, as all the properties that are not node-specific will be retrieved from the main server when the store servers requests to be registered.
Main Server Example, using an PostgreSQL database named hgvoloracle:
group.id=MainServer external.id=main engine.name=main db.driver=org.postgresql.Driver db.url=jdbc:postgresql://localhost:5435/pihgvolcs db.user=tad db.password=tad registration.url= sync.url=http://localhost:8080/openbravo/org.openbravo.replication.symmetricds/sync auto.registration=true auto.sync.triggers.at.startup=false job.routing.period.time.ms=10000 job.push.period.time.ms=10000 job.pull.period.time.ms=10000 node.offline.outgoing.dir=/tmp/$(nodeGroupId)-$(nodeId)/offline/outgoing node.offline.incoming.dir=/tmp/$(nodeGroupId)-$(nodeId)/offline/incoming
Store Server Example, using a PostgreSQL database named pihgvolss:
group.id=StoreServer external.id=store1 engine.name=store1 db.driver=org.postgresql.Driver db.url=jdbc:postgresql://localhost:5435/pihgvolss db.user=tad db.password=tad registration.url=http://localhost:8080/openbravo/org.openbravo.replication.symmetricds/sync sync.url=http://localhost:8070/openbravo/org.openbravo.replication.symmetricds/sync reload.servlet.user=Openbravo reload.servlet.password=openbravo job.routing.period.time.ms=10000 job.push.period.time.ms=10000 job.pull.period.time.ms=10000 node.offline.outgoing.dir=/tmp/$(nodeGroupId)-$(nodeId)/offline/outgoing node.offline.incoming.dir=/tmp/$(nodeGroupId)-$(nodeId)/offline/incoming
Giving registration access to a server
For a store server to be given registration access, the Can Be Registered flag of its entry in the Mobile Servers window must be activated. Until then, all its registration requests will be rejected.
How to configure the SymmetricDS log
SymmetricDS uses log4j for logging, so you can configure the Openbravo log4j.lcf file to configure its logging. This is an example on how to do it, storing the SymmetricDS log in the /tmp/logs/symmetric.log file:
log4j.category.org.jumpmind.symmetric=INFO, SYMMETRIC, C log4j.additivity.org.jumpmind.symmetric=false
log4j.appender.SYMMETRIC=org.apache.log4j.RollingFileAppender
log4j.appender.SYMMETRIC.File=/tmp/logs/symmetric.log
log4j.appender.SYMMETRIC.MaxFileSize=10000KB
log4j.appender.SYMMETRIC.layout=org.apache.log4j.PatternLayout
log4j.appender.SYMMETRIC.layout.ConversionPattern=%d{ISO8601} %p [%X{engineName}] [%c{1}] [%t] %m%n
log4j.logger.org.jumpmind.symmetric.util.PropertiesFactoryBean=ERROR, SYMMETRIC
We recommend setting the log leve of the org.jumpmind.symmetric.util.PropertiesFactoryBean class to ERROR, that class is is very verbose and the messages that it logs are not useful.
Setting the additivity to false will prevent the SymmetricDS log from being logged in the appenders defined in the root category. For instance, the console appender and the Openbravo file appender are defined in the root category. SymmetricDS log will be shown in the console because it has been explicitely selected when defining the category, but it will not be shown in the Openbravo file appender.
Troubleshooting
Error when restarting tomcat: The configured state does not match recorded database state
When you see a message like this in your console or Openbravo.log:
8504 [symmetric-engine-startup-0] ERROR org.jumpmind.symmetric.AbstractSymmetricEngine - The configured state does not match recorded database state. The recorded external id is 7EAAA333467244FE8527B7C57F082D4B while the configured external id is 2ABAF56A0FFD4764983814DE5A0C68F0. The recorded node group id is {} while the configured node group id is {}
This probably means that there is no symmetric-ds.properties file in the WEB-INF directory or it has invalid data.
FileNotFoundException in the log
When you see a message like this in the log:
Caused by: java.io.FileNotFoundException: http://main.openbravo.com:9080/openbravo/symmetric-ds/sync/registration?nodeGroupId=StoreServer&externalId=2ABAF56A0FFD4764983814DE5A0C68F0&syncURL=http%3A%2F%2Fstore1.openbravo.com%3A8080%2Fopenbravo%2Fsymmetric-ds%2Fsync&schemaVersion=%3F&databaseType=PostgreSQL&databaseVersion=9.3&symmetricVersion=development&hostName=oslo&ipAddress=192.168.1.14 at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1626)
This is caused by a wrong url in the symmetric-ds.properties file. So you should check the symmetric-ds.properties file.
Installing in Postgresql 9.1
If the install.symmetric.ds task is run against a Postgresql 9.1 database the following error will occur:
[java] 9624 [main] ERROR org.jumpmind.symmetric.db.postgresql.PostgreSqlSymmetricDialect - Please add "custom_variable_classes = 'symmetric'" to your postgresql.conf file
As the error messages describes, this problem is solved by adding this line to the postgresql.conf file and reloading postgresql:
custom_variable_classes = 'symmetric'