How to Set Up and Start the Synchronization Engine
Introduction
It is very easy to set up and start the SymmetricDS engine. The SetupSymmetricListener does it automatically, based on the contents of the symmetric-ds.properties file and the list of tables to be synchronized (see how to configure the synchronization of a table).
Note: for symmetric ds to work correctly it is important that the source.path property in the Openbravo.properties should be set correctly.
Install the module
First install the module by cloning it like this in the modules directory of your development project:
hg clone https://code.openbravo.com/erp/pmods/org.openbravo.replication.symmetricds
The module will be offered on the Openbravo forge in a later stadium.
Note: If you are on OB Commerce 16Q1 development then you need to apply a changeset to get the right dbsourcemanager. This is described in the README which is present in the replication symmetric ds module.
It most of the time makes sense to install an additional module which contains the sync definition (for syncing masterdata from main server to store server):
These modules have to be inserted both in the central and in the store servers.
Then rebuild the system using ant smartbuild -Dlocal=no.
Installing SymmetricDS in the Main Server
To install SymmetricDS in the main server, the install.symmetric.ds ant task needs to be invoked:
ant -f modules/org.openbravo.replication.symmetricds/build.xml install.symmetric.ds
That task will:
- Create the SymmetricDS database objects
- Create triggers in the Openbravo tables that should be synchronized
- Create the symmetric-ds.properties file and place it in the config/ folder of the synchronization module. That file can be updated to customize the synchronization properties, if needed.
Once the task has been invoked, the symmetric-ds.properties file needs to be copied to the WEB-INF folder, for instance by running the ant smartbuild task.
Installing SymmetricDS in a Store Server
There is no need to install SymmetricDS manually in the store servers, apart from cloning the modules as explained in the previous section. When Tomcat is started in a store server, the synchronization properties file will be automatically created based on the configuration of the mobile servers, and those properties will be used to set up and start the synchronization engine. As part of the start up procedure of the synchronization engine, it will issue a registration request to the main server. Once the store server has been successfully registered, it will load all the non node specific synchronization properties from the main server.
Uninstalling SymmetricDS
The SymmetricDS database objects can be uninstalled at any moment by running the ant uninstall.symmetric.ds task in the org.openbravo.replication.symmetricds module. This process must be executed also in the store servers, even though the install.symmetric.ds task is not run in those servers.
Updating an environment where SymmetricDS is installed
The following steps needs to be executed in order to update an environment where SymmetricDS is installed:
- update the environment as usual
- ant -f modules/org.openbravo.replication.symmetricds/build.xml update.symmetric.ds.configuration. This task will update the configuration of the synchronization, and will reinstall the triggers.
Giving registration access to a server
For a store server to be given registration access, the Can Be Registered flag of its entry in the Mobile Servers window must be activated. Until then, all its registration requests will be rejected.
How to give custom values to SymmetricDS parameters
The Default Synchronization Parameters window can be used to modify the default value of SymmetricDS's parameters. Check here the full list of configurable parameters. The contents of the Default Synchronization Parameters table are read only on Tomcat start, so it can be used to modify the value for both the Startup and Runtime parameters.
Note that, as that table is only read on Tomcat start, changes done to it will not take effect until Tomcat is restarted. To modify a SymmetricDS parameter live, the Synchronization Properties window must be used instead. The difference with the Default Synchronization Parameters window is that the changes done in the Synchronization Properties cannot be exported to modules, and will be reverted when Tomcat is restarted.
How to configure the SymmetricDS log
SymmetricDS uses log4j for logging, so you can configure the Openbravo log4j.lcf file to configure its logging. This is an example on how to do it, storing the SymmetricDS log in the ${catalina.base}/logs/symmetric.log file:
log4j.logger.org.jumpmind.symmetric=INFO, SYMMETRIC log4j.additivity.org.jumpmind.symmetric=false
log4j.appender.SYMMETRIC=org.apache.log4j.RollingFileAppender
log4j.appender.SYMMETRIC.File=${catalina.base}/logs/symmetric.log
log4j.appender.SYMMETRIC.MaxFileSize=10000KB
log4j.appender.SYMMETRIC.MaxBackupIndex=10
log4j.appender.SYMMETRIC.layout=org.apache.log4j.PatternLayout
log4j.appender.SYMMETRIC.layout.ConversionPattern=%d{ISO8601} %p [%X{engineName}] [%c{1}] [%t] %m%n
log4j.logger.org.jumpmind.symmetric.util.PropertiesFactoryBean=ERROR, SYMMETRIC
If you're using Openbravo PR19Q1 or newer, this is what you should add to store the SymmetricDS log on the log4j2-web.xml file:
<Loggers> ... <Logger name="org.jumpmind.symmetric" level="info" additivity="false"> <AppenderRef ref="Symmetric"/> </Logger> <Logger name="org.jumpmind.symmetric.util.PropertiesFactoryBean" level="error"/> ... </Loggers> <Appenders> ... <RollingFile name="Symmetric" fileName="${logDir}/symmetric.log" filePattern="${logDir}/symmetric-%i.log.gz"> <PatternLayout pattern="%d %p [%X{engineName}] [%c{1}] [%t] %m%n"/> <Policies> <SizeBasedTriggeringPolicy size="1000KB" /> </Policies> </RollingFile> ... </Appenders>
We recommend setting the log level of the org.jumpmind.symmetric.util.PropertiesFactoryBean class to ERROR, that class is is very verbose and the messages that it logs are not useful.
In a development environment, adding ', C' to log4j.category.org.jumpmind.symmetric will enable showing the SymmetricDS log in Eclipse's console.
Setting the additivity to false will prevent the SymmetricDS log from being logged in the appenders defined in the root category. For instance, the console appender and the Openbravo file appender are defined in the root category. SymmetricDS log will be shown in the console because it has been explicitely selected when defining the category, but it will not be shown in the Openbravo file appender.
Symmetric DS Clustering
Symmetric DS can be deployed in a clustered environment in combination with a clustered central server with multiple store servers. For background information, see this presentation.
To configure Symmetric DS for clustering, two properties must be provided to the SymmetricDS engine:
- cluster.lock.enabled: must be set to true to enable cluster support
- cluster.server.id: must have a different value for each cluster instance. It will be used to identify which cluster instance has the lock on each SymmetricDS Job
These properties will be automatically set if the hostname of the server (available from Java with System.getProperty("machine.name")) is defined. As described here, there are three ways to define the hostname:
- Starting from 3.0PR16Q3 it is possible to define the absolute path for this file by setting a JVM property named properties.path.
- Adding to the Tomcat's JVM a system property named machine.name which value is the hostName.
- If previous property is not set, it is taken from local host name. This name can be checked by executing ant host.name task in each node.
Note that if the cluster.lock.enabled and cluster.server.id properties are defined in a non-clustered environment, there will not be any negative repercusion and SymmetricDS will work as usual.
There is an additional configuration parameter for Symmetric DS clustering:
- cluster.lock.timeout.ms defines how much time can a cluster instance take hold of a job lock. If it takes longer than that time, another cluster instance could start the same locked job, which can cause database deadlocks. Left with the default value of 1800000 milliseconds.
Troubleshooting
Error when restarting tomcat: The configured state does not match recorded database state
When you see a message like this in your console or Openbravo.log:
8504 [symmetric-engine-startup-0] ERROR org.jumpmind.symmetric.AbstractSymmetricEngine - The configured state does not match recorded database state. The recorded external id is 7EAAA333467244FE8527B7C57F082D4B while the configured external id is 2ABAF56A0FFD4764983814DE5A0C68F0. The recorded node group id is {} while the configured node group id is {}
This probably means that there is no symmetric-ds.properties file in the WEB-INF directory or it has invalid data.
FileNotFoundException in the log
When you see a message like this in the log:
Caused by: java.io.FileNotFoundException: http://main.openbravo.com:9080/openbravo/symmetric-ds/sync/registration?nodeGroupId=StoreServer&externalId=2ABAF56A0FFD4764983814DE5A0C68F0&syncURL=http%3A%2F%2Fstore1.openbravo.com%3A8080%2Fopenbravo%2Fsymmetric-ds%2Fsync&schemaVersion=%3F&databaseType=PostgreSQL&databaseVersion=9.3&symmetricVersion=development&hostName=oslo&ipAddress=192.168.1.14 at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1626)
This is caused by a wrong url in the symmetric-ds.properties file. So you should check the symmetric-ds.properties file.
Installing in Postgresql 9.1
If the install.symmetric.ds task is run against a Postgresql 9.1 database the following error will occur:
[java] 9624 [main] ERROR org.jumpmind.symmetric.db.postgresql.PostgreSqlSymmetricDialect - Please add "custom_variable_classes = 'symmetric'" to your postgresql.conf file
As the error messages describes, this problem is solved by adding this line to the postgresql.conf file and reloading postgresql:
custom_variable_classes = 'symmetric'