Retail:Store Server Update Strategy
This document describes the update strategy for installing new features and (hot) fixes in an Openbravo multi-server environment. It presents the main steps and also the consequences and operational guidelines for special situations which might arise during the update.
This document does not describe the organization of the update process itself, still some general comments can be made:
- It is strongly advised to perform updates outside of office hours.
- All updates should be tested from a functional and performance perspective before deploying.
- Also the update procedure itself should be tested on a pre-production environment.
- Backups should be done before the update starts, rollback to a backup has been tested.
This document should be read together with the overview of the main windows/settings which play a role in updating store servers.
Environment & Versions - Main Aspects
The following points summarize the main characteristics of multi-server version handling and updating:
- There are three main servers/components in the environment which need to be updated:
- Central Server
- Store Server
- WebPOS system
- The store and central server each have a list of modules installed.
- The store and central server might have different modules installed but the assumption is that there are also modules which are the same.
- The version comparison between store and central server can only be done on these common modules which are installed both in central and the store.
- The central server is always assumed to be in the same or a newer version than any store server.
- Each store server will receive the list of installed central server modules (and their version) from the central server.
- With this list each store server can determine if it is still in the same version as the central server.
- If a store server is in a different version than the central server then the store server is in an incompatible version state.
- a store server in an incompatible version state will transition to offline.
- If a store server is not in the same version then the following applies:
- The store server may not receive replicated data from the central server, it can only send data to the central server.
- Requests from a webpos system are only allowed if the webpos is loaded from the store server. A webpos system which received its application code from another server (in a different version) may not communicate with a store server in an incompatible state.
The system provides several windows in the Openbravo backoffice to check version status, data pending to be synchronized and the list of modules installed in a store server:
WebPOS Client Mode
WebPOS clients can work in 2 modes:
- Central-Mode (standard/default): the WebPOS uses the url of the Central Server. The client will receive its application code/artifacts and updates from the central server but will send actions/get business data from the store server (if available). WebPOS clients are automatically updated. This means that in this mode the WebPOS client is in the same version as the central server.
- Store-Mode (offline): the WebPOS uses the store server url. For special situations the WebPOS system can get its application code from the store server. The switch to Store-Mode is a manual action (for example use different short cut on the desktop). In store-mode the WebPOS client is in the version of the store server. This is described in more detail in a later section.
The default mode for WebPOS clients should be Central-Mode. The Store-Mode is only used in special situations.
Openbravo does not provide an automatic mechanism to switch from central to store mode. The exact navigation depends very much on the custom situation, it is only needed rarely and can be solved easily on-site by for example adding a short-cut for store-mode on the users desktop.
Main (Happy Path) Approach: CS first, SS next
This section describes the update steps in the standard case:
- The update is done for the central server and all reachable store servers. If a store server can not be reached (no network connection) it can not be updated. This case is discussed separately.
- When starting the update the following store servers need to be identified (you can use the Store Server Version Overview window for this):
- Store servers which can not be reached (no network connection) → can not be updated until connection to the network is re-established
- Reachable Store servers which have data to be replicated from store to central → can be updated, but update should be done outside of shop opening hours
- Store servers which do not have data pending to be replicated from store to central → can be updated, update can start outside of office hours, can continue during office hours also.
- As the update is assumed to be outside of office hours most store servers will be the latter, reachable and no data to replicate.
- When a store has data to replicate and it can’t be updated before the store opens then it should not be updated. The update should happen when there is no more data to replicate from the store to central or when the update can be done when the store is not operating. This situation/handling is described in more detail in a later section.
The update steps:
- Stop all reachable (network connection) store servers. Although we can support an update with all store servers running it would mean that they would go to offline when the central is being updated. It seems more efficient to stop the store servers before.
- Stop the central server.
- Update the central server.
- Start the central server.
- Update the store servers, this can be done in parallel in sets of stores.
- Start each store server after they are updated.
If the update takes more time than anticipated and not all store servers could be updated and started (and which have no data to replicate to central):
- they can be updated also during store opening as the WebPOS clients will work on the central server.
- When the store server has been updated and goes live in the store the WebPOS clients will automatically detect and start using it.
- This assumes that the total number of stores/WebPOS clients in this situation is within the limit set by Openbravo.
The WebPOS clients will automatically obtain the update (from central) when the user opens the WebPOS and logs in.
Special Cases and solution strategies
Central Server Update Fails
In case the central server update fails the central server update should be rolled back and the update should be rescheduled for a later date (after analyzing and fixing the issue).
Store Server Update Fails - No data pending to replicate
This is the case that the store server has no data pending to replicate. The store server is down and being updated, but the update fails:
- The store server can remain down/stopped.
- The WebPOS clients will automatically switch to the central server and can operate on the central server.
- As the store can operate the store server update can continue in parallel, analyzing and solving the update issue.
- This assumes that the total number of stores working directly on the central server is within the limits set by Openbravo.
Store Server Update Fails - data pending to replicate
- There is data to replicate but the store server update fails.
- The store server should be rolled back to the previous version and started.
- From then the handling of this case is the same as for a Store Server which is offline. The store server is not updated, it will connect to central, but be offline as it is in a different version. Its WebPOS clients will/should move to store-mode (is a manual action to use a different url).
- As the store can still operate (in store-mode), the solution approach is to analyze the update issue and re-try the update outside of store opening hours.
Store Server not reachable - still in previous version
In this scenario most of the complete environment (central/stores) has been updated, except for (a few) stores which are not reachable due to network issues. There are several sub-cases which are discussed separately below.
Store internet connection down
If the store can not reach the central server (no internet connection) then the store continues to operate offline. Both the store server and the WebPOS clients in the store remain in the previous (store) version. When the internet connection is restored one of the next cases applies.
Store internet connection restored outside opening hours
In this case the store server can be updated before opening hours.
Store internet connection restored during opening hours
This is the case that the store and central are in different versions during operating hours. The webpos systems continue to work with the store server until the end of business day. After which the store server can be updated.
Store Server and Central Different Version - Store to Central Data Pending to be synchronized - operational internet connection
This is a special case:
- The store and central server are in different versions, and
- there is data pending to be synchronized. But the data replication does not happen due to a low level error, and
- there is (internet) connection for the store to central and back
The following will happen in this case:
- As there is an internet connection the WebPOS clients will automatically update from the central server (central-mode).
- The store server will detect that it is in a different version than the central server and will not accept requests from WebPOS systems in central-mode.
- The WebPOS client will not work with the store server then.
- If there is data pending to be synchronized from the store server to central then the WebPOS clients are not allowed to work with the central server.
- The WebPOS clients need to be manually switched to store-mode. Which means they will get their application code from the store server and work with the store server.
- The store server will go offline as the store and central server are in different versions and there is data to be synced.
- The store can continue to operate in offline mode.
To summarize, in this scenario the store server is in a different version than the central server. This means that:
- it will replicate data to the central server if it can
- The central server will not replicate data to the store server
- The WebPOS clients need to be manually moved to store mode.
- The store server will remain/switch to offline mode.
The next step is to analyze why the store can not replicate data. When that is resolved the update procedure can continue as described in other scenarios above.
Overview of states after update
To provide additional clarification. Assume an update from V1 to V2 for the environment then the following cases can exist:
- Central Server will be in V2.
- Store servers which have been updated are in V2 (this should be most/all store servers).
- WebPOS clients of an updated store server are in V2.
- WebPOS clients of a store server which is stopped and has no data to replicate, are in V2 and will work directly with the central server.
- Store servers which are not updated and working in V1 are in offline mode.
- WebPOS clients in V2 of stores with a store server in V1 which has no data to replicate will not work with the store server but will work with central server.
- WebPOS clients of stores with a store server in V1 and data to replicate work in store-mode in V1.
- WebPOS clients in V2 of stores with a store server in V1 will not work if the store has data pending to replicate. The user needs to switch to store-mode (to V1).
Hot Fix Update
A hot fix update is a special update which needs to be applied quickly to solve an urgent problem in the production environment. These types of updates are normally backward compatible. So no need to deny requests of a different version which is compatible with the version you have.
- Often a hot-fix will not change the version of any modules. If no version change is done then an hot-fix can be installed as is without any further actions. This will not trigger any version incompatabilities. The next bullets on setting a preference are then not needed either.
- To support this a preference can be set: 'ignore-version-check´. This preference can be used temporarily until a next non-backward compatible install is done.
- If this preference is set then the store servers will not execute any version-compatibility checks. This means that the store servers will operate normally as if they are in the same version as the central server. They will not go offline and allow connections from WebPOS.
- Before setting the preference one should be sure the update is backward compatible. An update version can be compatible with other versions if:
- It does not change the format of the json messages send or expected to/from other servers, and
- If it does not change the datamodel, and
- If it does not change the conceptual meaning of the datamodel or properties.
The hot fix process can then be summarized as follows:
- Set the 'ignore-version-check´ preference in the central server. It must be set on client level. The preference is replicated to the store servers.
- Deploy the hot fixes first in the central server and then restart the central server. This restart can be done in a short time.
- The WebPOS clients will update and get the hot fix version.
- When the WebPOS clients communicate with the store server or the store with the central server, the logic will detect version differences. But because the preference is set to ignore-version-check no version incompatibility logic is triggered.
- The hot fix can be deployed to store servers one by one. A store server can be stopped as long and updated as long as it does not have data to replicate to central. You can use the Store Server Version Overview window to check this.