Projects:Mobile Server Controller/Technical Specifications
This document gives an overview of the changes needed to implement a so called mobile server controller component for the store server project.
The mobile server controller will be present in the client, the store server and the central server. In each it will provide slightly different functionality:
- keep track which store servers are online and which are offline
- for the store servers which are offline actively ping these servers to identify the status of the server
- keep track if the store itself (in which the store server runs) is online, offline, or in a transitional state (going to online or going to offline)
- provide a webservice request handler which:
- can be extended by functional developers providing hooks/extension points to implement offline and online behavior
- implements generic behavior in case the store server is in a transitional state (by returning an error to WebPOS which should show the user a notification)
- provide store-back-online extension point which can be implemented by functional developers to trigger specific actions when a server gets back online
- changes to the multi-server routing layer to catch a store-transition state and notify the user.
- provide a generic communication layer/utils to easily communicate with other servers
These topics are discussed in more detail below.
Mobile Server Status
A mobile server can have one of 4 states:
- Online: the store server is reachable and can connect to a main server
- Transition to Offline: the mobile server is transitioning to offline, from this state it can still go back to online.
- Offline: the mobile server is not connected to the main backend server
- Transition to Online: the mobile server is connected to the main backend server but there could still be functional code which needs to synchronize data
The status is global for the mobile server and should be made available in a utility method:
The mobile server state is accessed from multiple threads. So code updating and reading the state should be synchronized.
In addition a webservice should be provided which can be called by an external system to retrieve the state of the mobile server. The authentication of the webservice should use the authenticated token approach used in multi-server environments. The idea is that this url/service is also used as the ping service to detect if a server is reachable or not.
A new read only (for the user) column/field should be added to the mobile server window/tab/table to register the current state of the store server. To know which record to update the 'Detecting who you are' section should be used. Note: the new column should be excluded from Symetric DS replication.
The mobile server state column is also used by the central server logic to keep track of the states of the store servers in its network.
The mobile server state is also used by the central server to register its own online status.
Detecting who you are
It is important that in the code we can see if we are a store server or a main server and which is your own server in the list of mobile servers.
The proposal is to add a 'Mobile Server Key' column to the mobile server table. It should be unique across servers.
In addition in the Openbravo.properties file we should support a new key: mobile.server.key
The value of this property is used to find the corresponding record in the mobile server definition table.
With this we should provide utility methods:
- to detect if a certain mobile server record is the current server. For example: isCurrentServer(MobileServerDefinition)
- to return if the current server is a store server or a main server:
This utility methods can be used by application code to detect and determine what code to run.
Generic communication tools
The mobile server controller component should provide utility methods which can/should be used by functional code to do requests to other servers. The functional code must use this code to make it possible (for the controller) to detect if a server is offline or online and start online to offline transitioning.
Store State Dependent Behavior
There will be functionality in the store server which should behave differently depending on the state of the store server. For example if the store server is online then a stock request can be done to the central server, if the server is offline then the stock request is executed within the store server by querying a local stock table.
To facilitate the development of different functionality depending on the state of the server the mobile server controller should provide a mechanism for the functional developer to use. The proposal is therefore to provide a new subclass of SecuredJSONProcess (it can also be an idea to extend from JSONProcessSimple instead) which should be used by functional code which needs to provide services which are store server state aware. Let's call this SecuredMobileServerProcess. This class will have the following methods:
- execWhenOnline: abstract method, needs to be implemented by subclasses
- execWhenOffline: as a default forward to the execWhenOnline
- execWhenTransitioningToOnline: default behavior should be to throw an error/exception which in the WebPOS will show a message: Server transitioning to online, please retry in a few minutes
- execWhenTransitioningToOffline: default behavior should be to throw an error/exception which in the WebPOS will show a message: Server transitioning to offline, please retry in a few minutes
Calling the central server from the store server
Many requests need to be forwarded to the main server from the store server.
When forwarding a request from the store server to the central server we should also send the mobile server key and the store server state. So that the central server knows continuously what the state of each store server is. This means that on the central server we need to do additional administration and have a slightly different implementation.
Therefore the proposal is to implement a subclass of SecuredJSONProcess (it can also be an idea to extend from JSONProcessSimple instead) for implementing central server webservices.
This class should in its generic part register/see from which store it is being called and update the store server status in the mobile server definition table.
Functional code should create a subclass of this new main server class to implement central server behavior.
Note: in general the same services/urls should be provided by the central server and the store server. As also webpos systems should be able to directly work with the central server. So the store server as well as the main server apis are installed on the central server.
Store: Detecting and transitioning to offline
The mobile server definition should be extended with a boolean field (Offline Indicator Server) which can be used to indicate which servers can be used to detect offline or online mode.
When the store server does a request to one of these servers and a specific exception is returned (if the called server itself reports offline) or if the request times out then the mobile server controller needs to start a process to detect if it should move to offline.
This transitioning-to-offline logic should be developed in a pluggable component so that it can be replaced by a module.
First, we should provide a method which can be used by the functional code to do requests to other servers. This method should detect that a server is offline, check if this server can be used to determine offline status and if so, start the transition to offline logic.
A proposed method name:
- callMobileServer(mobileServerKey, serviceName, parameters)
The offline controller part of the solution should continuously ping offline servers to check their state. The mobile server definition table should have a new field which can be used to define a url which can be called to do the ping for the status request. This part (pinging for status) is very similar to the code proposed for the central server to keep track of which servers are offline and which are online (see next section).
The offline logic should decide when to move the store server state to transitioning to offline and finally offline. The logic can for example be that it tries 5 (or 10, configurable number of retries) times to ping the failing server. If the server remains failing then the server moves to offline state.
The server remains offline as long as there are mobile servers with the 'Offline Indicator Server' flagged and which are in offline state.
So to summarize:
- functional code does request using utility method provided by mobile server controller
- request times out or fails with a specific exception thrown by server
- if the request times out then the called server is put to offline status.
- the offline controller is triggered, it checks each of the offline mobile servers and starts pinging them. It can keep track of the number of failed pings (new column in the mobile server definition table) since the last success. If it is more than a configurable treshold and the mobile server has the 'Offline Server Indicator' flagged then the store server moves to offline status
- the offline controller continuously pings the offline servers to detect when they come back online.
- when all the mobile servers with the indicator flag checked are reachable the transition-to-online process starts. That's discussed in a next section.
Store: Detecting and transitioning to online
The store server has the list of mobile servers and also keeps track of the servers which are in offline state. For these servers the mobile server should have a periodical (every 5 minutes or so) process running which checks and pings the offline servers to detect if they are offline or online.
The url to ping was already introduced in a previous section as a new column/field in the mobile server definition table.
Note: special case: the url call to the central server can be successfull and returns the state of the central server. If the central server is offline then also the store server should be set as being/remaining offline.
When all the mobile servers with the 'Offline Server Indicator' flag checked are back online (and return status online) then the mobile server can start the process to move online.
An important part of the transition to online phase is that functional application code is able to process transactions which are queued. Only when all the functional code is ready with all these queued transactions can the store move back to the online status. We should therefore provide an extension point where functional code register a hook which can be called by the mobile server online transition code. This can be done using weld components
It goes through the following steps:
- sets the store status to 'Transitioning to Online'
- call extension points one by one, let them execute their work, if one of them fails then move back to offline and retry after a specific time period
- if all extension points finish then set the state to online
Central: Tracking online status of the servers
The central server has as important role to keep track of the status of its servers in the network. When ever a request is done to a server which times-out then the central server should move over to a ping-approach and pinging the 'failing' server.
To implement this we need to provide the following:
- a callMobileServer(mobileServerKey, serviceName, parameters) method which can be used by functional code. It should handle the timeout by registering the server as being offline. It can use the server status field/column added to the mobile server definition table in a previous section.
- the central server should have continuous code running to check the status of offline servers using a ping/url. The url which to call for checking the status should be added as a new field in the mobile server definition table.
An extension can be that we register in the central server (in any server really) when the last successfull request was done from a server. For any server for which no recent (configurable) successfull request was received the central server will ping the mobile server.
Calling alternative servers
In the proposed method: callMobileServer we can also add a parameter: tryFailOver, if true then the system should try a failover server providing the same service if the call to the mentioned/named server fails or times out. If all failovers fail then the server should start the transition to offline logic.
2016Q1 Development topics
The import transaction process currently follows an asynchronous approach. So the transaction is stored in the import table and then processed asynchronously with a separate process. This has large advantages for high volume transaction environments.
However, for some environments it is needed to do synchronous processing and to reply to the WebPOS if the transaction succeeded or failed. So in this development we need to extend the solution and make it configurable so that some requests are processed immediately and the result is send back to the client. Many parts of the current architecture can be re-used for this. So it is probably mainly calling the current solution in a different way. Or alternatively research if the current preference to enable/disable the import entry function can be extended to support a mixed model (save in the import entry table but process directly).
Central Server - Store Server online/offline request
We should deliver several additional functions/features related to offline and online changes in store servers:
- Retrieve current store server statuses: a service running on the central server which can be called from an external system, the service should return the current status of the store servers.
- Actively call a separate service (by central server) if a store server status changes from offline to online or vice versa
Add ping and authenticate flag to mobile server definition
The mobile server definition should be extend with 2 new fields:
- Ping request service: is used to build to url to call to ping a server for its availability. The ping request url is then the combination of the server address and the server ping request service.
- Authenticate yes/no: a checkbox which controls if authentication tokens should be send as parameters to this server.
Both fields should be used by server side code as well as client side code when implementing the ping service.