Projects:HandleUnstableNetworkConnections
Contents |
Introduction
The goal of this project is to improve the robustness of Web POS in case of unstable network connections.
This project will improve the operation of Web POS in the following areas:
- offline mode detection
- failover in case of intermittent network connections
- data sync robustness when connections fail intermittent
Changes in the Web POS
Refactor of the way offline detection works
Currently the Web POS uses a ping request to detect whether the backend server is still reachable or not. This mechanism has several problems:
- It doesn't work well in unstable networks
- It has the potential to overload the server in the case of a big amount of terminals
Therefore, this mechanism is going to be removed. Instead of having a ping, each Ajax request will be responsible of having a correct fallback process in case the connection fails due to network error or timeout.
However, the ping will be preserved once the Web POS goes offline, to ensure that the synchronization is automatically initiated when the connection comes back.
The following tasks need to be done as part of this topic:
- The ping needs to be removed in general when working online.
- Changes to the Ajax request infrastructure (ob-datasource, OB.DS.Process, ...) to ensure that the new way of detecting online/offline is handled correctly when the request fails
- Review of all the places in the code which are using the ConnectedToERP variable (which will no longer be valid), and change them.
- Change the code so that in case the connection is lost the ping is restarted, but it's stopped again in case it comes back.
Refactor the way the terminal object is generated
During the login process, there are several requests to load the terminal object, which contains several configuration properties. During the first login, this is not avoidable. However, in subsequent logins this object can be retrieved from the local database, and refreshed asynchronously after the login is complete.
This has two main advantages:
- Perceived performance during the login will increase significantly. This is specially true in high-latency, and in high error ratio networks.
- Robustness of the login process will improve: As quite a bit less requests will be done during the login process, the process itself is less likely to fail under unstable networks.
The following tasks need to be done as part of this topic:
- The part of the login which currently loads the terminal needs to be changed so that it retrieves it from the local database instead when it's already there.
- When the terminal has been loaded from the database, then it needs to be refreshed asynchronously when the login is complete, in the same way the data models are currently handled right now
- As the terminal object may change during this refresh, it's possible that some parts of the Web POS need also to be changed to take this into account (check for example the document sequence, or similar properties).
Implementation of a common framework for all data-sending processes
Currently there is a great deal of logic in the OrderLoader to handle special cases which might happen when there are connection problems:
- The same document might be sent more than once if there are problems with the connection (there is logic to avoid saving the document more than once)
- There is also logic to save the document as an error if the process failed for some reason
- There are checks to verify that the database is still working correctly (ie. there was no catastrophic failure in the backend systems, we still have db connections available, ...).
- There is code to ensure that each document is handled in a transactional way.
This logic is currently in the OrderLoader, and some of it is likely as duplicated code in the other synchronization tasks (cash up, cash mgmt, customers). Some of it may even be missing in the other sync tasks.
As part of this project, we will create a common framework so that all these similar tasks (OrderLoader, cash up handling tasks, cash mgmt movement, ...) share this logic. This framework will also be used in any other new synchronization task.
The following tasks need to be done as part of this topic:
- A new abstract class which extends the JSONProcessSimple task will be created. This new class will contain all the logic which is relevant for all synchronization tasks.
- All the synchronization tasks will be refactored so that they extend this class.
Besides this, also changes will be done to make this part of the Web POS extendable. To achieve this, the following changes need to be made:
- The synchronization part of the client-side needs to be made generic. A list of the synchronization entities, alongside their corresponding URL to synchronize them, will be added in the Web POS. This list will be modifiable by modules, and the process will read it in order to know which entities need to be synced, in which order, and what backend processes need to be called for them.
- The "Errors while importing POS orders" window needs to be changed, to support extensibility in the types of documents which can be synchronized.
Ability to persist the client-side log as a file
The client-side log will be used to monitor the requests of the system (as shown in the next section), so a new functionality will be added to persist this log in the client as a file, so that in the case the network synchronization fails, the log will still be there.
This functionality will use the FileSystem API present in the HTML5 standard.
Testing and monitoring
As part of this project, there will be extensive testing of the Web POS in different types of unstable networks, and also we will study the possibility of adding some monitoring functionality to ensure that we can monitor customer environments in such a way that we can find out why they fail in case the network has problems
Testing
The testing phase will involve both simulating slow networks using specialized tools, and testing with a local network which we know is both slow and unreliable.
During this testing phase, we will test the main flows of the Web POS with different kinds of networks and different amount of data, to ensure that it works correctly in all cases.
As part of testing, we will also analyze some tools to monitor the network status, to see if they can be used in the networks of real customers.
http://oss.oetiker.ch/smokeping/doc/reading.en.html
Monitoring
The main idea here is to try to see if it's possible to find out the specific part of the Web POS is failing in case of a problem with the network.
One way to solve this would be to use an external library/utility to try to monitor at request level, such as this:
http://www.appdynamics.com/products/end-user-experience-management
However, this utility in particular seems to have some problems (would force us to inject their own code inside the Javascript Web POS code, and it's not clear if it works offline (most likely not)).
Therefore, we could also implement our own solution for this. One possible idea would be to use the client-side logging, in the following way:
- We can centralize the few remaining requests not currently done via our OB.DS.Process/OB.DS.Datasource APIs so that they use them too.
- After we do this, using the client-side logging, we can log all requests to the backend which fail (including all sort of relevant data, such as request information, parameters, error message, timestamp, ...)
- We can also log all requests which take more than some time, in order to detect performance problems (most likely related to network problems).
- The client-side logging will automatically send this information to be saved in the backend. However, we can modify the client-side logging so that it's also persisted locally, possibly in a file in the file system, using HTML5 technologies (it seems HTML5 allows at least some browsers to access the filesystem).
Documentation
- Feature request in mantis: https://issues.openbravo.com/view.php?id=26019