Projects:Retail Operations Buffer/Technical Specifications

Introduction

The retail operations buffer is a new function which will store each imported webpos transaction in an intermediate table before processing.

Goals/Requirements

The goal of the project is to make the integration between WebPOS and the backend more scalable and be able to handle very high WebPOS volumes which arrive in an unpredictable pattern.

An intermediate table will de-couple the WebPOS front-end from the backend and create a produce-consume buffer which can easily receive many records. The records are processed using the available server capacity.

Other advantages of an intermediate buffer table are:

• sending data from WebPOS to the backend will be much faster and not depend on the processing of the data itself, the data/record only needs to be stored in the backend. Processing happens asynchronously.
• there will be an exact and easy to read/check log of all the transactions received from WebPOS

Main Structure

The new function consists of the following parts:

• a new table to store imported data, and related window/tab
• a new table to archive processed data, with related window/tab
• code to process imported data entries and call relevant functional code to process data
• an extendable structure which makes it easy to process records in parallel using the available processors on the system

Technical Implementation

A new table will be introduced with the following specific columns (next to standard columns as client and org):

• type of data: determines the type of the data (order, business partner etc.), based on the type of data the correct processor is called.
• import status: can be Initial, Processed, Error
• stored datetime: the time the import record got stored in the table
• import datetime: the time the record got imported into the system
• data: text with length 40000
• error information: stores messages stack traces when an error occurs

The archive table will have the same fields.

The window to show import entries will be filtered by default only showing records with status Initial and Error.

The function will be implemented using a java class which gets triggered when a new import entry record is created. Also the import process should periodically run by itself to check if new records have been created. The import process will call the correct functional process to actually import the entry. For WebPOS this means that the existing OrderLoader, CustomerLoader etc. classes are called.

The import archive function periodically (hourly) checks if there are processed import entries (import status == 'Processed). If so then they are moved/copied to the import entry archive table and removed from the import entry table.

Handling Errors

When the import process encounters an error then the import record will move to the error status.

However, in retail there is already an POS error table. To prevent error records in 2 places when an error occurs in POS then they are stored in the POS error table and the record in the import table is set as processed.