Projects:External data loading process/Old Technical Specifications

Introduction

In this document are described the developments needed to create an engine to insert and import data from external source to Openbravo.

New API

• Create an "ImportEntryManager" class.
  • The main manager for performing multi-threaded and parallel import of data
  • It is a singleton/applicationscoped class it should preferably be obtained through Weld
  • Be able to manage new entries in entity ImportEntry
  • It runs a thread that it is going to manage the imported records periodically
  • The thread is going to be launched when tomcat starts and stopped when tomcat stops
  • Manage what process is the owner of importing the data
  • It provides methods for error handling and result processing

• Create an "ImportProcessContextListener" class.
  • Initializes and stops the import process layer by calling ImportEntryManager start and shutdown methods

• Create an "ImportEntryProcessor" abstract class.
  • Responsible for importing/processing instances for a specific TypeOfData.
  • It is a singleton/applicationscoped, it implements a generic approach to make it possible to do import of data in parallel threads
  • To avoid deadlocks every thread is going to have a key assigned. In that way all the data is going to be assigned to one key and it is not going to be possible to mix data from same key in different threads
  • Implement a method to handle the thread creation
  • Provides standard features related for error handling and transaction handling.
  • Using Java ExecutorService be able to create a threadpool with a fixed size
  • It does not provide a duplication data processing, this it should be done by the class implementation

• Create an "ImportEntryPreProcessor" class.
  • Abstract/default class used to process an import entry when it is created before it is stored in the database
  • Is called from the ImportEntryManager

• Create an "ImportProcessUtils" class
  • Contains some Utils methods to be used by ImportEntryManager

• Create an "DataSynchronizationProcess" abstract class
  • Provide the base for inserting de data in entity ImportEntry to be inserte later
  • Give the method to detect duplicated records
  • Give a method to insert the errors in Error Window

New Entity

Entity that it is going to be used as a temporal table to store the data to be imported

Import Entry

• Table: C_IMPORT_ENTRY
• Fields:
  • Stored
  • imported
  • Jsoninfo
  • ImportStatus
  • Typeofdata
  • Errorinfo

Sequence Diagram