View source | View content page | Page history | Printable version   

Projects:External data loading process/Technical Specifications Module

Contents

Introduction

In this document are described the technical specifications of the EDL module to load and extract data from external resources.

Module Definition

Name
External data integration in the ERP
DB Prefix
OBEDL
Java Package
org.openbravo.externaldata.integration
License
Openbravo Commercial

Module Windows and Tabs

EDL Process

System window where available EDL Processes are defined. The EDL Processes must belong to a third party module where the process logic is implemented.

Fields:

Name
Name of the EDL Process.
Search Key
Code of the EDL Processes. Used to identify the Java classes implementing the process with Qualifiers.
Type of Data
Determines the type of the data to be processed.
Storage Location
How is stored the Data that needs to be processed.
Has Input
Flag that determines if the process needs to receive the data.
Has Output
Flag that determines if the process produces data that needs to be send or stored by a Output process.
Synchronous
Flag that determines if Requests of this process are run synchronously or asynchronously.
Record Size
On asynchronous Processes determines the number of items that are included on each batch process.
Commit Items
On asynchronous Processes determines the number of items that are processes before doing a commit.

Constraints:

EDL Output Type

System window where available Output Types are defined. The Output Types can belong to third party modules.

Fields:

Search Key
Code of the Output Type. Used to identify the Java classes implementing the output process with Qualifiers.
Requires Flag
Several flags to show the corresponding field on the output configuration when the output process needs some of them.

Constraints:

Debug Output Type

Output type included in the EDL Module. It prints the output content on the Openbravo log file. Implemented by OutputDebuggerProcessor class.

WSRest Output Type

Output type included in the EDL Module. It sends the output content using Rest Web Services. It requires to configure the Path, User, Password and the Web Service Method to be used on the Web Service request. This Output Type implements the storeSendingData() method so all the content send by this output is stored and visible on the EDL Request window. Implemented by OutputWSRestProcessor class.

EDL Configuration

Client/System window where the available EDL Processes on each client is configured.

Header Fields:

EDL Process
The process to be configured.
Other fields
Depending on the selected process other fields might be included by the module where the EDL Process is defined.

Output tab Fields:

Sequence Number
To determine the order of the Output processes execution.
Output Type
The output process to be executed.
User, Password, Path, Filename, Web Service Method
configuration fields only visible based on the selected Output Type.
Other fields
Depending on the selected process or Output Type other fields might be included by the module where the EDL Process is defined.

EDL Request

Client/System window where all the EDL Requests are shown. All the requests are automatically processed when they are added. In this window it is possible to view the status of each one and reprocess them in case it is needed. If the request is asynchronous it is also possible to check the request lines with the batches and their process status. Some output types can store the output that has been generated for each request or request line if it is asynchronous. This content is also visible on this tab.

Header Fields:

Status
The status of the request: Initial, Processing, Error, Success and Voided
Organization
The organization owner of the request.
EDL Process
The process of the request.
Request Date
The time-stamp when the Request was added to the system.
Execution Date
The time-stamp when the Request process was finished.
Request Data
The raw input data in case the Database Location is selected as Storage Location.
Context Data
JSON String with the context data in case the EDL Process implements it.
Response
Text with the response of the EDL Process.

Request Line tab fields:

Status
The status of the line: Initial, Processing, Error, Success and Voided
Import Entry and Archive Import Entry
The Import Entry used to schedule the execution of the request line.
Error Msg
Text with the error message in case of failure.
Linedata
Text with the batch of items to be processed on the request line.

Ouput Content tab fields:

Request Line
The Request Line that has originated this output content.
Execution Date
The time-stamp when the content was generated.
Content
The text with the output content.

Processes

Process (header)
Reprocess the selected Request.
Discard (header)
Cancel the selected Requests in Error status. If a request fails and it is not desired to reprocess it again it is possible to void it by this button.
Process (Request Line)
Reprocess the selected Request Lines.

JSON Type of Data

Basic implementation of a type of data to manage items received as a JSONObjects in a JSONArray. Available for Asynchronous and Synchronous EDL Processes.

Implemented by following classes:

Public Classes

org.openbravo.externaldata.integration.process.ProcessRequest

Main class to manage requests. It implements the following public methods:

Abstract Classes

T and U types are defined by the Type of Data implementation. T is the type of each item and U is the type of a collection of items. In case of JSON type of data each item (T) is dafeined as a JSONObject and the items are received and batched (U) as JSONArray objects.

org.openbravo.externaldata.integration.process.DataProcessor< U>

Implemented by the Type of Data. Implementations must define a Qualifier with the search key of the Type of Data that are implementing.

It has to implement a method to convert the raw data as it is received to the U type. The abstract method has available several fields to get the needed information to retrieve the data.

org.openbravo.externaldata.integration.process.AsynchronousProcessor<T, U>

Class to implement by any Type of Data available to be executed asynchronously. Each implementer has to define the getDataBatcher(), getBatchFromList() and getItemIterator() abstract methods.

The items to be processed are grouped in batches of the type defined by the Type of Data. The size of the batch is defined by the EDL Process. Each batch is stored as a String in a Request Line that it is processed asynchronously. When the request line is processed the batch has to be iterated to retrieve the items in the type defined by the Type of Data.

org.openbravo.externaldata.integration.process.SynchronousProcessor<T, U>

Class to implement by any Type of Data available to be executed synchronously. Each implementer has to define the getItemIterator() abstract method.

org.openbravo.externaldata.integration.process.CustomStorageLocation

By default the module provides 3 different Storage Locations where the data to be processed on each request is stored.

Any module can add additional Storage Locations by adding a new option on the list reference OBEDL Storage Location and extending this class with a Qualifier with the code of the new option.

org.openbravo.externaldata.integration.process.CustomValidation

When a request is added some validations are executed. Extending this class it is possible to add new custom validations. There are 2 methods to implement:

org.openbravo.externaldata.integration.process.ItemProcessor<T>

Class to be extended by each EDL Process. It is in charge of implementing the logic needed to process each item.

It has to implement 2 abstract methods:

The class also have 3 other methods that can be overridden to execute additional logic:

org.openbravo.externaldata.integration.process.OutputDataProcessor

Class to be overwritten in order to implement different output data processes. These processes are executed when all the items of a request are successfully executed. Each class implementation is identified with the corresponding record added in the EDL Output Type window by a Qualifier and its value.

Each implementer has to implement 2 abstract method:

There are other 2 methods that can be overridden if needed:

org.openbravo.externaldata.integration.process.ProcessDataProcessor< U>

Class implemented by each EDL Process to customize how the data is stored and managed.

It includes a single abstract method:

This class includes other methods that can be overridden to cover other scenarios where the default Type Of Data implementation needs to be modified:

Retrieved from "http://wiki.openbravo.com/wiki/Projects:External_data_loading_process/Technical_Specifications_Module"

This page has been accessed 2,366 times. This page was last modified on 14 September 2016, at 15:44. Content is available under Creative Commons Attribution-ShareAlike 2.5 Spain License.