Modules:External Integration Infrastructure/UserGuide
Overview
This document will explain how to use the External Integration Infrastructure module to create a connector between Openbravo and an external system.
It will describe the classes to be extended and the interfaces to be implemented to achieve the desired level of customization.
It is divided in three parts:
- And at last this document describes how to define the functional mappings between Openbravo and the external system.
- Then the import process is described. Connectors should extend this functionality if they must synchronize data from an external system to Openbravo.
- At last the export process is described, and it is relevant if the connector to be implemented is supposed to transfer data from Openbravo to the external system.
Entity Mappings
One of the main of this module is to provide a way to define mappings between Openbravo and an external system. This is done at two levels: entity and property.
The mappings at entity level are defined in the Entity Mapping window:
The following fields must be filled in to define a new Entity Mapping:
- Module: A link to the module that will store the functional mapping
- System Type: The name of the external system the current mappings applies to. It must be defined as an entry in the External System Type list reference.
- Mapped Entity: The name of the entity in the external system
- Mapping Entity: The name of the entity in Openbravo. A check is done in an event handler to ensure the name is proper.
- Integration Direction: The direction of the synchronization of the entity mapping being defined. Available options are:
- From External System to Openbravo: To be selected in mappings that will be used to import data to Openbravo
- From Openbravo to External System: To be selected in mappings that will be used to export data from Openbravo to the external system
- Is First Level Mapping: Entity Mappings can be divided in two groups: Main and Secondary. For instance, the mapping for Invoice will be main, while the mapping for Invoice line will be secondary, because it is needed as part of the Invoice mapping but it is never synchronized by itself. Main entities must set the Is First Level Mapping flag to true.
- Integration API Type: How the entity will be integrated from/to Openbravo:
- Manual BaseOBObject: Imported records are created manually, each property is set using the property mappings. Records to be exported are retrieved by specifying an HQL Query.
- Retail API: Records are imported and exporting using the Retail API, in a similar way to how it works for Retail POS Terminals.
- Is Query Base Object: TDB
- Skip Audit Update: Available starting from version 2.0.1900. It only applies to the From External System to Openbravo entity mappings and it determines if the audit field of the imported business object should be updated when the import is completed.
Dependencies between Entity Mappings
It is possible to define dependencies between entity mappings, that will be taken into account in the export process, to ensure that data is exported in the proper order.
For instance, if there are entity mappings defined to export Invoices and Business Partner, the Invoice entity mapping should depend on the Business Partner mapping, because if a business partner is created in the WebPOS and right after that it is used to create an invoice, we should ensure the the business partner is exported before the invoice, because otherwise the invoice cannot be processed in the external system because it references a business partner that has not been imported yet.
In a later sections it is explained how to schedule the export background process and how to define which entity mappings that process will export. It is important that if there are dependencies between two entity mappings, both of them are exported within the same export process.
Property Mappings
Once a mapping is defined at Entity level, it is time to define the Property mappings. These mappings define how the relevant properties of the Openbravo entity are mapped to the properties of the external system entity.
There are two ways to define the property mappings:
- In the Property Mapping Instance subtab of Entity Mapping window, for those mappings that are quite straightforward and meet certain conditions
- Programatically by subclassing a Java class (JavaPropertyMappingHandler), for those mappings that cannot be defined in the Application Dictionary
Defining Property Mappings in the Application Dictionary
When the property mapping to define is straightforward, it can be define in the Property Mapping Instance subtab of the Entity Mapping window.
The fields to fill in in the Property Mapping Instance tab depends on the Integration API Type of the parent entity mapping. Some are common to both of them:
- Module: The module that will store the functional mapping
- Property Mapping Class: The type of property mapping (basic, date, boolean, one-to-many. All the available property mapping classes are defined in a later section.
- External Property Name: The name of the property in the external system
- Accepts Null Values: Specifies if the property mapping accepts null values.If a null value is found, the import/export process will fail for the particular entry being synchronized.
Some are available only to the Manual BaseOBObject Integration API Type:
- Openbravo Property Path: The property path to the target property in the Openbravo entity. It can be simple (i.e. if the entity is Order, a simple property path would be documentNo) or composite (i.e. businessPartner.searchKey). An event handler will check that the property path is correct for the entity defined in the header tab.
- Identifies Record Univocally: Relevant only in Import property mappings, specifies if the current property mapping can be used to identify univocally a record in the Openbravo database. It will be used by the import process to try to find in the Openbravo database the record that is being imported. In most cases it is enough to set this flag to true for the properties that have unique constraints. If that's not enough, it is possible to customize the process to locate the imported record in the Openbravo database by implementing the ImportedBaseOBObjectFetcher hook.
And some are available only to the Retail API Integration API Type:
- Retail API Property Name: The name of the property in the Retail API endpoint
Property Mapping Class
The following diagram shows all the available Property Mapping Classes. Some are available for entities that use both Integration API Types, other only for one of them.
- DirectPropertyMapping, JsonDirectPropertyMapping: Basic property mappings, where the type of the property in Openbravo is a String
- UUIDPropertyMapping: Mapping class to be used when exporting IDs, in case the UUID must be transformed. The transformation is defined by implementing the UUIDTransformer interface.
- BooleanDirectPropertyMapping, JsonBooleanDirectPropertyMapping: To be used when the property represents a boolean value. The BooleanPropertyFormatter interface must be implemented to define how boolean values are represented in the external system (Y/N, true/false, yes/no, etc).
- DateDirectPropertyMapping, JsonDateDirectPropertyMapping: To represent date properties. The DatePropertyMappingFormatter must be implemented to define the date format to be used to parse/format dates between Openbravo and the external system.
- DateTimeDirectPropertyMapping, JsonDateTimeDirectPropertyMapping: To represent date time properties. The DatePropertyMappingFormatter must be implemented to define the date format to be used to parse/format dates between Openbravo and the external system.
- BigDecimalDirectPropertyMapping: To be used when the Openbravo property is a BigDecimal
- LongDirectPropertyMapping: To be used when the Openbravo property is a BigDecimal
- EntityPropertyMapping, JsonEntityPropertyMapping: To be used in export property mappings that represent foreign keys, when the external system expects not only some identifier of the referenced record, but more properties. In that case, a new entity mapping must be defined for the referenced property, and selected in the Referenced Entity Mapping field.
- OneToManyEntityPropertyMapping, JsonOneToManyEntityPropertyMapping: To be used when the property in Openbravo is a one-to-many property (i.e. the orderLineList of the Order entity). A new entity mapping must be defined for the referenced property, and selected in the Referenced Entity Mapping field.
- JsonExternalIdentifierPropertyMapping: To be used when it is desired to keep in Openbravo the value that identifies a record in the external system. It allows to map the identifier of a record in the external system with the identifier of the associated record in Openbravo. To achieve this, this kind of mappings are using internally the Table of IDs.
This is an example of a OneToManyEntityPropertyMapping:
This is the secondary entity mapping referenced by the OneToManyEntityPropertyMapping. Note that the Is First Level flag is left unchecked.
Defining Boolean, Date and UUID formats
A module that implements the connector with an external system must define the boolean, date and UUID formats, if they need to support any of those property mapping classes.
/** * Interface with two methods (getTrueValue(), getFalseValue()), that will be used by the * BooleanDirectProperty to determine the value used to represent TRUE and FALSE in an external * System * * Classes that implement this interface must include the external system type in an annotation */ public interface BooleanPropertyFormatter { /** * @return the value that represents TRUE in the external system */ public Object getTrueValue(); /** * @return the value that represents FALSE in the external system */ public Object getFalseValue(); }
/** * Interface with the methods that will be used by date/date-time mappings to determine the format * of dates and times in an external system * * Classes that implement this interface must be annotated with the {@link ExternalSystemType} * annotation. */ public interface DatePropertyFormatter { /** * @return the format of dates in the external system */ public String getDateFormat(); /** * @return the format of date-times in the external system */ public String getDateTimeFormat(); }
/** * Classes implementing this interface can be used with a {@link UUIDPropertyMapping} to change the * representation of a UUID using a particular format. */ public interface UUIDTransformer { /** * @param uuid * A String containing a UUID * * @return a String with the UUID received as parameter, transformed with a format chosen by the * class implementing this interface. */ public String transformUUID(String uuid); }
This is an example of an implementation of the BooleanPropertyFormatter and the DatePropertyFormater for SAP Business One. Note how the @ExternalSystemType annotation is defined with the searchKey of the external system type to specify that the java component is related to it.
@ApplicationScoped @ExternalSystemType("SAPB1") /** * Class that implements the Boolean and Date formatters for SAP Business One * */ public class SapB1Formatter implements BooleanPropertyFormatter, DatePropertyFormatter { @Override public String getDateFormat() { return "yyyy-MM-dd'T'HH:mm:ss"; } @Override public Object getTrueValue() { return "Y"; } @Override public Object getFalseValue() { return "N"; } }
Selecting the Properties of Referenced Entities
When a referenced entity in an export mapping (Openbravo to External System direction) is defined through an EntityPropertyMapping all the properties defined in the entity mapping of the referenced entity are displayed by default.
However it is possible to define a subset of the properties that can be displayed by configuring the entity mapping of the referenced entity as follows:
- In the header of the Entity Mapping, select the option Those Selected for the Selected Properties On Entity Referencing
- Then in the Property Mapping Instance subtab, we can decide for each of the property mappings if the property is shown or not with the Select On Entity Referencing combo field, by choosing one of the available values:
- As Identifier: this property will be shown as the identifier of the referenced entity. There can only be one property mapping with this option selected per each entity mapping.
- As Information: the property will be shown as part of a special property (<propertyName>_info) that will keep all the properties marked with this option.
- The field is left empty: the property will not be shown
So let's suppose that we have a property mapping with a property name named myproperty that references to an EntityPropertyMapping making use of this configuration. This will result in two properties added to the final export result:
- myproperty: a property with the value of the property selected As Identifier
- myproperty_info: a property with all the values of the properties selected As Information. It also includes the property selected As Indentifier.
Defining Java-Based Property Mappings
When a mapping between the Openbravo and the external system property is complex, it should be defined using a Java-Based Property Mapping.
This is done by creating a subclass of ImportJavaPropertyMappingHandler (for import mappings) or of ExportJavaPropertyMappingHandler (for export mappings).
In both cases, the subclass must:
- Declare the @EntityMappingId annotation, to specify the entity mapping these properties are being defined in.
- Overwrite the getPropertySorting method, to list the list of properties that will be mapped in the JavaPropertyMappingHandler being defined:
/** * @return a Map with the names of all the properties than can be generated with this * JavaPropertyMappingHandler. The key for every property name is an Integer representing * the sequence number of the related JavaPropertyMapping. */ public abstract Map<Integer, String> getPropertySorting();
ImportJavaPropertyMappingHandler
Implementations of ImportJavaPropertyMappingHandler must implement the setPropertyInBaseOBObject method to specify how each mapping should be applied to set a property in the Openbravo object being imported.
They can also overwrite the itemShouldBeImported method, in case there are some cases where the record provided by the external system must be discarded.
/** * This method must be overridden by the subclasses in order to validate if the * SynchronizableBusinessObject passed as parameter should be imported. * * @return true if the business object meets the requirements to be imported, false otherwise. */ protected boolean itemShouldBeImported(SynchronizableBusinessObject item) { return true; } /** * @return an Object with the value calculated dynamically for the property whose name is passed * as parameter. */ public abstract void setPropertyInBaseOBObject(T bob, String mappingName, SynchronizableBusinessObject sbo);
ExportJavaPropertyMappingHandler
Implementations of ExportJavaPropertyMappingHandler must implement the getValueOfProperty method to specify how each mapping should be applied to obtain a property from the Openbravo record being exported.
They can also overwrite the itemShouldBeExported method, in case there are some cases where the record being exported should not be processed and sent to the external system.
/** * This method must be overridden by the subclasses in order to validate if the business object * passed as parameter should be exported. * * @return true if the business object meets the requirements to be exported, false otherwise. */ protected boolean itemShouldBeExported(T item) { return true; } /** * @return an Object with the value calculated dynamically for the property whose name is passed * as parameter. */ public abstract Object getValueOfProperty(T bob, String mappingName);
This class also offers a method that can be overridden to define if any of the properties mapped with it should be displayed when the entity is referenced:
/** /** * Retrieves the java properties that should be exported when the entity related to this export * mapping handler is referenced by another entity * * @return the set of java properties that should be exported when entity is referenced. By * default it returns an empty set. */ public Set<String> getSelectedPropertiesOnEntityReferencing() { return Collections.emptySet(); }
The Table of IDs
The Table
The Table of IDs (OBEI_IDENTIFIER_MAPPING) is the table that keeps the information that associates a register in the external system with its corresponding record in Openbravo. This is done by storing their identifiers together with additional information. In particular, this table contains the following columns (apart from the mandatory columns required for required for security and auditory purposes):
- ad_table_id: the ID of the AD_TABLE which the Openbravo record belongs.
- record_id: the ID of the Openbravo record.
- external_instance_identifier: the identifier of the external system instance. See below for additional information.
- mapped_entity: the name of the entity which the record in the external system belongs.
- external_record_identifier: the identifier of the record in the external system.
How To Populate
There exists different sources from which this table can be populated:
- SynchronizableBusinessObjectExternalExporter (export flow): classes extending this one are inserting the information in the table once each record is exported to the external system.
- JsonExternalIdentifierPropertyMapping mappings (import flow): when an import entity mapping has a property mapped with this kind of mappings, then the information of the imported record will be automatically inserted in the table.
- Dataset: using a standard dataset. This is commonly used for "static" information where the identifiers in the external system are always the same like countries or currencies.
External Instance Identifier Providers
The Table of IDs has a column where the identifier of the external system can be stored.
When the synchronization with multiple instances of the same external identifier is supported it is necessary to have a mechanism to identify each instance. This is because the infrastructure needs to know which is the instance of each synchronized record.
This mechanism can be created by implementing the ExternalInstanceIdentifierProvider interface:
/** * This interface should be implemented for each external system with the logic to select the * identifier of the external instance that should be used when saving the external identifiers in * the Openbravo table of identifiers. Classes implementing this interface should be annotated with * the {@link ExternalSystemType} annotation to specify the external system. */ public interface ExternalInstanceIdentifierProvider<T> { /** * Retrieves the identifier of the external instance that should be used when saving records in * the table of identifiers related with that instance. * * @param source * Contains information that can be used to calculate the identifier of the external * instance. * @return a String that identifies a particular instance of an external system. */ public abstract String getIdentifier(T source); }
If no ExternalInstanceIdentifierProvider is provided then the identifier of the external system will be used. This is the same value used for the @ExternalSystemType annotation.
External ID Mappings Window
The contents of the Table of IDs can be seen in the External ID Mappings window.
The SynchronizableBusinessObject (SBO) abstraction
The connector infrastructure defines the SynchronizableBusinessObject class, an abstraction to represent an intermediate format for the data being imported/exported. Data is stored internally as a Map<String,Object>.
Modules that define specific connector implementations must provide means to convert the data being imported to a SBO, and an SBO to the data format that the external system expects.
The base connector infrastructure knows how to apply the property mappings to a record being exported to create a SBO, and to apply the property mappings to a SBO to build a record that will be created/updated in Openbravo.
Export Process Overview
The following image describes the main steps of the export process:
- A background job is periodically executed.
- For each entity being exported, the export process determines what records should be exported:
- If the Manual BaseOBObject Integration API Type is used, a query is executed to obtain the list of records that must be exported. The query will return the UUID of the records pending to be exported.
- If the Retail API Integration API Type is used, and endpoint of the Retail API will be invoked, and it will return a JSONArray.
- An EDL Request is created per exported entitiy, where the data to be exported (list of IDs or JSONArray, depending on the Integration API Type) will be stored.
- When the EDL Request is processed, the mappings are applied to the Openbravo records being exported and a list of SBOs is created.
- The specific connector module processes those SBOs and transform them into the data format expected by the external system.
- The specific connector module sends the transformed data to the external system
Selecting the records to be exported
The way to obtain the records that should be exported depends on the Integration API Type of the entity mapping:
Manual BaseOBObject
In order to define the query that will be used to select the records that will be exported, an implementation of the QueryExporter interface must be included per exported entity in the module that will contain the functional mappings.
This component, as well as all the other java components that operate at entity mapping level, need to declare the @EntityMappingId annotation.
/** * Interface to be implemented by the classes that define the HQL query to export an entity mapping * * Classes that implement this interface must declare the {@link EntityMappingId} annotation */ public interface QueryExporter extends Prioritizable { /** * @return a String containing an HQL query used to identify those records pending to be * synchronized by the {@link ExportExternalSynchronizationProcess} */ public String getSynchronizationQuery(); }
This is an example implementation of QueryExporter, that will be used to export products:
@EntityMappingId(PRODUCT_ENTITY_MAPPING_ID) public class ProductExporter implements QueryExporter { @Override public String getSynchronizationQuery() { final StringBuilder query = new StringBuilder(); query.append("select p.id from Product p "); query.append("where p.$incrementalUpdateCriteria "); query.append("and p.$clientCriteria "); query.append("and p.sapobmpIsreadytoexport = true "); return query.toString(); } }
Note that it is possible to use placeholders, keywords that start with '@', that will be replaced by the connector infrastructure with valid HQL filters.
This is the full list of available placeholders:
- $activeCriteria: Replaced with active = true
- $orgCriteria: Replaced with organization.id = :orgId, :orgId being the current user organization
- $naturalOrgCriteria: Replaced with organization.id IN (:naturalOrgList), :naturalOrgList being the natural tree of the current user organization
- $childOrgCriteria: Replaced with organization.id IN (:childOrgList), :childOrgList being the child tree of the current user organization
- $clientCriteria: Replaced with client.id = :clientId, :clientId being the current client of the user
- $incrementalUpdateCriteria: Replaced with updated >= (:lastUpdated), :lastUpdated being the date with the last time the current entity was synchronized. It should be used when modifications done on an existing record should be exported.
- $incrementalCreationOnlyCriteria: Replaced with creationDate >= (:lastUpdated), :lastUpdated being the date with the last time the current entity was synchronized. It should be used when a record should be exported only once, after being created. Subsequent updates on that records will not be exported
Retail API
To obtain the records that should be exported using the Retail API, an implementation of the CustomApiExporter interface must be done per exported entity:
/** * Interface to be implemented to export data from Openbravo to an external system. */ public interface CustomApiExporter extends Prioritizable { /** * Retrieves the elements to be exported to the external system. * * @param params * A JSON Object with the parameters required to use the export API. * @return a JSONArray with the items to be exported * @throws Exception * if there is a problem retrieving the records to be exported from Openbravo */ public JSONArray getItemsToExport(JSONObject params) throws Exception; }
This is an example to use the Retail API to obtain the Products pending to be synchronized:
@ApplicationScoped public class ProductExporter implements CustomApiExporter { @Override public JSONArray getItemsToExport(final JSONObject params) throws Exception { StringWriter writer = new StringWriter(); final OBPOSApplications posTerminal = MagentoTerminalHandler.getInstance().getMagentoTerminal( params); OBContext.getOBContext().setCurrentClient(posTerminal.getClient()); OBContext.getOBContext().setCurrentOrganization(posTerminal.getOrganization()); WeldUtils.getInstanceFromStaticBeanManager(Product.class).exec(writer, params); return new JSONObject("{" + writer.toString() + "}").getJSONArray("data"); } }
Parameters required by the Retail API (for instance, POS Terminal ID and the date of the previous export to do a incremental update) are automatically provided by the connector infrastructure in the params parameter.
Manual BaseOBObject Integration API Type - Export Process Customizations
Post SynchronizableBusinessObject Creation Actions
A SynchronizableBusinessObject is generated as the result of applying the export entity mapping. The following interface allows to define hooks to apply custom changes in this SynchronizableBusinessObject after it has been generated:
public interface ExportedSBOPostCreationHook<T> extends Prioritizable { /** * Applies custom changes on the export {@link SynchronizableBusinessObject} generated for an * {@link EntityMapping} in particular, like for example adding additional information. * * Important Note: the received {@link SynchronizableBusinessObject} contains the result of * applying the export entity mapping, so it is not recommended to use it for example to add new * properties into this object because they will be like unknown properties, i.e., they will be * outside of the entity mapping definition. * * @param sbo * The {@link SynchronizableBusinessObject} used for the export operation where * additional changes can be applied on. * @param itemToExport * The item to export used as source to generate the provided * {@link SynchronizableBusinessObject}. Important: this item is read-only, changes * should not be applied on it. */ public void applyChanges(SynchronizableBusinessObject sbo, T itemToExport); }
Import Process Overview
The following image describes the main steps of the import process:
There are two possible events that trigger an import synchronization:
- An import background process. The connector infrastructure will invoke the getRequestsInformation method that will be implemented in the specific connector module. When that method is invoked a request will be done to the external system, providing the entity being synchronized and a date the represents the previous time that same entity was synchronized. Or,
- The external system itself triggers the synchronization, and it provides to Openbravo the data that should be imported.
- In both cases, an EDL Request is created with the data to be imported.
- When the EDL Request is processed, the specific connector infrastructure will convert the data provided by the external system to an SBO.
- Then, the base connector infrastructure will apply the defined mappings on the SBO. Depending on the Integration API Type of the entity being imported, the import process will either create and save a BaseOBObject, or will invoke a Retail API endpoint.
Storing Imported Data in the Database
The way the imported data is stored in the database depends on the Integration API Type of the entity mapping being imported:
Manual BaseOBObject
The connector infrastructure will create (for new data) or update (for updated data) a raw BaseOBObject. The property mappings will be applied, and the resulting BaseOBObject will be manually stored in the database using DAL.
This section describes the hooks available to customize how the imported data is stored using the Manual BaseOBObject Integration API Type.
Retail API
To import data using the Retail API Integration API Type, an implementation of the CustomApiImporter must be done per imported entity:
/** * Interface to be implemented to import into Openbravo a JSONObject provided by an external system. */ public interface CustomApiImporter extends Prioritizable { /** * Imports a record into Openbravo. * * @param objectToImport * the JSONObject to be imported * @param params * A JSON Object with the parameters required to use the import API. * @throws Exception */ public void saveRecord(JSONObject objectToImport, JSONObject params) throws Exception; }
The connector infrastructure will invoke the saveRecord method with two parameters:
- JSONObject objectToImport: The result of applying the mappings to the record imported from the external system
- JSONObject params: Some parameters required by the Retail API that are automatically populated, such as the POS Terminal ID.
This is an example on how to implement a CustomApiImporter to import business partners using the Retail API:
@ApplicationScoped @EntityMappingId(IMPORT_BP_ENTITY_MAPPING) public class BusinessPartnerImporter implements CustomApiImporter { @Override public void saveRecord(final JSONObject objectToImport, final JSONObject params) throws Exception { MagentoUtils.runRetailImporter(CustomerLoader.class, objectToImport); } }
Triggering the Import
The External Integration Infrastructure module provides two ways of triggering the import of records. It can be done periodically, triggered by the execution of an Openbravo background process. And it can also be done on-demand, by invoking a class that will create the EDL Requests that will import the given records into Openbravo.
Background Process
If the discovery of the data to import should be done periodically, the ImportExternalSynchronizationProcess class must be extended:
/** * Abstract class to be extended by the background processes meant to import data from external * systems into Openbravo */ public abstract class ImportExternalSynchronizationProcess extends ExternalSynchronizationProcess { /** * Returns a list of ImportRequestInfo, for each one of these an EDL Request to import data will * be created * * @param lastUpdate * the last time the entity being integrated was exported * @return a list of ExportRequestInfo that will be used to create EDL requests */ protected abstract ExternalDataIterator<RequestInfo> getRequestsInformation(String mappedEntity, Date lastUpdate); ...
A connector with an external system that wants to import data periodically should extend ImportExternalSynchronizationProcess and implement the getRequestsInformation method.
The getRequestsInformation method receives:
- mappedEntity: The name of the entity to be imported (i.e. Product, BusinessPartner)
- lastUpdate: A timestamp that specifies when was the last time that the given entity was imported. It should be used to import data incrementally.
The method should return a ExternalDataIterator<RequestInfo>. An ExternalDataIterator is nothing more than an Iterator with an extra method that will be invoked each time one of its returned elements have been processed:
/** * An iterator that offers an extra method that should be called once the iterator consumer has * finished processing the current iterator element * * @param <E> * the type of elements returned by this iterator */ public interface ExternalDataIterator<E> extends Iterator<E> { /** * Method to call when the processing of the current element of the iterator has finished */ public void currentElementProcessed(); }
For import processes, the ExternalDataIterator should return instances of ImportRequestInfo:
/** * This class is used to keep all the information required to configure a particular import EDL * process. */ public class ImportRequestInfo extends RequestInfo { private Object objectsToImport; public ImportRequestInfo(Object objectsToImport) { this.objectsToImport = objectsToImport; } /** * @return the objects that have to be imported. The type of the imported objects depend of the * subclass of ImportExternalSynchronizationProcess */ public Object getObjectsToImport() { return objectsToImport; } }
The returned objectsToImport will be serialized and stored in the database as the content of EDL request lines.
On demand
Another option is that the discovery of the data to be imported to Openbravo is not done from a scheduled process, but on demand. For instance, this option should be chosen if it is the external system the one who triggers the import process (i.e. by making a request to a webservice defined in Openbravo).
The External Integration Infrastructure module provides a class that creates the EDL Requests, the EdlRequestBuilder. This is an example of its use:
EdlRequestBuilder.newInstance(IMPORT_PROCESS_ID, entityMapping, requestInfoIterator) // .setExecutionMode(ExecutionMode.SYNCHRONOUS) // .create();
Manual BaseOBObject Integration API Type - Import Process Customizations
The SynchronizableBusinessObjectImporter class is in charge of receiving the content of a record to be imported, applying mappings on it to create a BaseOBObject (if it is a new record), updated (if it did not exist) or delete it.
Matching the contents of an imported record with an existing record
The usual way to match the contents of an entry to be imported with an existing record in the Openbravo database is by defining a property mapping that will map a property of the external system with a unique property of the entity in Openbravo. For instance, the following image shows how
If there is no property in the external system that can be matched with an Openbravo unique property, the External Integration Infrastructure module provides a way to manually obtain an Openbravo record based on the information being imported:
/** * Interface to be implemented when there is a custom way to find a match for the imported record in * the database. This is only needed when it is not possible to define property mappings that * identify univocally the mapped record in the database * * Classes that implement this interface should declare the {@link MappedEntity} qualifier to define * the target SystemType and EntityMapping name * */ public interface ImportedBaseOBObjectFetcher { /** * Given a SynchronizableBusinessObject, uses its properties to try to fetch the mapped record * from the database. If the records exists it will be returned, if it does not exist, the method * will return null * * @param item * the SynchronizableBusinessObject whose properties will be used to fetch a record from * the database * @return the BaseOBObject extracted from the database or null if no mapped records exists */ public <T extends BaseOBObject> T fetch(SynchronizableBusinessObject item); }
The SynchronizableBusinessObject class has an properties attribute, a map that contains all the pair property name - property value provided by the external system.
Specifying when an imported change represents a deletion/deactivation
The connector infrastructure needs to know whether some data provided by the external system represents a records that must be created/updated, or if it represents that the records has been deleted from the external system.
If a given external system supports, that, then its connector implementation must specify how to tell if the data provided by the external system is a deletion. This is done by defining an implementation of the ImportedItemDeletionPolicy interface. It will provide a SBO as a parameter, and it must return true if the given SBO represents a deletion, and false otherwise.
It is important to note that when an external system notifies that a record should be deleted, the records is not deleted from Openbravo, it is just disabled. This is done to prevent referencial integrity problems.
/** * Interface to be implemented by those external system connectors that support deleting records in * the import process. * * Classes that implement this interface should declare a @ExternalSystemType Qualifier to specify the system type it * represents * */ public interface ImportedItemDeletionPolicy { /** * Given an SynchronizableBusinessObject, returns true if the imported item is meant to be deleted * * @param item * the item that will be taken into consideration to decide if it should be deleted * @return true if the item should be deleted, false otherwise */ public boolean isDeletion(SynchronizableBusinessObject item); }
Initializing a new BaseOBObject
Sometimes the data provided by the external system to create a new record in Openbravo does not contain all the information needed to initialize the BaseOBObject.
The following interface can be implemented to initialize a BaseOBObject when it is created as a result of importing an entry from an external system.
/** * Interface to be implemented when there is a need to initialize the properties of an entity that * are not mapped in the Entity Mapping definition in the AD. * * The {@link #initialize(Object, SynchronizableBusinessObject)} method will be invoked by the * {@link SynchronizableBusinessObjectImporter} when the imported record does not exist in Openbravo * yet. * * Note that those implementations that do not need to use the SynchronizableBusinessObject used for * the import operation, can just implement the {@link #initialize(Object)} method which is invoked * by the {@link #initialize(Object, SynchronizableBusinessObject)} method with its default * implementation. * * Classes that implement this interface should declare the {@link EntityMappingId} qualifier to * define the target SystemType and EntityMapping name */ public interface ImportedOBObjectInitializer<T> extends Prioritizable { /** * Initializes the properties that are not defined in the Entity Mapping in the AD. If it is * necessary to access to the SynchronizableBusinessObject used in the import operation, then * {@link #initialize(Object, SynchronizableBusinessObject)} should be implemented instead. * * @param bob * The entry to be initialized */ public default void initialize(T bob) { throw new NotImplementedException("No implementation found for any initialize method"); } /** * Initializes the properties that are not defined in the Entity Mapping in the AD. By default it * just invokes {@link #initialize(Object)}. * * @param bob * The entry to be initialized * @param sbo * The SynchronizableBusinessObject with all the data required by the import operation */ public default void initialize(T bob, SynchronizableBusinessObject sbo) { initialize(bob); } }
Classes implementing this interface must include the @EntityMappingId annotation, and must specify the ID of the entity mapping the initializer refers to. They can either implement the initialize(BaseOBObject bob) method or in case they need to access to the SynchronizableBusinessObject used for the import operation they can implement the initialize(BaseOBObject bob, SynchronizableBusinessObject sbo) method instead.
This initialization will be invoked by the SynchronizableBusinessObjectImporter before applying the mappings, so the values set in that method will be available by the JavaPropertyMappingHandlers.
Executing custom actions when the imported changes have been flushed
If some actions must be taken when importing a record right after the database triggers has been executed, the an implementation of the ImportedBaseOBObjectAfterFlushHook interface must be provided.
/** * Hook that is executed after importing a record, it will be invoked just after flushing the * changes to the database */ public interface ImportedBaseOBObjectAfterFlushHook extends Prioritizable { /** * afterFlush is executed right after the new/updated BaseOBObject has been imported in openbravo, * but before committing the transaction * * @param bob * The BaseOBObject that was just created/updated * @param item * the SynchronizableBusinessObject where the info for the new/updated record was taken * from */ public void afterFlush(BaseOBObject bob, SynchronizableBusinessObject item); }
Annotations for Java Components
All java components defined so far must include an annotation to specify the System Type they refer to (if they are system level hooks) or the entity mapping they refer to (for entitiy mapping level hooks).
For system-level hooks, the @ExternalSystemType("<SystemTypeSearchKey>") annotation must be used. The system level hooks available in the first version of the base connector infrastructure are:
- BooleanPropertyFormatter
- DatePropertyFormatter
- UUIDTransformer
- ImportedItemDeletionPolicy
For entity mapping-level hooks, the EntityMappingId("<EntityMappingId>") qualifier must be declared. The system level hooks available in the first version of the base connector infrastructure are:
- SynchronizableBusinessObjectExporter
- ImportJavaPropertyMappingHandler
- ExportJavaPropertyMappingHandler
- ImportedBaseOBObjectInitializer
- ImportedBaseOBObjectFetcher
- ImportedBaseOBObjectAfterFlushHook
Incremental Synchronization
As it has already been discussed, synchronization (both in the import and the export process) is done incrementally. The connector implementation will request the data pending to be synchronized and it will provide the entity whose instances must be synchronized, and the date of the previous synchronization.
The OBEI_Entity_Synchronization table keeps track of the last time that each entity mapping was synchronized. If an entity is going to be synchronized for the first time and there is no entry in OBEI_Entity_Synchronization, a new entry will be created with an old date, to force a full refresh.
If a full refresh is not needed and the synchronization must start from a given date, the OBEI_Entity_Synchronization table can be initialized using the Initialize Entity Synchronization Table process.
Synchronization Background Processes
This section explains how to define and schedule the import and export background processes.
Import Background Process
Specific connector implementations must first create an EDL Process. It is important to select Not Stored as Storage Location, to check the Has Input and Hash Output flags, and to uncheck the Synchronous flag.
For instance, this is the EDL Process defined for the import process of the SAP ECC connector:
The import background process must extend the ImportExternalSynchronizationProcess class, and implement the following methods:
/** * @return the ID of the EDL process to be used to synchronize the data */ protected abstract String getEDLProcessId(); /** * @return the search key of the source/target system of a data integration */ protected abstract String getSystemType(); /** * Returns a list of ImportRequestInfo, for each one of these an EDL Request to import data will * be created * * @param lastUpdate * the last time the entity being integrated was exported * @return a list of ExportRequestInfo that will be used to create EDL requests */ protected abstract ExternalDataIterator<RequestInfo> getRequestsInformation(String mappedEntity, Date lastUpdate);
The getRequestInformation method will be invoked periodically by the connector infrastructure to request the data that must be imported for a given mapped entity, that has been modified after a given date.
For instance, this is the implementation of the ImportExternalSynchronizationProcess for SAP ECC:
public class ImportSynchronizationProcess extends ImportExternalSynchronizationProcess { private static final String EDL_PROCESS_ID = "9859BD32C4D94BFE9E987B016E4E7D6F"; @Override protected String getSystemType() { return SapEccConstants.SYSTEM_TYPE; } @Override protected String getEDLProcessId() { return EDL_PROCESS_ID; } @Override protected ExternalDataIterator<RequestInfo> getRequestsInformation(String mappedEntity, Date lastUpdate) { ExternalDataIterator<RequestInfo> testRequestInformation = null; try { IdocSynchronizer iDocSynchronizer = IdocSynchronizerFactory.getIdocSynchronizer(); testRequestInformation = iDocSynchronizer.getIdocIterator(mappedEntity); } catch (IOException e) { throw new RuntimeException("Error while creating the ImportRequest for the EDL Process", e); } return testRequestInformation; } }
Export Backgroud Process
The implementation of the export background process is much simpler. The ExportExternalSynchronizationProcess class must be extended, and the getSystemType overwritten:
/** * @return the search key of the source/target system of a data integration */ protected abstract String getSystemType();
Then, the specific connector module must define a subclass of BaseSynchronizableBusinessObjectExporter, and implement the export method:
/** * Writes a list of {@link SynchronizableBusinessObject} using a format (XML, JSON, CSV, etc.) * chosen by the class implementing this interface. * * @param writer * A Writer where the list of SynchronizableBusinessObject will be written. * @param sbos * The list of {@link SynchronizableBusinessObject} to be written. */ public void export(Writer writer, List<SynchronizableBusinessObject> sbos) throws IOException;
As an example, this is an excerpt of the implementation of this method for the SAP ECC Connector:
@Override public final void export(Writer writer, List<SynchronizableBusinessObject> synchronizableObjects) throws IOException { for (SynchronizableBusinessObject sbo : synchronizableObjects) { String idocContents = iDocGenerator.generate(getIdocType(), sbo.getProperties()); String fileName = getFileName(); attachIdocToRecord(idocContents, fileName, sbo); IdocSynchronizer iDocSynchronizer = IdocSynchronizerFactory.getIdocSynchronizer(); iDocSynchronizer.export(fileName, toInputStream(idocContents)); writer.write("File " + fileName + " exported\n\n"); writer.write(idocContents); } }
Everything written in the provided writer will be stored as log in the EDL Request window (more on this in a later section).
How to Schedule the Background Processes
Once the import/export processes are defined, they can be scheduled in the Process Request window.
As any other process request, it is possible to specify how often the background process should be executed:
It is possible to trigger the synchronization for all defined entity mappings, in that case all will be exported/imported with the same periodicity. To do that, check the Integrate All Entities flag in the process request and do not add any record to the Entity to Integrate subtab.
In some cases it is useful to synchronize differente entitiy mappings with different periodicities.
There are two ways to synchronize only some entities in a process request:
- Synchronize all by default and manually exclude some entities. To do that, check the Integrate All Entities flag and then add the excluded entity mappings in the Entity to Integrate subtab, checking the flag Is Excluded.
- Do not synchronize any entity by default, manually include some entities. To do that, uncheck the Integrate All Entities flag and then add the included entity mappings in the Entity to Integrate subtab, unchecking the Is Excluded flag.
The EDL Request Window
The EDL Request Window keeps track of all the records that have been synchronized. Each time the import/export synchronization runs, an entry will be created in the EDL Request window per entity that had records pending to be synchronized.
The EDL infrastructure will then split the data to be synchronized in batches, and will process each batch in parallel, asynchronously. Each EDL Request Line will store the data being synchronized. It will keep track of the status of each request line:
- Initial if the request has not been processed yet.
- Processing if it is currently being processed.
- Success if the request has been processed sucessfully.
- Error if there has been an error while processing the request.
It is possible to manually change the EDL Request Line data and reprocess the request at any time.