Toolbox

The Openbravo Data Access Layer (DAL) provides functionality to automatically generate hibernate mapping and database schemas from the Application Dictionary. At runtime the Openbravo DAL provides a service layer which is used to retrieve, update and create business objects. The DAL takes care of standard security concepts (like organisation/client filtering) and standard update actions. The DAL also contains an automatic thread-based session and transaction handling system.

Model Driven Software Development - Domain Model Definition

The Openbravo DAL is based on model driven software development concepts. The domain model is defined in Openbravo itself and is the basis for generating different software artifacts.

Application Dictionary

The domain model is defined in the application dictionary. The application dictionary defines the tables and columns. In addition the application dictionary will define the mapping from the table and columns to java classes and fields. This mapping is used at runtime to generate hibernate mappings and to instantiate the correct java object (pojo) when retrieving information from the database.

Changes to Application Dictionary: to support the new DAL the application dictionary has to be adapted:

A table should have a field to define its entity name (the name used in hibernate queries).
For foreign-key relations the target table has to be defined in the column definiton.
The table should have a field for the java class implementing it.
A column should have a field to specify the name of the java member implementing it.
A column should have an additional field (yes/no) which defines if the column is implemented in a static (development time) java member or as a dynamic (map-like) feature. See dynamic model for more information.

At runtime the application dictionary is used to create an in-memory runtime model. This runtime model is used by the system to support dynamic models and for application tasks such as security and data import and export.

Dynamic Domain Model - Dynamic entities

The Openbravo DAL will allow the domain model to be changed at runtime. This means that at runtime the database schema and hibernate mapping needs to be re-generated. To support a dynamic domain model the system should at runtime be able to extend the existing java entity classes with dynamic features/attributes. In addition it should be possible to add new tables at runtime with the mapping from the table to a dynamic business object.

The support for dynamic models will be implemented as follows:

The DAL will support a generic java business object which has a map-like data structure to contain any set of attributes, including references.
Each statically (development time) java entity will inherit from this generic business object. This makes it possible to (at runtime) add new attributes/references to an existing java entity.

The dynamic structure is type-checked at runtime using the in-memory domain model. The hibernate mapping generation uses the column information to map the dynamic members in a different way than the static members.

The above implementation choices means that the DAL can consist of a mix of statically created java entity classes and dynamic generic objects. Statically (development-time) create java classes are less flexible than dynamic business objects. However, the main reason for using statically created java entity classes is that IDE support (e.g. code completion, refactoring) for statically created java classes is far better. Resulting in faster development and higher code quality. To combine flexibility and development efficiency, the DAL will support all variants from completely statically created java entities to completely dynamic business objects.

Extendability: factory approach

The DAL should be developed with extendability in mind.

The proposed changes to the application dictionary introduce runtime extendability/adaptebility. The application dictionary defines which java class is used for each table. This information can be changed at runtime to use a custom java class with custom behavior.

The DAL development can be used to introduce factory-based instantiation in other parts of the system. This makes it possible to extend/adapt the internals of Openbravo. From an implementation perspective there are two choices:

use a dependency injection framework like Spring or Guice
develop a custom/proprietary factory configuration mechanism

There are pros and cons for either of the two: using Spring/Guice is more standard but can be considered more 'heavy' from a configuration standpoint, using a custom approach is proprietary but probably more light-weight.

Generated Artifacts

The domain model definition in the application dictionary can be used to generate the following software artifacts:

Database schema (initial and incremental updates)
Hibernate mapping (initial and incremental updates)
Java entity classes

Database schema generation

The database schema generation is currently available but needs to be adapted:

the logic needs to be re-structured to operate in a running system without restart.
not all technical information (required for db schema generation) is currently available in the application dictionary. This information has to be added (possibly in separate tables).

Hibernate Mapping generation

The application dictionary is the basis for the hibernate mapping. The application dictionary contains the mapping from the relational schema to the java classes/members. This information can be used directly to generate the hibernate mapping. The system will work with an in-memory hibernate mapping which is generated when the application initializes or when the DAL is re-configured (see next topic).

(Initial) Generation of Static Java Entity Classes

Openbravo currently has about 425 tables. These tables translate to about 425 Java entity classes. Instead of manually creating these classes it can make sense to generate the initial set of Java entity classes. The next step is to make this generation step a core-function in the Openbravo development environment. This can facilitate larger development efforts.

If a generative approach is chosen for the entity pojos then the proposal is to use the XPand templating language for java code generation. XPand is part of the Eclipse Model-To-Text project and is very popular. For more information see here: http://www.openarchitectureware.org/.

Dynamic Regeneration

The system should support update of the database schema and hibernate mapping at runtime. There should be a separate 'refresh/reconfigure' option which the administrator can run. This action should do the following:

Update the database schema
Update the in-memory hibernate mapping
Re-initialize the hibernate session factory

Dynamic update of the database schema and in-memory hibernate mapping is straightforward to develop. There is however one topic which requires extra attention, nl. a database schema update normally/often requires a database lock. So during a database schema update it is not really possible for other users/programs to access the system. The system should support a blocking mechanism to temporarily block access to the system until a database schema update has been performed.

DAL Service Layer

The Openbravo DAL will offer its functionality through a service layer. The service layer hides the internals of the DAL for (application) users of the DAL. The service layer will offer the following functionality:

Retrieve business objects using a query api with a filter mechanism
Update/create business objects
Remove business objects

The filter mechanism should specifically support filtering on:

Organisation and Client
Active yes/no
Page location and page size for support of paginated views
Sorting on multiple columns

Openbravo Context: OBContext

The DAL requires a context concept to be aware of the environment it currently runs in. This context concept is called the OBContext.

The OBContext contains the following information:

The current user.
The user organisation and client.
The user language.
The accessible organisations (used to read information from the database).
The client and organisation list for which the user may update information.

The OBContext is stored in a ThreadLocal during the thread, and between threads it is stored in a http session variable.

When a request starts the OBContext is retrieved from the http session. If it is not there then a new OBContext is created. The next step is that the information in the http session is used to set the information in the OBContext. This is done for every request, only the changed information is updated.

After the OBContext has been initialized it is placed in a ThreadLocal.

The application code can always retrieve the OBContext by calling the method OBContext.getOBContext().

Session/transaction handling

The dal uses the open session in view pattern. The 'open session in view' pattern is implemented using a servlet filter and a session handler class:

The servlet filter is the same filter instance as used for the OBContext, the transaction commit/rollback is done at the end of the request (see below).
The SessionHandler class creates a Session and begins a transaction when a thread requires it for the first time. The SessionHandler class also support commit/rollback of transactions and other transaction specific logic.

As mentioned above, the current implementation performs the commit of the transaction at the end of the request.

Hibernate performs many actions at commit time (for example flush sql statements to the database). This means that in a late stage of the request/response cycle, errors can still occur. A better concept is to use two transactions within one request/response cycle (this approach is implemented by Seam). This approach can be considered in a future version when also the UI layer of Openbravo is redesigned.

Model: Common Object Features/Class Hierarchy

The DAL has a java pojo for each business object in the Openbravo application. A business object corresponds to a table in the current Openbravo system. Many Openbravo business objects have a number of features in common:

A single id field (numeric or string, mostly numeric/long)
A set of fields for audit tracking (updated, updatedBy, created, createdBy)
A link to a Client and an Organisation
A name and a description
Translatable version (see section on translatable data below)

The idea is to design a class hierarchy which implements part of the features at a higher level than the direct java entity. The class hierarchy should take care of the implementation while the actual definition of the concept is done through interfaces. For example the following interfaces can be envisioned:

Traceable: for objects that have audit data
ClientOrganisationEnabled: for objects that are defined by Client and Organisation
Identifiable: objects that have an identifier

A first proposal for a class hierarchy (implemented in the prototype):

BaseOBObject: has an id and implements Traceable
GenericOBObject: inherit from BaseOBObject and add name/description
CommonOBObject: inherit from GenericOBObject and implement ClientOrganisationEnabled

The java entities themselves should be organised in packages which corresponds to the current table naming/division. The prototype follows this approach:

org.openbravo.dal.model.ad: contains Client, Organisation, Language, User, Role etc.
org.openbravo.dal.model.c: contains types such as Currency, etc.
org.openbravo.dal.model.m: contains types such as Product, Transaction, etc.
....

Security

The DAL should support standard security checking (for example client/organisation) for update, removal and retrieving of information. The security logic can be implemented in two distinct locations:

In the DAL service layer: the code which implements the DAL api performs security checks
In the database layer, in Hibernate: in this case the security code is implemented in pre-save and pre-update listeners

The second choice has as advantage that it is ensured that all updates through hibernate are security checked. On the other hand the disadvantage is that there is certain system information which is updated as part of a user action (for example update log or audit table). Directly updating this information by the user is not allowed. For the save/update event listeners it is not possible to distinguish between a direct user update of information (the user directed the system to update an object in the db) or an update which is driven by the system itself (as the result of a user update of information).

Performing security checks in the DAL service layer has as advantage that the security is checked on the objects which the user wants to update directly. So the security check is done in the layer which knows the action taken by the user. On the other hand the system is 'less' secure because it is possible to update information through hibernate even when the user does not have enough authorisation. For example Hibernate will automatically update related objects if an object is saved. A security check in the DAL layer can not catch/check/prevent the update of these related object.

The proposal is to support security checking in the DAL service layer. The rest of the application (in lower levels) should not be worried about security. In addition the cascade update behavior described in the previous paragraph is less relevant for Openbravo because most (if not all) relations are many-to-one relations for which update-cascades should be disabled anyway.

Testability

The DAL should be testable directly and separately from the main Openbravo application using the DAL service layer. The DAL testcases are implemented in a separate development: OpenbravoTest.

The OBContext and transaction/session handling implementation facilitates testing by making it easier to setup a environment in which to run the testcases. The OBContext completely defines the environment of the DAL and the session/transaction handling is transparent for a testrun.

Data Access Layer Hibernate Prototype Specifics

Prototype Software Package Structure

The data access layer (dal) is implemented in the org.openbravo.dal package in the main Openbravo development project (inside the src source directory). The dal consists of a number of subpackages:

core: this package contains classes for thread handling, request filter, session factory creation.
exception: contains common exceptions (which logs themselves) used in the dal layer.
model: this package contains the entity beans for the tables in the current Openbravo system. In addition the hibernate mapping can be found here.
security: contains code for checking security.
service: contains the external Dal interface.
util: some utility classes.

Dal Service Layer: filtering, querying, paging and sorting

The Dal service layer is implemented in the org.openbravo.dal.service package. It consists of three classes:

OBDal: this is the external interface offered to application software components. The OBDal offers an API to save/create, remove, query and count business objects.
OBFilter: filter/search criteria (and aspects such as paging and order by) are handled through the OBFilter class. The OBDal API expects an instance of OBFilter as a parameter to the count and query service.
OBFilterQuery: internal class of the Dal layer used to create the actual query.

The OBDal layer internally passes the OBFilter instance to the OBFilterQuery class to create a filtered query. The OBDal layer will automatically also add organisation and client filtering (for security reasons).

DalThreadHandler/DalRequestFilter

The DalThreadHandler and DalRequestFilter classes are implemented in the org.openbravo.dal.core packages. These classes ensure that when a thread runs inside the Openbravo business layer it has always an Openbravo context object available and the thread has access to a Hibernate Session.

Threadcontext Pattern

The DAL makes use of ThreadLocals to keep state throughout the lifetime of a thread. The thread concept makes use of the ThreadHandler class (see the org.openbravo.dal.core package in the prototype). This class implements a common pattern to run the actual logic within a try/finally block and perform initialize and clean up actions when the thread returns.

In a servlet environment it's particularly important to clean up the thread context at the end because threads are re-used by the Tomcat servlet container.

ORM Topics: Mapping, identifier

Mapping

The business objects are mapped to Hibernate using a separate mapping file per type. The mapping file can be found in the same class as the business objects. Although the business objects are modeled in a class hierarchy, it is not possible to follow the same structure in the mapping file. This is because the id is stored in a different column for each table, for example the id of the currency is stored in the currency_id column, the id of the user is stored in the user_id field. If all types would have used the same column name then it would have been possible to map certain properties at superclass level.

A few specific remarks regarding the mapping:

The mapping contains postgresql specific column types, so probably to support different databases, different mapping files need to be created.
The id generation uses a custom id-generation strategy (to support the Openbravo identifier logic).
All classes are mapped as lazy allowing cglib proxying.
All classes are mapped with cache usage read-write (to enable second-level caching).
Many-to-one relations are mapped with the lazy attribute not set, this defaults to proxying.

Identifier Creation

Openbravo controls the id-ranges of the instances of business objects through the ad_sequence table. This table stores, for each type, the next available id. When a new instance is created then the id is retrieved from this table and the ad_sequence table is updated. Before this logic was implemented in a stored procedure.

When using Hibernate this logic has to be implemented in a separate IdentifierGenerator class. The dal implements this in the OBIdentifierGenerator class (org.openbravo.dal.core). The id retrieval and update is done in a separate transaction. So no stored procedure is used. The OBIdentifierGenerator is used in the mapping of each business object.

Initialization and Configuration (Second-Level caching)

Hibernate is initialized when the DalRequestFilter object is initialized. Hibernate reads the hibernate.properties in the root of the classpath (during development). In a later version the hibernate.properties will be placed in the WEB-INF directory. The Hibernate SessionFactory is managed by the SessionFactoryController class (org.openbravo.dal.core) class. The classes which are persisted through Hibernate are added to Hibernate in this class.

The dal will use ehcache as the second level cache provider. The configuration of the second level cache is done through the ehcache.xml file in the root of the classpath (see the src directory in the Openbravo project).

Update/Save Event Handling

Hibernate makes it possible to add event handling logic when an object is saved, loaded or deleted. The dal adds specific behavior at save (=new object) and update (=existing object). This behavior is implemented in the OBEventListener class (org.openbravo.dal.core). This class implements the following behavior:

When a new object is saved then the created/updated audit fields are set (using the user from the OBContext)
When an existing object is saved then the updated audit fields are set
A security check is done when an object is saved or created

The security check is implemented in the org.openbravo.dal.security.SecurityChecker class.

Process versus View Objects

The dal requirements document makes a distinction between process and view objects. Process objects only contain primitive data (string, integer, etc.) and no references to other types. View objects on the other hand have both the primitive data and have resolved references to other types. The idea behind the distinction between the process and view object is that retrieving process objects is faster because no references to other types need to be resolved (resulting in less joins and extra queries).

Hibernate in combination with lazy-loaded proxies makes it possible to remove the distinction between a process and a view object. When an object is retrieved from the database then its references are by default not resolved (it is basically a process object). When its references are accessed then Hibernate will automatically resolve them (resulting in extra queries or cache hits), supporting the view object requirement.

Future Topics

Translatable Data

Some of the tables in Openbravo have a separate translation table which holds the translated content for several languages. For example the ad_country table has a ad_country_trl table which holds the translated country names. The translated information is used in various locations (to fill listboxes for example).

The dal should provide a standardized way to retrieve the translated version of a translatable object.

The current idea is to introduce a separate interface translatable with a method: getTranslated() which returns the translated version of the object (using the current user's language). The translatable business objects (such as currency and country) should implement this method to return a translated version of themselves. The Translatable interface can be used by business functionality to determine if an object is translatable and use the translated version if required.

The getTranslated method will use Hibernate to retrieve the translated version. For performance reasons the second level cache should be initialized for these types of objects. As translatable objects will not change that much it can be safely assumed that they will be present in the second-level cache and remain there.

Project Plan: main development steps (TBD)

The main development topics and their logical ordering.

Add new fields to application dictionary
Re-configuration program: incorporate database schema updates into Openbravo system
Implement runtime model which reads the application dictionary and creates a runtime model representation
Implement hibernate mapping generation on the basis of the runtime model
Integrate hibernate configuration in application initialization and database re-configuration
Design class hierarchy for business objects
Create templates for generating standard java pojo entity classes
Design/Define DAL api and logic
Create full-set of DAL testcases
incorporate DAL code in current generated servlet code

Retrieved from "http://wiki.openbravo.com/wiki/Projects:Data_Access_Layer/Technical_Design"

This page has been accessed 8,767 times. This page was last modified on 8 June 2012, at 05:27. Content is available under Creative Commons Attribution-ShareAlike 2.5 Spain License.