Log in / create account
Openbravo.com
Issues
Forge
View source | View content page | Page history | Printable version   

Projects:Data Access Layer/Technical Design

Contents

Overview

This page describes the main components of the Openbravo data access layer.

The Openbravo Data Access Layer (DAL) provides functionality to automatically generate hibernate mapping and database schemas from the Application Dictionary. At runtime the Openbravo DAL provides a service layer which is used to retrieve, update and create business objects. The DAL takes care of standard security concepts (like organisation/client filtering) and standard update actions. The DAL also contains an automatic thread-based session and transaction handling system.

Model Driven Software Development - Domain Model Definition

The Openbravo DAL is based on model driven software development concepts. The domain model is defined in Openbravo itself and is the basis for generating different software artifacts.

Application Dictionary

The domain model is defined in the application dictionary. The application dictionary defines the tables and columns. In addition the application dictionary will define the mapping from the table and columns to java classes and fields. This mapping is used at runtime to generate hibernate mappings and to instantiate the correct java object (pojo) when retrieving information from the database.

Changes to Application Dictionary: to support the new DAL the application dictionary has to be adapted:

At runtime the application dictionary is used to create an in-memory runtime model. This runtime model is used by the system to support dynamic models and for application tasks such as security and data import and export.

Dynamic Domain Model - Dynamic entities

The Openbravo DAL will allow the domain model to be changed at runtime. This means that at runtime the database schema and hibernate mapping needs to be re-generated. To support a dynamic domain model the system should at runtime be able to extend the existing java entity classes with dynamic features/attributes. In addition it should be possible to add new tables at runtime with the mapping from the table to a dynamic business object.

The support for dynamic models will be implemented as follows:

The dynamic structure is type-checked at runtime using the in-memory domain model. The hibernate mapping generation uses the column information to map the dynamic members in a different way than the static members.

The above implementation choices means that the DAL can consist of a mix of statically created java entity classes and dynamic generic objects. Statically (development-time) create java classes are less flexible than dynamic business objects. However, the main reason for using statically created java entity classes is that IDE support (e.g. code completion, refactoring) for statically created java classes is far better. Resulting in faster development and higher code quality. To combine flexibility and development efficiency, the DAL will support all variants from completely statically created java entities to completely dynamic business objects.

Extendability: factory approach

The DAL should be developed with extendability in mind.

The proposed changes to the application dictionary introduce runtime extendability/adaptebility. The application dictionary defines which java class is used for each table. This information can be changed at runtime to use a custom java class with custom behavior.

The DAL development can be used to introduce factory-based instantiation in other parts of the system. This makes it possible to extend/adapt the internals of Openbravo. From an implementation perspective there are two choices:

There are pros and cons for either of the two: using Spring/Guice is more standard but can be considered more 'heavy' from a configuration standpoint, using a custom approach is proprietary but probably more light-weight.

Generated Artifacts

The domain model definition in the application dictionary can be used to generate the following software artifacts:

Database schema generation

The database schema generation is currently available but needs to be adapted:

Hibernate Mapping generation

The application dictionary is the basis for the hibernate mapping. The application dictionary contains the mapping from the relational schema to the java classes/members. This information can be used directly to generate the hibernate mapping. The system will work with an in-memory hibernate mapping which is generated when the application initializes or when the DAL is re-configured (see next topic).

(Initial) Generation of Static Java Entity Classes

Openbravo currently has about 425 tables. These tables translate to about 425 Java entity classes. Instead of manually creating these classes it can make sense to generate the initial set of Java entity classes. The next step is to make this generation step a core-function in the Openbravo development environment. This can facilitate larger development efforts.

If a generative approach is chosen for the entity pojos then the proposal is to use the XPand templating language for java code generation. XPand is part of the Eclipse Model-To-Text project and is very popular. For more information see here: http://www.openarchitectureware.org/.

Dynamic Regeneration

The system should support update of the database schema and hibernate mapping at runtime. There should be a separate 'refresh/reconfigure' option which the administrator can run. This action should do the following:

Dynamic update of the database schema and in-memory hibernate mapping is straightforward to develop. There is however one topic which requires extra attention, nl. a database schema update normally/often requires a database lock. So during a database schema update it is not really possible for other users/programs to access the system. The system should support a blocking mechanism to temporarily block access to the system until a database schema update has been performed.

DAL Service Layer

The Openbravo DAL will offer its functionality through a service layer. The service layer hides the internals of the DAL for (application) users of the DAL. The service layer will offer the following functionality:

The filter mechanism should specifically support filtering on:

Openbravo Context: OBContext

The DAL requires a context concept to be aware of the environment it currently runs in. This context concept is called the OBContext.

The OBContext contains the following information:

The OBContext is stored in a ThreadLocal during the thread, and between threads it is stored in a http session variable.

When a request starts the OBContext is retrieved from the http session. If it is not there then a new OBContext is created. The next step is that the information in the http session is used to set the information in the OBContext. This is done for every request, only the changed information is updated.

After the OBContext has been initialized it is placed in a ThreadLocal.

The application code can always retrieve the OBContext by calling the method OBContext.getOBContext().

Session/transaction handling

The dal uses the open session in view pattern. The 'open session in view' pattern is implemented using a servlet filter and a session handler class:

As mentioned above, the current implementation performs the commit of the transaction at the end of the request.

Hibernate performs many actions at commit time (for example flush sql statements to the database). This means that in a late stage of the request/response cycle, errors can still occur. A better concept is to use two transactions within one request/response cycle (this approach is implemented by Seam). This approach can be considered in a future version when also the UI layer of Openbravo is redesigned.

Model: Common Object Features/Class Hierarchy

The DAL has a java pojo for each business object in the Openbravo application. A business object corresponds to a table in the current Openbravo system. Many Openbravo business objects have a number of features in common:

The idea is to design a class hierarchy which implements part of the features at a higher level than the direct java entity. The class hierarchy should take care of the implementation while the actual definition of the concept is done through interfaces. For example the following interfaces can be envisioned:

A first proposal for a class hierarchy (implemented in the prototype):

The java entities themselves should be organised in packages which corresponds to the current table naming/division. The prototype follows this approach:

Security

The DAL should support standard security checking (for example client/organisation) for update, removal and retrieving of information. The security logic can be implemented in two distinct locations:

The second choice has as advantage that it is ensured that all updates through hibernate are security checked. On the other hand the disadvantage is that there is certain system information which is updated as part of a user action (for example update log or audit table). Directly updating this information by the user is not allowed. For the save/update event listeners it is not possible to distinguish between a direct user update of information (the user directed the system to update an object in the db) or an update which is driven by the system itself (as the result of a user update of information).

Performing security checks in the DAL service layer has as advantage that the security is checked on the objects which the user wants to update directly. So the security check is done in the layer which knows the action taken by the user. On the other hand the system is 'less' secure because it is possible to update information through hibernate even when the user does not have enough authorisation. For example Hibernate will automatically update related objects if an object is saved. A security check in the DAL layer can not catch/check/prevent the update of these related object.

The proposal is to support security checking in the DAL service layer. The rest of the application (in lower levels) should not be worried about security. In addition the cascade update behavior described in the previous paragraph is less relevant for Openbravo because most (if not all) relations are many-to-one relations for which update-cascades should be disabled anyway.

Testability

The DAL should be testable directly and separately from the main Openbravo application using the DAL service layer. The DAL testcases are implemented in a separate development: OpenbravoTest.

The OBContext and transaction/session handling implementation facilitates testing by making it easier to setup a environment in which to run the testcases. The OBContext completely defines the environment of the DAL and the session/transaction handling is transparent for a testrun.

Data Access Layer Hibernate Prototype Specifics

Prototype Software Package Structure

The data access layer (dal) is implemented in the org.openbravo.dal package in the main Openbravo development project (inside the src source directory). The dal consists of a number of subpackages:

Dal Service Layer: filtering, querying, paging and sorting

The Dal service layer is implemented in the org.openbravo.dal.service package. It consists of three classes:

The OBDal layer internally passes the OBFilter instance to the OBFilterQuery class to create a filtered query. The OBDal layer will automatically also add organisation and client filtering (for security reasons).

DalThreadHandler/DalRequestFilter

The DalThreadHandler and DalRequestFilter classes are implemented in the org.openbravo.dal.core packages. These classes ensure that when a thread runs inside the Openbravo business layer it has always an Openbravo context object available and the thread has access to a Hibernate Session.

Threadcontext Pattern

The DAL makes use of ThreadLocals to keep state throughout the lifetime of a thread. The thread concept makes use of the ThreadHandler class (see the org.openbravo.dal.core package in the prototype). This class implements a common pattern to run the actual logic within a try/finally block and perform initialize and clean up actions when the thread returns.

In a servlet environment it's particularly important to clean up the thread context at the end because threads are re-used by the Tomcat servlet container.

ORM Topics: Mapping, identifier

Mapping

The business objects are mapped to Hibernate using a separate mapping file per type. The mapping file can be found in the same class as the business objects. Although the business objects are modeled in a class hierarchy, it is not possible to follow the same structure in the mapping file. This is because the id is stored in a different column for each table, for example the id of the currency is stored in the currency_id column, the id of the user is stored in the user_id field. If all types would have used the same column name then it would have been possible to map certain properties at superclass level.

A few specific remarks regarding the mapping:

Identifier Creation

Openbravo controls the id-ranges of the instances of business objects through the ad_sequence table. This table stores, for each type, the next available id. When a new instance is created then the id is retrieved from this table and the ad_sequence table is updated. Before this logic was implemented in a stored procedure.

When using Hibernate this logic has to be implemented in a separate IdentifierGenerator class. The dal implements this in the OBIdentifierGenerator class (org.openbravo.dal.core). The id retrieval and update is done in a separate transaction. So no stored procedure is used. The OBIdentifierGenerator is used in the mapping of each business object.

Initialization and Configuration (Second-Level caching)

Hibernate is initialized when the DalRequestFilter object is initialized. Hibernate reads the hibernate.properties in the root of the classpath (during development). In a later version the hibernate.properties will be placed in the WEB-INF directory. The Hibernate SessionFactory is managed by the SessionFactoryController class (org.openbravo.dal.core) class. The classes which are persisted through Hibernate are added to Hibernate in this class.

The dal will use ehcache as the second level cache provider. The configuration of the second level cache is done through the ehcache.xml file in the root of the classpath (see the src directory in the Openbravo project).

Update/Save Event Handling

Hibernate makes it possible to add event handling logic when an object is saved, loaded or deleted. The dal adds specific behavior at save (=new object) and update (=existing object). This behavior is implemented in the OBEventListener class (org.openbravo.dal.core). This class implements the following behavior:

The security check is implemented in the org.openbravo.dal.security.SecurityChecker class.

Process versus View Objects

The dal requirements document makes a distinction between process and view objects. Process objects only contain primitive data (string, integer, etc.) and no references to other types. View objects on the other hand have both the primitive data and have resolved references to other types. The idea behind the distinction between the process and view object is that retrieving process objects is faster because no references to other types need to be resolved (resulting in less joins and extra queries).

Hibernate in combination with lazy-loaded proxies makes it possible to remove the distinction between a process and a view object. When an object is retrieved from the database then its references are by default not resolved (it is basically a process object). When its references are accessed then Hibernate will automatically resolve them (resulting in extra queries or cache hits), supporting the view object requirement.

Future Topics

Translatable Data

Some of the tables in Openbravo have a separate translation table which holds the translated content for several languages. For example the ad_country table has a ad_country_trl table which holds the translated country names. The translated information is used in various locations (to fill listboxes for example).

The dal should provide a standardized way to retrieve the translated version of a translatable object.

The current idea is to introduce a separate interface translatable with a method: getTranslated() which returns the translated version of the object (using the current user's language). The translatable business objects (such as currency and country) should implement this method to return a translated version of themselves. The Translatable interface can be used by business functionality to determine if an object is translatable and use the translated version if required.

The getTranslated method will use Hibernate to retrieve the translated version. For performance reasons the second level cache should be initialized for these types of objects. As translatable objects will not change that much it can be safely assumed that they will be present in the second-level cache and remain there.

Project Plan: main development steps (TBD)

The main development topics and their logical ordering.

Retrieved from "http://wiki.openbravo.com/wiki/Projects:Data_Access_Layer/Technical_Design"

This page has been accessed 8,977 times. This page was last modified on 8 June 2012, at 05:27. Content is available under Creative Commons Attribution-ShareAlike 2.5 Spain License.