Projects:UI Technology/UserInterface Modules Definition

Contents 1 Introduction 2 Kernel Module 2.1 Different Templates for different Components 2.2 Template overriding 2.3 Supporting different Templating languages 2.4 Generating a component, implementing a Component Provider 2.5 Module Extendability 2.6 Data Model 2.7 Validation and Compression 2.8 Caching of Components/Javascript 2.8.1 Caching and refreshing of static js files 2.8.2 Caching and refreshing of components 2.8.3 Information regarding HTTP caching 3 JSON REST Module 4 Datasource Module 4.1 Data source two implementations 4.2 Additional data source fields 4.3 Datasource REST api 4.4 Data Model 5 Smartclient Module 6 Freemarker Template Module 7 Selector Module 7.1 Grid Component 7.2 Data Model 7.3 Defining Selectors 8 Model Property Selector 8.1 Implementing the datasource 8.2 Application Dictionary Setup 8.3 Testing 9 Developer or Consultant? 10 To be done (design and implementation)

Introduction

This page contains a functional description and design of the main modules which implement the new Openbravo user interface. The description covers both modules implementing core functionality as well as 'prototype' modules (e.g. the selector).

The new user interface for Openbravo will be implemented using different modules. Each module provides a unique part of the functionality. The choice has been made to split the functionality in different smaller modules because:

• facilitate extendability: functionality or specific implementations can be added by other modules. For example for templating, the system will support pluggable templating languages. To support this from the start the initial templating language is included as a module.
• modules can be upgraded/released independently from eachother: this makes it easier to solve issues for individual modules and more gradually extend the functionality over time.
• using modules and explicit module dependencies ensures that the overall architecture is sound and correct

The drawback of using multiple smaller modules is that this requires a larger install effort initially and that relationships between modules can be complex. The advantages outlined above however, far outweigh this disadvantage.

This document often uses the terms component and component type. To clarify, a component is a distinct piece of functionality which is generated to be used in the client (the browser). A component can have a user interface (like an editable grid) or not (like java script to create a data source on the client). The selector described in this document is for example a component, but also the data source is a component 'rendered' on the client.

Kernel Module

The Kernel module is the core of the new user interface architecture. Its responsibilities:

• serve (javascript) components to the client (browser)
• caching/compression/compiling of components
• generation of components using templates

The main concept of the kernel module is the template. Templates are explicitly defined in the module.

Template definition and creation is a technical task done mostly by programmers.

Different Templates for different Components

Modules are able to add new component types and add new templates to visualize existing component types. The system should be able to visualize one type of component in different ways. To support this, a template is valid for a certain component type and for one component type there can be several templates.

The component type is a new list AD_REFERENCE implemented in the Kernel module. Other modules can add new values to this list AD_REFERENCE.

Template overriding

The system makes it possible for a module to override/replace another template without actually overwriting the other template. This explicit overriding ensures that the original template is still present and can be upgraded (for bug fixes for example).

Templates can only override other templates which have the same component type.

Template overriding is implemented through a foreign key from a template to the template which is overridden. This data model is simple but has as one drawback that two separate modules can override the same template. The system should log a warning in this case and make an arbitrary choice which template to use.

Supporting different Templating languages

The system should support different templating languages from the start. Initially all templates delivered by Openbravo are implemented in freemarker. Freemarker is a mature and easy-to-use templating language.

To support several templating languages, a new AD_REFERENCE (list) is added: template language. A module providing a template language needs to do the following:

• Add a new value to this list AD_REFERENCE
• Register a template processor in a central template processor registry (provided by the kernel module) for the new list AD_REFERENCE

Generating a component, implementing a Component Provider

This section describes which steps are involved in generating a component. A component is delivered by a module (see the selector module below). To deliver a component a module must implement the following:

• add a new value to the component type AD_REFERENCE
• add an implementation of the ComponentProvider interface, this is an interface called by the kernel module to retrieve a component
• register the component provider in the kernel using a unique name

A request for a component then goes through the following steps:

• the kernel receives the request, the request has a url of the form: /kernel/[component_provider_name]/[component_specific_part]. The /kernel/ part is just an example, the kernel module can register another url pattern also.
• the kernel receives the request, gets the component provider using the component_provider_name
• the component provider is called to generate the response (a java script string)
• the component provider uses the component_specific_part and the parameters to generate the response
• the response is returned to the client

The kernel will implement a default component provider which uses the template definition to generate a response.

The component provider should also implement the following functions/api:

• global resources: returns the resources which should be added to each page
• order: a unique number which sorts the declaration of global resources in a page
• current version: a unique id which defines the latest version, if the component (module) detects a change then the version id should be regenerated to a new version number. The version id is used to generate a unique url in the page so that caching is either encouraged (if the version did not change) or is prevented (if the version changed).

Module Extendability

A module can make use of the described extendability in different ways:

• add a new templating language
• add a new template for an existing component type
• add a new component type (for example a special-popup-editable-grid), a visualization template and a table to define the new component
• add a new template which overrides another template.

Data Model

The COMPONENT_TEMPLATE table models the template per component type concept. Having multiple templates for one type of component (for example an editable grid), makes it possible to support different visualizations for one component type.

The COMPONENT_TEMPLATE table has the following main columns:

• name and description
• component type: this ensures that the template is only used for specific component types (it does not make sense to use a template for an editable grid to visualize a selector).
• overrides template: a nullable foreign key to a template which is overridden.
• template language: defines the templating language and the template processor used to render the template
• template: the template itself, either as a path in the system or stored in the database

(In addition this table has client/organization and audit info fields).

Validation and Compression

The generated javascript is validated using jslint. For the integration with java the jslint4java library is used. Note that an older version of this library is used (version 1.2.1) because the newest jslint4java include Rhino which clashes with the YUI Compressor which also includes Rhino (a different version).

The compression is done before the validation. The compression is done using the YUI Compressor. As noted above the YUI Compressor also ships with Rhino included and even replaces certain Rhino classes. This is the cause for the conflict with newer versions of jslint.

Errors and warnings during validation and compression are logged.

Caching of Components/Javascript

To improve the user experience and performance it is important to make use of caching functionality in the browser as well as on the server. For caching a distinction can be made between components and static js files. Components are generated on the basis of definitions in the database and can contain language specific strings. Static js files define javascript widgets which are re-used by components.

Caching and refreshing of static js files

A static js file contains a library or a standard widget which is used by components.

The link to a static js file is created when the application starts and is generated in the 'top' of the page. Static js files are loaded once during the user session.

Static js files are assumed to be cached in the client by the browser. Static js files may change during upgrades or development. To enforce reload of a static js file it is mandatory that a static js file link contains an explicit version tag. This tag can be a version number or other string, the only requirement is that it is unique. The logic which generates the hyperlinks to the static resources should be aware of this unique tag and make sure that the links to the static resources contain this unique tag.

The following is proposed to implement this requirement: A static js file is always provided by a module. A module publishes its static resources to the kernel module. The kernel module takes care of gathering all static resources (in order of the module dependencies) and generates a set of script tags which are placed in the top of the main html file. The kernel module appends a version parameter to each static js link. For example if a module publishes this resource:

org.openbravo.userinterface.selector/js/OB_Selector_Widget.js

and the module currently has version 1.0.0alpha (version id and version tag combined) then the eventual script tag will be:

<script src='../../org.openbravo.userinterface.selector/js/OB_Selector_Widget.js?version=1.0.0alpha"/>

This ensures that with upgrades of a modules also the client side libraries are refreshed.

A distinction is made for developers mode. Developers mode is enabled when at least one module has inDevelopement set to Y. In this case the url will contain the latest time millis (System.currentTimeMillis):

<script src='../../org.openbravo.userinterface.selector/js/OB_Selector_Widget.js?version=129123123123"/>

This ensures that when developing nothing gets cached.

Caching and refreshing of components

Components are considered to be dynamic and contain runtime data read from databases. Components are generated on request and cached on the server. The server side can validate if a component has changed since the last request. This validation is not possible on the client as data on the server may have changed.

To support the concept of server side validation the idea is to make use of the ETag concept. An etag is like a hashcode which is used to determine if content has changed.

The etag is used as follows:

1. a request is received
2. the content is generated and the etag is computed
3. the content and etag are send back in the response
4. a request with an etag is received
5. using the request information the etag is computed and compared with the etag received from the request
6. if the etags are different the content is generated and send back with the new etag
7. if the etags are the same then a not-modified response is send back (without generating the content)

When using etags the savings are two-fold:

• depending on the etag-computation-logic the response content does not need to be generated
• the not-modified response is very small and saves bandwith compared to sending back the content itsefl

There are different approaches to generating an etag:

• generate the content (for example the javascript) and compute a hash (MD5) on the javascript string
• use last modified information from the database

The computation of an etag is left to the component implementation. One specific thing is that if the component contains language-specific information (like translated labels) then the etag needs to encode the current language of the user.

To make etag computation more efficient it can make sense to distinguish between different parts of a component and their volatility, i.e. if they change often at runtime or at development time. Runtime is when the application is live and used by end-users. Development time is when developers or consultants are developing. The mode (runtime or development time) is detected by checking the value of the isInDevelopment column of a module, if there is at least one module with isInDevelopment set to Y then the system is considered to be in development mode.

A component can consist of many parts. For example the selector depends on the column, the reference, the selector and selector field definitions and on translated labels. For each of these parts a different change volatitily can be distinguished:

• labels: label translations will almost never change at runtime (without restarting the application) and seldom at development time. Note that reading labels for column/fields can take a considerable amount of time because it can result in individual queries for each column/field.
• Application Dictionary Definition: column and reference definitions will not change at runtime (without restarting the application) and only limited changes are done at development time (without restarting the application).
• User Interface Definition: the definition of the selector field for example. At runtime this will not change much but it can change, at development time this definition will change frequently.

This definition can be used to define a more efficient etag computation for a component:

• components are always regenerated after an application restart. So part of the etag will contain the start time of the application (see below for a variant). This makes sure that components are regenerated after changes in the application dictionary and changes in internationalization.
• components are generated when their direct definition (for example the selector and selector field definition) changes, so in this case the last modified of the related database records needs to be used to compute an etag.

The above proposes to use the start time of the application for part of the etag. Instead, for a live production application it can make sense to use the version information stored in modules instead of the application start time. This version information is more stable than application start time and will result in more cache-hits.

Information regarding HTTP caching

• http://www.infoq.com/articles/etags
• http://www.oreillynet.com/onjava/blog/2004/07/optimizing_http_downloads_in_j.html
• http://www.xml.com/pub/a/2006/02/01/doing-http-caching-right-introducing-httplib2.html
• http://bitworking.org/news/150/REST-Tip-Deep-etags-give-you-more-benefits
• http://blogs.atlassian.com/developer/2007/12/cachecontrol_nostore_considere.html
• http://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html#sec13

JSON REST Module

The JSON REST module provides JSON access to database using a REST-like api. The JSON module is an independent module from the other modules in the user interface architecture. It is used by the data source module.

Datasource Module

The data source module makes it possible to define data sources in the system. A data source is responsible for providing the client with data from the database using REST-like requests. Data sources have a runtime (query and update) behavior and an initialization time behavior (generate javascript to create the data source on the client). The query and update logic is implemented using the JSON module, the initialization time behavior uses the freemarker template module and the kernel module.

Simple data sources can be defined by consultants, more complex data sources require a java implementation which is provided by a programmer.

Data source two implementations

Data sources exist in different forms:

• they can either be derived directly from a table with an additional where clause to limit the selection, or
• they can be implemented completely in a custom implementation

The first type of data source is not created explicitly in the system but is available automatically for each entity in the system. The name of the data source is the entity name, there is a BusinessPartner data source, a Order data source, etc. Using the data source name the system can automatically create the data source in-memory.

Additional data source fields

A data source contains a set of fields. Data source fields are derived from table columns/entity property but can also be defined explicitly. Explicit data source fields are used in different ways:

• for a standard table-driven data source they can be used to define extra fields which are read from the database and send to the client. For example for a business partner datasource, the extra data source field paymentTerm.netdays would return the value of the netDays of the paymentTerm of the business partner. The default data source implementation should be able to handle these types of fields automatically.
• they can be used to define computed fields which are not directly related to the database, in this case the implementation class of the data source will compute the field values
• they can be used by other parts of the system to make it possible to link for example a form field to a (computed) data source field.

The module provides a default data source implementation which can be extended by other modules.

Datasource REST api

The data source module adds a REST api which is used by the client side to query/retrieve data, update, create and delete data using the defined data sources.

Data Model

This module adds a new component type (data source) to the component type AD_REFERENCE.

This module adds a template to the COMPONENT_TEMPLATE table. This template is used to create the client-side representation of the data source.

This module has a table: DATASOURCE. This table has the following columns:

• name and description
• ad_module_id: the module of the data source, a module can add new data sources
• component type: link to the templates which are used to render the data source on the client. Only the templates which have the Datasource component type are displayed here.
• ad_table_id: foreign key to a table, in this case the data source will read data from this data and support updating of this table
• whereclause: an HQL where-clause to limit the data read from the table. Note that additional where-clauses can also be defined in the components itself or in the AD_REFERENCE.
• classname: the classname of an implementing class, if this field is empty then the default data source implementation is used. The implementing class must implement the DataSource interface.

(in addition this table has client/org and audit info columns)

In addition there is a DATASOURCE_FIELD table which makes it possible to define additional fields which are provided by the data source implementation. The DATASOURCE_FIELD can also be used to ensure that extra related information is also made available to the client.

The DATASOURCE_FIELD table has the following columns:

• name and description
• foreign key to the parent data source table

(in addition this table has client/org and audit info columns)

Smartclient Module

This module provides several things:

• the smartclient javascript libraries
• the smartclient skin which gives an Openbravo look
• Openbravo specific startup and skin load facilities

The skins are stored in the folder: web/org.openbravo.userinterface.smartclient/openbravo/skins. There is one skin enabled: 2.50_emulation. Any module is allowed to add components to this skin.

A module can also add its own skin. The skin which is used is determined by a global variable.

Freemarker Template Module

This module adds the freemarker template language to the system. This module provides two things:

• a value for the template language AD_REFERENCE: freemarker
• an implementation of a template processor which handles freemarker templates.

The module will register a context listener to be able to register its template processor from the start.

Selector Module

This module makes it possible to define a selector. In the end-user user interface, a selector consists of two main parts:

• the selector field in the form itself with a suggestion box behavior
• the selection grid displayed in the popup div

For the definition the selector will be defined using 2 concepts:

• a selector itself (the header) for example defining the datasource
• one or more selector fields (the children of the selector), the selector fields define which fields are displayed in the suggestion box and the popup window

This module will also contain an implementation of a new selector reference. This will be implemented when the new extendable AD_Reference design is finalized.

A selector is linked to a module, so modules can add new selector or add new selector fields to existing selectors.

Grid Component

This section gives an overview of the requirements for a new Grid component.

Grids are defined in the Application Dictionary. In the new UI architecture grids should be changeable at runtime without recompiling or restarting the system.

The new grid component should support the following functionality:

• Scrollable: grid should allow scrolling through larger datasets (data is read and rendered on demand)
• Show related data: the grid should not only show the record of one table but also of related/associated tables.
• Column functionality:
  • hiding and ordering: it should be possible to hide and re-order columns.
  • freeze: the ability to freeze columns on the side, while scrolling horizontally through the columns
  • resize columns
  • sorting: support initially only single column sort and later multi-column sort
• Summary/formula columns: it should be possible to define a column as a formula with the underlying record (and related data) as the parameters in the formula. A next step/level would be to support more advanced computation with server-side computations. This will however have performance impacts.
• Support large number of columns: the grid should support many columns and still render fast (columns should be rendered on demand).
• Multi-row per row: per row in the grid show multiple rows of information
• Row grouping: grouping of rows by a certain column (depends on server side support and performance impact for this feature).
• Actions/menu:
  • header context menu: make it possible to add actions to a header context menu
  • tool/action bar with a grid together with single and multiple select capabilities.
• Cell/row context menu: make it possible to add action to a row or show a context menu on a cell.
• Hide columns cells: hide values in a cell using a display logic formula/definition using the record shown in the row and of related data and other context data as parameters.
• Filtering:
  • Column-filter: support filtering by column.
  • Complex filtering/querying: make it possible to define a complex filter/query based on the columns on the grid (and related data).
• Store state: the state of the grid (sort, size, order, visibility, filter and complex query) should be stored in the client (preferably) or on the server, so that when the user revisits the grid the same settings are applied.
• Header grouping: multi-level grouping of column headers.
• Support inline grid editing (and inserting) with direct save or batch save.

Data Model

This module adds the following to the kernel module:

• a new value for the list reference (Component Type): selector
• a new component template for the selector which can generate a form and grid representation of the selector.

The selector itself is defined in two tables, the SELECTOR and SELECTOR_FIELD tables.

The SELECTOR table has the following columns:

• name and description
• ad_module_id: foreign key to a module (makes it possible to add new selectors through a module)
• ad_table_id: can be used to select an automatic datasource
• whereclause: a HQL whereclause used to limit the data shown in the selector
• sortbyclause: an initial sortby clause (HQL)
• datasource_id: the foreign key to a data source, can be used when no default data source is used
• component_template: a link to the template responsible for generating the client-side rendering, shows the templates with component type == selector

(in addition this table has client/org and audit info fields)

The SELECTOR_FIELD table has the following fields:

• selector_id: foreign key to the parent table
• name and description
• ad_module_id: foreign key to a module (makes it possible to add new selectors fields through a module)
• ad_column_id: foreign key to a column, applies when the datasource is defined in the parent SELECTOR record, the column must be either from the table of the datasource.
• datasource_field: can be used to refer to a specific data source field
• sort: yes/no, allow sorting with this field
• filter: yes/no, allow filtering using this field
• showInSuggestionBox: yes/no, show the field in the drop-down suggestion box
• showInGrid: yes/no, show the field in the popup grid

(in addition this table has client/org and audit info fields)

Defining Selectors

To define a selector a consultant has to go through the following steps:

• Define a selector with the correct fields to display.
• Define a selector reference with the following Model and WAD implementations:
  • org.openbravo.userinterface.selector.wad.WADSelector
  • org.openbravo.userinterface.selector.SelectorDomainType
• The parent of this selector reference is the base selector reference.

Model Property Selector

A specific selector will be implemented to select properties from the model in the Openbravo User Interface. The implementation of a model selector involves creating a data source class and setting up the datasource and reference in the AD.

As a test case the column MODELPROPERTY of the table: OBUISEL_SELECTOR_TEST will make use of the new model selector.

Implementing the datasource

In java implement a DataSourceService (extending BaseDataSourceService). Only the fetch method has to be implemented (see below for a description). There is already a class which can be used: ModelDataSourceService.

The fetch method should do the following:

1. read the table id from the parameters (the name of the parameter will probably be ad_table_id or inpad_table_id).
2. Read the table from the db and get the Entity from the ModelProvider (using the ad_table.name as the entity name)
3. then check what the user has typed (is passed in as a parameter to fetch), match what the user has typed with the properties in the entity and the referenced objects. For example, for the entity Product (stored in the M_Product table):
   1. the user types: shelf, the system returns with a list of three names: shelfHeight, shelfWidth, shelfDepth
   2. the user types: discontinued, the system returns with a list of three names: discontinued, discontuedBy
   3. the user types: taxCategory. (note the dot), the system returns with all properties of the taxCategories prefixed with the word: taxCategory.

The system returns a json string which contains the data found in the steps above. See for a description of the response format: here. For examples of current code check out the DefaultJSONDataService.fetch method (at the end).

The data part of the response consists of json objects which have a field/attribute corresponding to the name of the datasourcefield. In this case only one property is returned (defined as the first datasourcefield in the datasource). To get the name of the datasourcefield call this method: getDataSource().getOBSERDSDatasourceFieldList().get(0).getName(). The name of the first datasource field defined in the datasource (see below).

Application Dictionary Setup

1. define a datasource in the AD (see the datasource menu option in the AD). The java class field in the datasource should contain the class name of the java class (see step 1). The datasource should be defined in the datasource module.
2. add a datasource field to the datasource. Give this datasource field the name 'modelProperty' (is re-used below!) and select the String reference.
3. go to the Reference window. Create a new Reference with the following settings:
   1. name: start with OBUISEL, for example: OBUISEL_Model Selector
   2. parent reference: OBUISEL_Selector Reference
   3. model implementation: org.openbravo.userinterface.selector.model.domaintype.ModelElementDomainType
4. for the created referenced go to the 'defined selector' and create a new defined selector:
   1. set the name and description
   2. and set the datasource to the datasource created in step 2, the other fields can be kept empty
5. for the created defined selector create a selector field:
   1. set the name and description
   2. set the field property to the value: modelProperty
6. then back in the Defined Selector, select the new selector field as the value and displayfield
7. go to the AD: Table and Columns, open the OBUISEL_SELECTOR_TEST table and then go to the MODELPROPERTY column. Set the reference. As a base reference select the 'OBUISEL_Selector Reference and as a subreference the new reference created in step 3.

This concludes the AD setup. To make them live, recompile the Selector Test window: ant compile -Dtab=SelectorTest.

Testing

Put breakpoints in the datasource implementation class in the fetch method.

Start Openbravo go to AD > Selector Test, go to the edit view and put the cursor in the model property field and start typing. You should see requests in the console and the system should reach the breakpoint in the fetch method. Check out the parameters to see what is posted as part of the request.

Developer or Consultant?

The modules described in this document both contain core-technical modules as well as end-user oriented modules. The module structure has been setup to make it easy to extend, override and add (core) functionality. The question is which person (with what profile) works with these module. This section tries to answer that particular question:

• Developers can add a new templating language
• Developers can add new datasource implementations in modules
• Developers are responsible for creating new templates and component types.
• Consultants define new selectors but will normally not be involved in defining new templates.

To be done (design and implementation)

Design topics to be done:

• extend the design to allow definition of global templates which are included in all other templates. This can possibly be done by using a special component type value: global.
• use the ad_reference to generate custom types in smartclient and use these types in the datasource generation
• implement correct date and number formatting on the client and take care of conversion in the json data service
• design how to do translation.

Implementation topics to be done:

• The selector and selector_field tables need a trl variant for translation
• In the datasource window: a check needs to be added that at least the classname or table are set.
• In the selector and selector_field window tabs:
  • if the selector is linked to a datasource then the selector field may only use columns defined for the datasource (the table of the datasource) or datasource fields (if the datasource has fields).
  • if the selector has a table then the selector_field may only use columns from the selected table.
  • if the selector has a table set then the datasource field can not be set (and the other way around)
  • the whereclause is only editable if the table is set in the selector.
• Add the concept of a datasource_field table/entity. This can be used to limit/set the fields returned by the dataset. The datasource field can be a reference to a column or to a reference (defining its type). The datasource field can only be set if the datasource has a classname set (custom implementation of datasource).
• Extend the selector_field to refer to the datasource field table if the selector is linked to a datasource which has explicit fields defined.

Category: Deprecated