View source | Discuss this page | Page history | Printable version   

MasterdataController

Contents

Introduction

Openbravo now includes a new component that manages the master data models, called MasterdataController.

This is a read-only component that allows to define and provides access to all of the master data models. At the same time, the master data models define an API, which is detailed in the present document, and that can be used to retrieve their data.

Prerequisites

This article assumes you have knowledge on some of the basic IndexedDB concepts. If this is not the case we recommend to start by taking a look at this guide.

Defining a Master Data Model

To define a Master Data Models the following steps should be followed:

Creating a Master Data Model Definition

The client side definition of a master data model is done by defining a Javascript class that extends OB.App.Class.MasterdataModel:

 
  class SalesRepresentative extends OB.App.Class.MasterdataModel {
    constructor() {
      super();
      this.indices = [
        new OB.App.Class.Index({
          name: '_identifier_idx',
          properties: [{ property: '_identifier' }]
        })
      ];
    }
 
    getName() {
      return 'SalesRepresentative';
    }
  }

Note that here we define the indices of the model which we will later explain in this section.

Registering a Master Data Model

To register a master data model we make use of the MasterdataController and its registerModel function:

 
  OB.App.MasterdataController.registerModel(SalesRepresentative);

The registerModel function receives the class name of the model to be registered.

Once the model has been registered the MasterdataController will expose it so we can access it using OB.App.MasterdataModels:

 
  OB.App.MasterdataModels.SalesRepresentative;

It is strongly recommended to create and register the model in the same function. Thus, we are not exposing the model class definition and the model can be accessed from a single central point (with OB.App.MasterdataModels):

 
(function SalesRepresentativeDefinition() {
  class SalesRepresentative extends OB.App.Class.MasterdataModel {
    constructor() {
      super();
      this.indices = [
        new OB.App.Class.Index({
          name: '_identifier',
          properties: [{ property: '_identifier' }]
        })
      ];
    }
 
    getName() {
      return 'SalesRepresentative';
    }
  }
 
  OB.App.MasterdataController.registerModel(SalesRepresentative);
})();

Creating the Master Data Model DataSource

Each master data model is linked to a backend component that takes care of providing the model data (records). This component is a Java class that extends the MasterDataProcessHQLQuery class.

A class extending MasterDataProcessHQLQuery should:

The code snippet below shows an example of the structure of a MasterDataProcessHQLQuery:

 
@MasterDataModel("SalesRepresentative")
public class SalesRepresentative extends MasterDataProcessHQLQuery {
  public static final String salesRepresentativePropertyExtension = "OBPOS_SalesRepresentativeExtension";
 
  @Inject
  @Any
  @Qualifier(salesRepresentativePropertyExtension)
  private Instance<ModelExtension> extensions;
 
  @Override
  protected List<HQLPropertyList> getHqlProperties(JSONObject jsonsent) {
    ...
  }
 
  @Override
  protected List<String> getQuery(JSONObject jsonsent) throws JSONException {
   ...
  }
 
  @Override
  protected boolean bypassPreferenceCheck() {
    ...
  }
 
  @Override
  protected Map<String, Object> getParameterValues(JSONObject jsonsent) throws JSONException {
    ...
  }
 
  @Override
  public List<String> getMasterDataModelProperties() {
    return getPropertiesFrom(extensions);
  }
}

The getPropertiesFrom is a method that retrieves the HQL properties of the model from its extensions. If a master data model does not have extensions and its properties are directly retrieved from the related DAL class, there is also another method available with the same name but it receieves the DAL class as argument instead.

Creating Test Cases

Ideally, when creating a new MasterDataProcessHQLQuery it must be accompanied by jUnit test cases to check that the expected records are fetched when fully or incrementally refreshing of the model.

The MasterdataModelTestHelper utility class can be used to build this kind of test cases. This test class should be instantiated by providing the name used to annotate the MasterDataProcessHQLQuery with the @MasterDataModel annotation:

MasterdataModelTestHelper salesRepresentativeModel = new MasterdataModelTestHelper("SalesRepresentative")

And then it can be used to test the records retrieved either with a full refresh...

Map<String, Object> contextParameters = Map.of("organization", VALL_BLANCA_STORE);
salesRepresentativeModel.fullRefresh(contextParameters);

... or with an incremental refresh providing the date of the last update

Date lastUpdate = DateUtils.hoursAgo(1);
salesRepresentativeModel.incrementalRefresh(contextParameters, lastUpdate);

See a real example of use here.

IndexedDB Masterdata Database

All the information retrieved from the master data models is stored in an IndexedDB database managed by the MasterdataController. In particular, the data is persisted on an single object store for each model.

IndexedDB Masterdata Database

The MasterdataController is also in charge to populate and keep this database updated when the data is loaded in Web POS. See here to learn about the different modes to load the master data.

In addition, the MasterdataController has a recreation mechanism that detects model changes (both in the client and in the backend side). This can happen, for example, if after a module upgrade a new property is added into a model. In that case, on login the MasterdataController will drop the database entirely and recreate it from scratch using the new model definition(s).

Masterdata Model API

All of the master data models provide an API that is the mechanism that should be used to access to their data. Note that the functions of this API perform asynchronous tasks to retrieve the information from IndexedDB and for this reason they are declared as async functions.

Next all these master data model API functions are detailed.

withID

This function is used to retrieve an object of the master data model by its ID.

 
  const salesRepresentative = await OB.App.MasterdataModels.SalesRepresentative.withId(id);

find

This function is used to retrieve all the objects that match a criteria. To define a criteria the Criteria API is used. This API provides a set of functions that can be easily combined using function chaining:

Bulbgraph.png   Before start using find in your code it is strongly recommended to read the Performace Considerations section in order to ensure that this function is used in a performant way.

Next some basic usage examples are shown:

Find with Single Criterion

Find the object of the Sales Representative model with user name equal to vallblanca:

 
  const criteria = new OB.App.Class.Criteria()
      .criterion(new OB.App.Class.Criterion('username', 'vallblanca'))
      .build();
  const salesRepresentative = await OB.App.MasterdataModels.SalesRepresentative.find(criteria)[0];

which is equivalent to:

 
  const criteria = new OB.App.Class.Criteria()
      .criterion(new OB.App.Class.Criterion('username', 'vallblanca', 'equals'))
      .build();
  const salesRepresentative = await OB.App.MasterdataModels.SalesRepresentative.find(criteria)[0];

because equals is used by default if no comparison is used.

Bulbgraph.png   The first level properties of a Criteria that use the equals comparison are known as query properties.

In the previous example the Criteria has one query property: username.

Find with Multiple Criterion (AND)

The and operator is the default one, so we do not need to specify it:

 
  const criteria = new OB.App.Class.Criteria()
      .criterion('active', false)
      .criterion('username', ['vallblanca', 'demouser'], 'in')
      .build();
  const salesRepresentatives = await OB.App.MasterdataModels.SalesRepresentative.find(criteria);

Find with Multiple Criterion (OR)

If we want to use the or operator we need to set out it with the operator function:

 
  const criteria = new OB.App.Class.Criteria()
      .operator('or')
      .criterion('active', false)
      .criterion('username', ['vallblanca', 'demouser'], 'in')
      .build();
  const salesRepresentatives = await OB.App.MasterdataModels.SalesRepresentative.find(criteria);

Find with Multiple Criterion (AND + OR)

We can combine operators using multiCriterion:

 
  const criteria = new OB.App.Class.Criteria()
      .multiCriterion(
        [
          new OB.App.Class.Criterion('username', 'vallblanca', 'not'),
          new OB.App.Class.Criterion('name', 'L', 'includes')
        ],
        'or'
      )
      .criterion('active', true)
      .build();
  const salesRepresentatives = await OB.App.MasterdataModels.SalesRepresentative.find(criteria);

Find with Limit

The API supports defining the maximum number of objects that can be returned with find by using the limit function:

 
  const criteria = new OB.App.Class.Criteria().limit(5).build();
  const salesRepresentatives = await OB.App.MasterdataModels.SalesRepresentative.find(criteria);

Note that if no limit is provided (the limit function is not used), the infrastructure sets a default limit of 300 elements.

Find and OrderBy

We can also sort (ascending or descending) the results of a search with find using orderBy:

 
  const criteria = new OB.App.Class.Criteria().limit(5).orderBy('name', 'asc').build();
  const salesRepresentatives = await OB.App.MasterdataModels.SalesRepresentative.find(criteria);

Observe that it also exists an orderBy function that can be directly invoked from any Masterdata model.

Find with Inner Criteria

An inner criteria can be used when it is needed to define more complex conditions:

 
  const criteria = new OB.App.Class.Criteria()
      .criterion('active', true)
      .innerCriteria(
        new OB.App.Class.Criteria()
          .multiCriterion(
            [
              new OB.App.Class.Criterion('username', null, 'not'),
              new OB.App.Class.Criterion('name', 'vallblanca', 'not')
            ],
            'and'
          )
          .multiCriterion(
            [
              new OB.App.Class.Criterion('_identifier', null, 'not'),
              new OB.App.Class.Criterion('name', 'L', 'includes')
            ],
            'and'
          )
          .operator('or')
      )
      .build();
  const salesRepresentatives = await OB.App.MasterdataModels.SalesRepresentative.find(criteria);

orderedBy

The master data models also have a function that allows to directly retrieve their objects sorted by some specific properties:

 
  const salesRepresentatives = await OB.App.MasterdataModels.SalesRepresentative.orderedBy(['_identifier', 'active'], ['asc','desc']);

Performance Considerations

When writing queries using the API explained in the previous section it is very important to take into account some restrictions imposed by the IndexedDB design. In general, if we want to make a performant query against a model, we should try to ensure that the query uses a proper index.

Creating indices

To create a new index we make use of the OB.App.Class.Index class whose constructor receives two properties:

Bulbgraph.png   Remember always to set as true the isBoolean attribute for the boolean properties of the index
Bulbgraph.png   Remember always to set as true the isNullable attribute for the nullable properties of the index

We can register indices of the model when creating the model or we can extend an existing model and register new indices on it using the addIndices function as below:

 
  OB.App.MasterdataModels.SalesRepresentative.addIndices(
    new OB.App.Class.Index({
      name: 'name_idx',
      properties: [{ property: 'name' }]
    })
  );

By default all the models are created with a default index by their id property, therefore it is not necessary to explicitly create an index by id.

Effective Index Usage

Now the we know how to create an index, we need to understand when they should be created. To this we need to learn how our Criteria IndexedDB queries are executed.

Criteria Execution Process

The process to execute a Criteria IndexedDB is the following:

  1. First the results are filtered using a query at database level, if possible. To execute this query we need an index. The infrastructure will try to find an index which have the same properties (in the same order) as the query properties of the criteria. Remember: the query properties are the first level properties that use the equals comparison.
    1. If an index with exactly all the query properties (in the same order) is found, then that index will be used
    2. If not found but there is an index with the first n query properties (in the same order), then that index will be used.
    3. If no index can be found, then the query will fail, throwing an error.
  2. Once we have filtered the results using the selected index, the rest of the criteria will be applied (non query properties: comparisons not using equal and/or inner criteria). This can be the most expensive part of the query as this filtering is done in memory through an IndexedDB cursor.
  3. When all the objects matching the criteria are found or the query limit is reached, then they will be returned. If the result should be ordered, results will be ordered in memory before being returned.

User Model Example

Let's illustrate the process described above with an example. Suppose that we have a dummy model called User. An example object of this model would be as follows:

 
  {
    id: '3073EDF96A3C42CC86C7069E379522D2',
    name: 'Vall Blanca Store User',
    username: 'vallblanca',
    type: 'A',
    birthDate: '1980-07-08T00:00:00.000Z',
    credit: 23.09,
    active: true,
    expired: false
  }

We register our model with an index using the MasterdataController:

 
class User extends OB.App.Class.MasterdataModel {
  constructor() {
    super();
    this.indices = [
      new OB.App.Class.Index({
        name: 'active_type_idx',
        properties: [
          { property: 'active', isBoolean: true },
          { property: 'type' }
        ]
      }),
      new OB.App.Class.Index({
        name: 'username_idx',
        properties: [
          { property: 'username', isNullable: true}
        ]
      })
    ];
  }
}
OB.App.MasterdataController.registerModel(User);

Then, let's suppose that we want to execute the following query:

 
  const criteria = new OB.App.Class.Criteria()
      .criterion(new OB.App.Class.Criterion('active', true))
      .criterion(new OB.App.Class.Criterion('type', 'B'))
      .criterion(new OB.App.Class.Criterion('credit', 25, 'lowerThan'))
      .orderBy('name', 'asc')
      .build();
  const users = await OB.App.MasterdataModels.SalesRepresentative.find(criteria);

The previous query has:

Therefore, the query will be executed as follows:

  1. The User model has an index that exactly matches with the query properties: active_type_idx. This index will be used to filter the results with a database query in order to retrieve the active users of type 'B'.
  2. For each of the filtered result, an in-memory validation will be done to filter the results that have a 'credit lower than 25.
  3. The filtered results are ordered by name.

Note that if we had defined the query in this way:

 
  const criteria = new OB.App.Class.Criteria()
      .criterion(new OB.App.Class.Criterion('type', 'B'))
      .criterion(new OB.App.Class.Criterion('active', true))
      .criterion(new OB.App.Class.Criterion('credit', 25, 'lowerThan'))
      .orderBy('name', 'asc')
      .build();
  const users = await OB.App.MasterdataModels.SalesRepresentative.find(criteria);

The infrastructure would not have been able to find an index to execute the query, because we do not have an index by the query properties in the same order. In this case, the query would eventually fail.

Bulbgraph.png   An index can be selected if its properties match with all or part of the query properties in exactly the same order

Of course to execute the previous query we could add an index type_active_idx into the model to match with the query properties but this is not recommended because adding a new index implies an increase on the storage size used by the application. If the model has a lot of objects then the index size can be big. So in this case we recommended to reorder the query properties defined in the query to match with an existing index.

Bulbgraph.png   Try to reuse the model indices as much as possible. It is recommended to always use the same order in the query properties of the queries done for a model.

Finally, if we wanted to execute this query:

 
  const criteria = new OB.App.Class.Criteria()
      .criterion(new OB.App.Class.Criterion('active', true))
      .criterion(new OB.App.Class.Criterion('type', 'B'))
      .criterion(new OB.App.Class.Criterion('username', 'demouser'))
      .criterion(new OB.App.Class.Criterion('credit', 25, 'lowerThan'))
      .orderBy('name', 'asc')
      .build();
  const users = await OB.App.MasterdataModels.SalesRepresentative.find(criteria);

In this case the query would be executed using the active_type_idx index also. This is because although the index does not have all the query properties (it is missing the username property), the infrastructure is able to do a key range filtering using the matching properties (active and type). And the rest of the criteria properties would be filtered in memory.

High Volumes Configuration

In case a model has a high volume of data, it could happen that having an index is not enough to make a query work in a performant way.

If we were in such case, each of the master data models provides a mechanism that allows to execute the queries using an in-memory cache which in general brings a faster execution of the data filtering. To enable this mechanism it should be specified which object properties will be kept in the model cache.

It is possible to enable the cache at the same moment of registering a new model with the searchProperties property:

 
class User extends OB.App.Class.MasterdataModel {
  constructor() {
    super();
    this.indices = [
      new OB.App.Class.Index({
        name: 'active_type_idx',
        properties: [
          { property: 'active', isBoolean: true },
          { property: 'type' }
        ]
      }),
      new OB.App.Class.Index({
        name: 'username_idx',
        properties: [
          { property: 'username', isNullable: true}
        ]
      })
    ];
    this.searchProperties = ['active','name'];
  }
}
OB.App.MasterdataController.registerModel(User);

Or it is possible to extend existing models to include new properties into the cache using the addSearchProperties function:

 
OB.App.MasterdataModels.SalesRepresentative.addSearchProperties(['type', 'username']);

When the cache mechanism is enabled for a model, the internal process to execute the query against that model is slightly different:

  1. First the query mode is selected (index or cache):
    1. If an index with exactly all the query properties (in the same order) is found, then that index will be used
    2. If not found and all the properties referenced in the query are present in the cache, then the results will be filtered with the cache
    3. If it is not possible to use the cache, but there is an index with the first n query properties (in the same order), then that index will be used.
    4. If no index can be found, then the query will fail, throwing an error.
  2. If we have filtered the results using an index, the filter by non query properties should be done.
  3. When all the objects matching the criteria are found or the query limit is reached, then they will be returned. If the result should be ordered, results will be ordered in memory before being returned.

Please note that enabling the cache mechanism has some implications:

Bulbgraph.png   Do not enable the cache mechanism unless it is strictly necessary.
Bulbgraph.png   Try to keep into the cache as less model properties as possible.

Masterdata Listeners

Bulbgraph.png   Available only in core2, not in mobile.core

It is possible to register a post update hook, to execute a function when specified masterdata models are updated.

Example code to register a masterdata post hook:

 
OB.App.MasterdataController.postUpdateHookRegister({
  modelsToListen: ['Keymap','KeymapVariant'],
  updateFunction: async refreshMasterdataMethod => {
    if (refreshMasterdataMethod === 'save') {
      // masterdataEndpoint messages, with the debounce we execute only once, instead once per model
      await updateFunctionDebounced();
    } else {
      await updateFunction();
    }
  }
});

Masterdata Model Configuration

Bulbgraph.png   This feature is available starting from 3.0PR23Q3.

Masterdata Model Application Dictionary Definition

It is possible to configure the mode which the data of a master data model is retrieved. To be able to set up this configuration first the model must be registered in the application dictionary through the Master Data Model window.

Master Data Model Window

In this window the following fields must be populated:

Bulbgraph.png   Declaring that a model supports remote should come together with an assessment on its performance.

Model Requirements to Support Remote Mode

To make a master data model support remote the corresponding MasterDataProcessHQLQuery has to meet some requirements:

  1. It must implement the getHqlProperties method
  2. The HQL queries must contain the $filtersCriteria filter on its where clause and the order by clause must be $orderByCriteria

Model Configuration

By default all the master data models work in Local mode. But it is possible to change this behavior at configuration level, using the Master Data Model Configuration window.

Master Data Model Configuration Window

Here the fields to populate are:

Bulbgraph.png   If no configuration is defined in the current session organization the closest one in the ancestors tree will is taken.
Bulbgraph.png   If the configuration from a master data model of a mobile application changes from Remote to Local this will cause a full refresh of the master data.

Remote Mode Requests

As explained above, when a model is configured to work in remote mode, the master data is retrieved by querying the information to the backend through HTTP POST requests. This requests are executed within the timeout configured for the org.openbravo.mobile.core.master.MasterDataLoader mobile service.

Retrieved from "http://wiki.openbravo.com/wiki/MasterdataController"

This page has been accessed 60,370 times. This page was last modified on 16 May 2023, at 19:09. Content is available under Creative Commons Attribution-ShareAlike 2.5 Spain License.