View source | View content page | Page history | Printable version   

Projects:Include Logs Clean Up Utility In Distribution/Technical Specifications

Contents

Overview

Log Clean Up Utility module is an already existing module first published now 16 months ago (Oct 2013) which has been matured in production environments proving to be useful to maintain log tables. It allows to define some tables to be periodically clean up by deleting old records.

The reasons for cleaning up log tables are:

As this is a common task, the goal of this project is to include this module within Openbravo 3 distribution including some standard pre-configuration to make easier to use it.

More details on how it works can be found here.

Configuration

Log Clean Up Utility module configuration is maintained per instance and consists in 2 parts:

Default configuration

The idea of adding this module to the distribution is to enable it by default based on some standard configuration, so everyone installing or updating to PR15Q2 will automatically take advantage of it.


Table Keep records newer than n days/Filters Notes
AD_PInstance 15 days Keeps track of process executions, deletions in this table cause also deletions in AD_PInstance_Para which stores information about parameter values used in each execution.
AD_Process_Request 15 days, only for completed requests Table used to schedule processes, it typically grows up with immediate executions.
AD_Process_Run
  • 15 days for successful executions
  • 30 days for executions with error
Keeps information of executions of background scheduled processes
AD_Session 40 days Stores a record per each session in Openbravo. Deleting this table causes also deletions on AD_Session_Usage_Audit

Processes scheduled to be run weekly Sunday 0:00 (server local time)

Default configuration for retail

Table Keep records newer than n days/Filters Notes
OBMOBC_LogClient
  • 30 days for warn and error messages
  • 15 days for the rest
Keeps track of logs client logs generated in WebPOS

Extended features

In addition to existing features, two new capabilities will be implemented within the scope of this project.

Table truncation

When wanting to delete all data in a table, executing truncate statement has several advantages than a delete one:

This will be implemented as a new optional flag in the table configuration.

Limitations

Table truncation can be only done in some cases:

Vacuum when working in PostgreSQL

PostgreSQL, when a row is deleted it is considered a dead tuple but it continues using disk space that cannot be reused. Only after executing vacuum command this space is reclaimed and can be re-used.

Due to performance problems in the vacuum full implementation in 8.x PostgreSQL versions, the selected option to execute is vacuum analyze. Which will reclaim the space to be re-used, but will not free this space, in instances with stable log creation and periodical clean up this should keep stable the amount of disk used for logs.

Performance considerations

Default configuration does not execute table truncation but it keeps log data generated during the last days. In order to perform these deletions log tables need to be queried. We consider to add indexes to the columns (created) used for this queries in order to speed up the process. Finally, we decided not to go for this option because adding indexes would penalize in more space consumed for existent records and slower insertions for new ones. As the clean up processes should be run when the system is lowly used, we understand the benefit of these new indexes wouldn't pay for the overhead they would add.

Closed topics

New module vs merge within an existing one

It is pending to decide whether the inclusion will be done by adding the module to the distribution or by merging it within one already there.

Decided: new module

New configuration active by default

Should the configuration be active by default?

Decided: active by default

Conflicting new configuration with existent configurations

Adding a default configuration might collide with instances that have already the module installed as new deletion rules will be added.

The solution is to mark the new ones as inactive in the existent instances, but this needs to be notified and executed before 1st execution of the clean up process.

Decided: document it

Review/approve proposed default configurations

Defined above.

Approved

Retrieved from "http://wiki.openbravo.com/wiki/Projects:Include_Logs_Clean_Up_Utility_In_Distribution/Technical_Specifications"

This page has been accessed 2,271 times. This page was last modified on 18 February 2015, at 12:30. Content is available under Creative Commons Attribution-ShareAlike 2.5 Spain License.