ERP module integration with CI

Contents 1 Introduction 2 Life Cycle 3 Sanity check 4 Compilation and Create OBX 4.1 Prerequisite 4.2 Process 5 Database consistency check 5.1 Process 6 JUnit test 6.1 Prequisite 6.2 How to invoke JUnit in CI 7 Install and Uninstall of module 7.1 Prerequisite 7.2 Process 8 Selenium test 8.1 Prerequisite 8.2 Process 9 Check Module Update 9.1 Prerequisite 9.2 Process 10 Create New Jobs in hudson CI for modules 10.1 New job 10.2 Configuration 10.2.1 Generic Settings 10.2.2 Module specific settings 10.2.3 Sanity check of the module. 10.2.4 Database consistency check 10.2.5 JUnit tests for module 10.3 Post build Action 11 How to Debug 11.1 Build Failure 11.2 Selenium Failure 12 Script reference 13 Next steps

Introduction

Openbravo 2.50 comes with the new concept of Modularity: the ability for developers to build, package and distribute Extension Modules and for users to install, uninstall and update Extension Modules.

This documents explains on how to check and test the development phase of each modules using continuous Integration farm. Which helps in fast tracking the status of module build, installation, test and publish

Life Cycle

Module once enters in to development phase, it is necessary it undergoes following stages before it get published in Central Repository for customers

• Sanity Check
• Source compilation check and Create OBX
• Database consistency test
• Module's JUnit test
• Installation of the generated OBX
• Un-Installation of the installed module
• Selenium test (module smoke)
• Upgrade the module from previous published version in CR to generated OBX

Note: Upgrade test can be done once the module has first version published in CR

Sanity check

This is the basic check mandatory for all the modules, it checks for the following items and alerts the ones which do not meet the required criteria

License Audit:

• Specification public(OBPL) or commercial(OBCL) license of the module
• Appropriate license text for OBPL/OBCL
• Appropriate license link for OBPL/OBCL
• Author tag must have "Openbravo S.L.U."
• File Headers in module directories must have appropriate license text for OBPL/OBCL. Below is an example for OBCL

* Copyright (C) 2010 Openbravo S.L.U.
* Licensed under the Openbravo Commercial License version 1.0
* You may obtain a copy of the License at http://www.openbravo.com/legal/obcl.html 
* or in the legal folder of this module distribution.

Other Checks:

• Help and Description columns should not be empty
• URL tag must point to module's project page in forge
• If a module is not the first version then update information tag must be filled

Note: It is necessary to take care of these items in modules development to avoid failure during integration process.

Compilation and Create OBX

This stage is to check for the source compilation and on success create obx. Below is the work flow of this stage

Prerequisite

• ERP source is made to point to
  • Any of the branch in erp/devel (takes the latest changeset in the branch)
  • If erp/devel/pi then it takes the latest MP
• Dependency modules can be obtained in two ways either from
  • SCM (latest source of all dependency modules)
  • CR (latest published version of dependency modules)
• Build the OBX on the minimum version of ERP_SOURCE supported by the module
  • Compile and generate obx over the ERP version specified in AD_MODULE_DEPENDENCY.xml
  • Invoke the creatOBX with minimal argument

Note: One can go for SCM option, When the AD_MODULE_DEPENDENCY.xml is specified with appropriate dependency list, else opt for OBX that are published in CR.

Process

• Workspace is updated with all the dependencies in $WORKSPACE/modules/
• Performs install.source
• Clone the main module to be monitored in to $WORKSPACE/modules/
• Performs smartbuild -Dlocal=no
• Finally generate obx

Now the obx is available to download from here

Database consistency check

This is to check the database consistency after including the module.

Process

• Set Core in development
• Run ant trl.clean and ant compile -Dtab=xx to run the translate with Core set in development
• Run the synchronize terminology process
• Run ant export.database -Dforce=yes
• Verify if there are differences and therefore if the database is consistent or not

JUnit test

Check for the JUnit inside module

Prequisite

• Modules must have the JUnit classes defined in "org.openbravo.module.modulename/src-test/org/openbravo/test/"
• build.xml to invoke the JUnit test in "org.openbravo.module.modulename/build.xml"

How to invoke JUnit in CI

On satisfying the perquisite above invoke the jUnit as below

Example: ant -f $WORKSPACE/erp_source/modules/$MOD_NAME/build.xml test

Install and Uninstall of module

On completing the stage-2 the obx is available with latest changeset of the module and ready to check the installation and uninstallation

Prerequisite

• ERP source is made to point to
  • Any of the branch in erp/devel (takes the latest changeset in the branch)
  • If erp/devel/pi then it takes the latest MP
• Dependency modules are obtained form CR or SCM as specified initialy
• Install the generated obx on the minimum version of ERP_SOURCE supported by the module
  • Invoke the testModInstall with minimal argument

Process

• Workspace is updated with source and with all the dependencies in $WORKSPACE/modules/
• Download the dependency modules from CR or source repository in to $WORKSPACE/modules/
• Performs install.source, deploy the context and available to access via web UI.
• Invoke the selenium test target to install the module's obx via Module Management console(MMC)
• Invoke the selenium test target to un-install the module via MMC

Selenium test

Here is to run any selenium test for modules

Prerequisite

• Have the selenium class in your teams automation repo
• Have the test defined in "/src-test/com/openbravo/test/integration/erp/modules/<module_name>/"
• Have build.xml to invoke appropriate test defined

Process

• The modules in installed using the obx generated as in Install_and_Uninstall_of_module
• Context is made available with modules installed
• Invoke the selenium test suite.

Check Module Update

This check is performed when the module has at-least one published version in CR

Prerequisite

• ERP source is made to point to
  • Any of the branch in erp/devel (takes the latest changeset in the branch)
  • If erp/devel/pi then it takes the latest MP
• Test module update using the miimum version of ERP_SOURCE supported by the module as specified in AD_MODULE_DEPENDENCY.xml
  • Invoke the creatOBX with minimal argument

Process

• Workspace is updated to have the latest ERP changeset
• Performs install.source, deploy the context and able to access on web interface.
• Install the module published in CR via MMC using selenium test suite
• Invoke the selenium test target to install the module's obx via Module Management console(MMC)

Create New Jobs in hudson CI for modules

New job

Template job is available in hudson, to create a new job

• Browse to modules.builds
• Hit New job
  • Fill in the Job name as module name
  • Select Copy existing job option and fill module _template

with this a new job will be created for your module

• Configure the job as in the following section
• Once configured click on Build Now to have first build of your module

Note: No space and dot are allowed in the job name, example of jobname assertreport, assest_report

Configuration

Now the job need to be configured to have details about the modules and frequency in which it has to be triggered etc

Browse to the new job configuration new job's configure window

Generic Settings

• Enable project-based security
  • By default the job will have Anonymous user with Job read permission
  • Add your teams account and enable permission (Delete,Configure,Read,Build,workspace)

• Tie this project to a node Master always
  • Hudson in modules.builds is configured with two machines master and slave
  • The stages like sanity check, createOBX, dbconsistency, JUnit will be done in master
  • The selenium related test will be done in slave1

• Source Code Management
  • Select Mercurial and fill the Repository URL with the repository path of the module to be monitored

• Build Triggers
  • Select on Poll SCM, purpose of this is to trigger the job on certain frequency only when there is a new changeset available in module's repository.

 Setting this is similar to cronjob setting 
 Example: # To execute the job daily at 3 am
 0 3 * * * 

• Locks
  • This feature is to lock the machines when one job is in execution status so that to avoid collusion between jobs
  • By default the slave1 will be locked to avoid two parallel selenium jobs at a same time

Module specific settings

This is to set which is the script to be executed and what inputs to be given to the scripts Specify the module type either it is public or commercial module, please enable/comment the token accordingly

#export MOD_TYPE="public"
#export MOD_TYPE="private"

Specify the Modules project name here as specified in forge and uncomment the token (this variable is required for sanity check).

#export MOD_PRO_NAME=

Specify the module you want to monitor, the name as specified in SCM

# Example : export MOD_NAME="org.openbravo.userinterface.selector"
export MOD_NAME="org.openbravo.userinterface.selector"

Please specify which selenium repo for to be referred for selenium test classes.

export SMOKE_REPO=pi-ci

Please specify the source repo for ERP

export SOURCE_REPO=pi-engdev

Please specify the dependency type if you want to include. Uncomment the token appropriate for you. The dependency from source then specify as "scm", note this works only when AD_MODULE_DEPENDENCY.xml has all the dependency set init. If you want to pull the latest published version of modules then specify as 'obx'

#export DEP_TYPE="scm"
#export DEP_TYPE="obx"

Please specify the project names of the modules as in forge, this is mantary when u specify OBX

# Example export DEPENDENT_MODULES="obsmartclient openbravoseam Openbravoclientkernel openbravoclientfreemarker DatasourceService openbravojsonrest"

Sanity check of the module.

sh /srv/hudson_setup/module_integration/sanity-check.sh

Database consistency check

sh /srv/hudson_setup/module_integration/dbconsistency

JUnit tests for module

ant -f $WORKSPACE/erp_source/modules/$MOD_NAME/build.xml test

Post build Action

• Email Notification
  • Navigate to job configuration page -> Post build Action
  • Enable Editable email notification
    • Add your team's mailing list to Global Recipient List
    • Click Advance and add trigger for failure and fixed to notify team on failure and build fixed
• Enable "Publish JUnit test result report", Specify the log-file path ad

SANDBOX/automation/pi/last-results/*/logs/*.xml

How to Debug

Here are simple guidelines on how to check the status of job and error logs

Build Failure

• Build/job failure will be shown as red ball and success with green ball
• Console output of the build can be used to check appropriate failure/success logs. This is similar we execute the scripts in command prompt. this is available for every build executed for a job. url format to check the log

http://modules.builds.openbravo.com/job/<JOB_NAME>/<BUILD_NUMBER>/consoleFull 

Note: Replace the JOB_NAME and BUILD_NUMBER with your job

Selenium Failure

Selenium Failure here is the test like installation, un-installation, upgrade of modules are checked the via the ERP application web interface and we use selenium for it. The status of these test are recored using selenium test scripts. Below are how to check them

• Installation, un-installation, upgrade of modules are done using automation/pi-ci and its logs/screenshots can be accessed using the format

http://modules.builds.openbravo.com/job/<JOB_NAME>/ws/SANDBOX/automation/pi-engdev/last-results/

• Module related smoke test are done using your own automation repo in here and its logs/screenshots can be accessed using the format

http://modules.builds.openbravo.com/job/<job_name>/ws/SANDBOX/automation/<automation-repo>/last-results/

Script reference

• The scripts referred in configuration section is in SCM
• The selenium test classes need to appropriately placed in teams development branch in here

Next steps

• Automate module publishing process

Category: Continuous Integration