ERP module integration with CI
Contents |
Introduction
Openbravo 2.50 comes with the new concept of Modularity: the ability for developers to build, package and distribute Extension Modules and for users to install, uninstall and update Extension Modules.
This documents explains on how to check and test the development phase of each modules using continuous Integration farm. Which helps in fast tracking the status of module build, installation, test and publish
Life Cycle
Module once enters in to development phase, it is necessary it undergoes following stages before it get published in Central Repository for customers
- Sanity Check
- Source compilation check and Create OBX
- Database consistency test
- Module's JUnit test
- Installation of the generated OBX
- Un-Installation of the installed module
- Selenium test (module smoke)
- Upgrade the module from previous published version in CR to generated OBX
Note: Upgrade test can be done once the module has first version published in CR
Sanity check
This is the basic check mandatory for all the modules, it checks for the following items and alerts the ones which do not meet the required criteria
License Audit:
- Specification public(OBPL) or commercial(OBCL) license of the module
- Appropriate license text for OBPL/OBCL
- Appropriate license link for OBPL/OBCL
- Author tag must have "Openbravo S.L.U."
- File Headers in module directories must have appropriate license text for OBPL/OBCL. Below is an example for OBCL
* Copyright (C) 2010 Openbravo S.L.U. * Licensed under the Openbravo Commercial License version 1.0 * You may obtain a copy of the License at http://www.openbravo.com/legal/obcl.html * or in the legal folder of this module distribution.
Other Checks:
- Help and Description columns should not be empty
- URL tag must point to module's project page in forge
- If a module is not the first version then update information tag must be filled
Note: It is necessary to take care of these items in modules development to avoid failure during integration process.
Compilation and Create OBX
This stage is to check for the source compilation and on success create obx. Below is the work flow of this stage
Prerequisite
- ERP source is made to point to
- Any of the branch in erp/devel (takes the latest changeset in the branch)
- If erp/devel/pi then it takes the latest MP
- Dependency modules can be obtained in two ways either from
- SCM (latest source of all dependency modules)
- CR (latest published version of dependency modules)
- Build the OBX on the minimum version of ERP_SOURCE supported by the module
- Compile and generate obx over the ERP version specified in AD_MODULE_DEPENDENCY.xml
- Invoke the creatOBX with minimal argument
Note: One can go for SCM option, When the AD_MODULE_DEPENDENCY.xml is specified with appropriate dependency list, else opt for OBX that are published in CR.
Process
- Workspace is updated with all the dependencies in $WORKSPACE/modules/
- Performs install.source
- Clone the main module to be monitored in to $WORKSPACE/modules/
- Performs smartbuild -Dlocal=no
- Finally generate obx
Now the obx is available to download from here
Database consistency check
This is to check the database consistency after including the module.
Process
- Set Core in development
- Run ant trl.clean and ant compile -Dtab=xx to run the translate with Core set in development
- Run the synchronize terminology process
- Run ant export.database -Dforce=yes
- Verify if there are differences and therefore if the database is consistent or not
JUnit test
Check for the JUnit inside module
Prequisite
- Modules must have the JUnit classes defined in "org.openbravo.module.modulename/src-test/org/openbravo/test/"
- build.xml to invoke the JUnit test in "org.openbravo.module.modulename/build.xml"
How to invoke JUnit in CI
On satisfying the perquisite above invoke the jUnit as below
Example: ant -f $WORKSPACE/erp_source/modules/$MOD_NAME/build.xml test
Install and Uninstall of module
On completing the stage-2 the obx is available with latest changeset of the module and ready to check the installation and uninstallation
Prerequisite
- ERP source is made to point to
- Any of the branch in erp/devel (takes the latest changeset in the branch)
- If erp/devel/pi then it takes the latest MP
- Dependency modules are obtained form CR or SCM as specified initialy
- Install the generated obx on the minimum version of ERP_SOURCE supported by the module
- Invoke the testModInstall with minimal argument
Process
- Workspace is updated with source and with all the dependencies in $WORKSPACE/modules/
- Download the dependency modules from CR or source repository in to $WORKSPACE/modules/
- Performs install.source, deploy the context and available to access via web UI.
- Invoke the selenium test target to install the module's obx via Module Management console(MMC)
- Invoke the selenium test target to un-install the module via MMC
Selenium test
Here is to run any selenium test for modules
Prerequisite
- Have the selenium class in your teams automation repo
- Have the test defined in "/src-test/com/openbravo/test/integration/erp/modules/<module_name>/"
- Have build.xml to invoke appropriate test defined
Process
- The modules in installed using the obx generated as in Install_and_Uninstall_of_module
- Context is made available with modules installed
- Invoke the selenium test suite.
Check Module Update
This check is performed when the module has at-least one published version in CR
Prerequisite
- ERP source is made to point to
- Any of the branch in erp/devel (takes the latest changeset in the branch)
- If erp/devel/pi then it takes the latest MP
- Test module update using the miimum version of ERP_SOURCE supported by the module as specified in AD_MODULE_DEPENDENCY.xml
- Invoke the creatOBX with minimal argument
Process
- Workspace is updated to have the latest ERP changeset
- Performs install.source, deploy the context and able to access on web interface.
- Install the module published in CR via MMC using selenium test suite
- Invoke the selenium test target to install the module's obx via Module Management console(MMC)
Create New Jobs in hudson CI for modules
New job
Template job is available in hudson, to create a new job
- Browse to modules.builds
- Hit New job
- Fill in the Job name as module name
- Select Copy existing job option and fill module _template
with this a new job will be created for your module
- Configure the job as in the following section
- Once configured click on Build Now to have first build of your module
Note: No space and dot are allowed in the job name, example of jobname assertreport, assest_report
Configuration
Now the job need to be configured to have details about the modules and frequency in which it has to be triggered etc
Browse to the new job configuration new job's configure window
Generic Settings
- Enable project-based security
- By default the job will have Anonymous user with Job read permission
- Add your teams account and enable permission (Delete,Configure,Read,Build,workspace)
- Tie this project to a node Master always
- Hudson in modules.builds is configured with two machines master and slave
- The stages like sanity check, createOBX, dbconsistency, JUnit will be done in master
- The selenium related test will be done in slave1
- Source Code Management
- Select Mercurial and fill the Repository URL with the repository path of the module to be monitored
- Build Triggers
- Select on Poll SCM, purpose of this is to trigger the job on certain frequency only when there is a new changeset available in module's repository.
Setting this is similar to cronjob setting Example: # To execute the job daily at 3 am 0 3 * * *
- Locks
- This feature is to lock the machines when one job is in execution status so that to avoid collusion between jobs
- By default the slave1 will be locked to avoid two parallel selenium jobs at a same time
Module specific settings
This is to set which is the script to be executed and what inputs to be given to the scripts Specify the module type either it is public or commercial module, please enable/comment the token accordingly
#export MOD_TYPE="public" #export MOD_TYPE="private"
Specify the Modules project name here as specified in forge and uncomment the token (this variable is required for sanity check).
#export MOD_PRO_NAME=
Specify the module you want to monitor, the name as specified in SCM
# Example : export MOD_NAME="org.openbravo.userinterface.selector" export MOD_NAME="org.openbravo.userinterface.selector"
Please specify which selenium repo for to be referred for selenium test classes.
export SMOKE_REPO=pi-ci
Please specify the source repo for ERP
export SOURCE_REPO=pi-engdev
Please specify the dependency type if you want to include. Uncomment the token appropriate for you. The dependency from source then specify as "scm", note this works only when AD_MODULE_DEPENDENCY.xml has all the dependency set init. If you want to pull the latest published version of modules then specify as 'obx'
#export DEP_TYPE="scm" #export DEP_TYPE="obx"
Please specify the project names of the modules as in forge, this is mantary when u specify OBX
# Example export DEPENDENT_MODULES="obsmartclient openbravoseam Openbravoclientkernel openbravoclientfreemarker DatasourceService openbravojsonrest"
Sanity check of the module.
sh /srv/hudson_setup/module_integration/sanity-check.sh
Database consistency check
sh /srv/hudson_setup/module_integration/dbconsistency
JUnit tests for module
ant -f $WORKSPACE/erp_source/modules/$MOD_NAME/build.xml test
Post build Action
- Email Notification
- Navigate to job configuration page -> Post build Action
- Enable Editable email notification
- Add your team's mailing list to Global Recipient List
- Click Advance and add trigger for failure and fixed to notify team on failure and build fixed
- Enable "Publish JUnit test result report", Specify the log-file path ad
SANDBOX/automation/pi/last-results/*/logs/*.xml
How to Debug
Here are simple guidelines on how to check the status of job and error logs
Build Failure
- Build/job failure will be shown as red ball and success with green ball
- Console output of the build can be used to check appropriate failure/success logs. This is similar we execute the scripts in command prompt. this is available for every build executed for a job. url format to check the log
http://modules.builds.openbravo.com/job/<JOB_NAME>/<BUILD_NUMBER>/consoleFull
Note: Replace the JOB_NAME and BUILD_NUMBER with your job
Selenium Failure
Selenium Failure here is the test like installation, un-installation, upgrade of modules are checked the via the ERP application web interface and we use selenium for it. The status of these test are recored using selenium test scripts. Below are how to check them
- Installation, un-installation, upgrade of modules are done using automation/pi-ci and its logs/screenshots can be accessed using the format
http://modules.builds.openbravo.com/job/<JOB_NAME>/ws/SANDBOX/automation/pi-engdev/last-results/
- Module related smoke test are done using your own automation repo in here and its logs/screenshots can be accessed using the format
http://modules.builds.openbravo.com/job/<job_name>/ws/SANDBOX/automation/<automation-repo>/last-results/
Script reference
- The scripts referred in configuration section is in SCM
- The selenium test classes need to appropriately placed in teams development branch in here
Next steps
- Automate module publishing process