Release Management/Smoke test with Hudson
Contents |
Introduction
This document describes what is Continuous Integration (CI), why we need it, what is Hudson and why we use it. What is Openbravo ERP smoke test, How QA develop and manage it, and finally how smoke test can be integrated to Hudson and its benefits.
Continuous Integration
The term "continuous integration" refers to a process that builds and tests the code under development phase on a frequent basis. Vision of CI is to assists development teams and QA team with the following challenges:
- Software build automation (on-demand compilation)
- Continuous automated build verification (scheduled compilation, like for every commit trigger a build / daily / weekly / monthly)
- Continuous automated build testing (schedule and execute test suits [junit/smoke])
- Post-build procedure automation (product packaging / deploy to other QA/customer machines / create artifacts in SCM)
Fore More info on Continuous Integration check here
Hudson
Hudson is a Continuous Integration tool written in Java. Helps to repeat the same steps we do in our day to day build/test. The advantage here is Hudson tool records/stores the steps and triggers them on every execution (basically process automation) and we only monitory the results and do the need full action on failure.
How QA develop and manage Smoke test
Development
As Openbravo ERP is subject to change with its feature and process, smoke test automation scripts are maintained for every version
when ever a ERP update found to affect smoke test automation results, a fix will be done to smoke test script and committed to appropriate repositories
Daily Test
As ERP SCM is updated daily with developers commit QA performs daily test over the application to track on bugs early
- Do Compilation on update ERP source
- ant update.database
- ant core.lib wad.lib trl.lib
- ant compile.complete.development
- On successful builds, update the smoke automation scripts and execute "ant test.integration.smoke"
- Test results are notified to staff.qa mailing list
- Check for the reports, on failure report appropriate team based on error
- Smoke scripts failure - report to QA and fix the script
- ERP related failure - report to openbravo ERP development team
- Environmental failure (tomcat, database etc) - report to Release Management team
Smoke test in Hudson
System Specification
Smoke test are set in High-CPU Medium Instance which has about
- 1.7GB RAM,
- 5 EC2 Compute Unit,
- 350 GB storage,
- 32-bit platform,
- moderate I/O Performance,
- Cost $0.20/hour.
- operating system is Ubuntu (jaunty)
One shared oracle instance for all versions of ERP-smoke test
Work Flow
Initial Setup
Configure the smoke test through Hudson's configure UI
- Create new job note: Job's name becomes the workspace directory name
- Specify the SCM URL [sample: http://code.openbravo.com/erp/devel/main]
- set Openbravo.proprties
- Do compilation "ant install.source"
- Checkout smoke automation scripts to </path/to/workspace/smoke_test>
- Set </path/to/workspace/smoke_test/config/OpenbravoERPTest.proprties>
- Connect to VNC server console and trigger smoke test "DISPLAY:1 ant test.integration.smoke"
- Enable junit test report, specify the log path as smoke_test/log/*.xml
Daily Execution
- Update the ERP workspace to the latest with repository
- Do ant install.source
- On successful compilation, Update the smoke test automation scripts
- Connect to VNC server console DISPLAY:1 of the smoke test machine
- Trigger ant test.integration.smoke
- Generate junit report sample test results trend
- Notify to staff.qa and staff.rm mailing list
Advantages
- Smoke are run through the VNC server console and we can secure ssh into appropriate system to check the actual flow.
- Build Trends are kept in track, which helps the developer/management to walk through the percentage of success/failure.
- Test Trends are kept in track, which helps QA to walk through the percentage of success/failure.
- captures console out put, which gets updated very second and displayed over the master web UI, this makes us feel working on command-line environment
- Configure the job on the fly. where you can specify
- Notification on builds results (irc, mailing list, individual commiter, etc)
- Scheduling are done as same as cron jobs
- Smoke test machine can be added to Hudson as slave and can be made to report the results to builds.openbravo.com.
How to Debug Errors
On smoke test failure what we do
- Check for the log in Hudson test results screen
- Connect to VNC and check by just execute the failed test class using -Dclass option
Forward yout localhost port to smoketest machine ssh <smoke_test_machine_IP> -p <ssh-port> -L 5901:localhost:5901
View the console vncviewer -encodings "copyrect tight hextile" localhost:01 -compresslevel 7 -quality 5
- Execute the test case manually through ERP application UI of the appropriat econtext. You can browser into appropriate instance like http://79.125.47.198:8080/openbravo_<version>_smoke_<database>