DevelopmentProcess/Launch

The purpose of the Release Launch phase is to stabilize a new release, as produced by the Construction process, to take its quality and robustness to production levels.

This publication process consists of two major cycles, alpha testing and beta testing, each preceded by an acceptance test.

Alpha Release

The purpose of an alpha phase is to provide testers with a full featured build of the release for evaluation purposes.

During the alpha phase, both the backlog of bugs coming from previous releases and bugs reported by alpha testers are fixed. The goal is to take the product to a level of quality that testers deemed sufficient for production deployment.

Alpha Release Acceptance Testing

Entry Criteria

1. A build of the product that includes 100% of the features targeted for the release.
2. The binary installers for all the supported platforms
3. Published Release Notes

Objectives

1. The software is available as a community appliance
2. The software can be installed from sources using both Oracle and PostgreSQL as databases
3. Users can upgrade from any maintenance pack version of the previous production release to the new software. If a minimum maintenance pack for upgrading is needed this must be defined upon entering the Alpha phase and must be communicated.
4. The software can be accessed using all supported browsers

Process Description

1. A week before the expected start of the Acceptance Test an open call for volunteers is sent through:
   • A blog post. See example.
   • A post in the Early Releases Discussion forum. See example.
   • A direct mail to all the partners from the Partner Management Director
   • A direct mail to participants in previous testing exercises from the QA Lead
2. As soon as the installer becomes available, the QA Lead gives volunteers early access to the installer through a private FTP server. Volunteers essentially receive the release at the same time as our QA team.
3. Volunteers are given access to the Acceptance Testing plans so that they have proper functional guidance (but they can test any flow they like as well).
4. All testers, both volunteers and QA staff, execute the test cases
   • During the execution testers are expected to report progress daily to the QA Lead.
   • The QA Lead publishes a daily status update on the Early Releases Discussion forum.
   • Questions, doubts and announcements are discussed on the Early Releases Discussion forum.
   • Testers are expected to report all found issues as bugs following the Bug Reporting Guidelines. Note: guidelines need to be updated.
5. In case critical issues are found, the bugs are corrected and a new installer is published and the process is repeated from step 2.
   • The timing of the publication of the new installer is decided by the QA Lead evaluating competing priorities:
   • Need to keep the testing environment as stable as possible in order to attempt as many test cases as possible before restarting (the goal is to identify as many issues as possible before a new installer is built)
   • Need to quickly get to the final installer.

Exit Criteria

• Successful installation and operation on all combinations on platforms, databases and browsers
• 100% execution of the acceptance test flows
• 0 critical open bugs

Expected Duration

• We aim at completing this process in 3 months.
• The actual duration may be shorter depending on the speed of stabilization.

Alpha Release Public Testing

Entry Criteria

1. A successful completion of the Alpha Release Acceptance Testing
2. It must be possible to upgrade from the previous production release

Objectives

1. Stabilize the product and bring it to a quality level where it is possible to use it for production purposes.
2. Validate 100% of the functionality.

Process Description

1. The Release Manager creates a public tag named after the release with the label alpha X, where X is a progressive number starting from one (example: 2.40 alpha 1, 2.40 alpha 2) and prepares the Univeral Installer based on that tag.
2. The QA Team executes an automated acceptance test and verifies all bugs that have been fixed since the previous build.
3. The release is posted on SourceForge in the Early Releases package and an announcement is posted to the Community to advice of its availability.
4. The QA Team executes the full set of test cases, exercising 100% of the functionality.
5. The QA Team also verifies all bugs fixes that have been previously verified.
6. The Engineering team focuses on fixing all classes of bugs:
   • Backlogs of bugs logged against previous releases
   • New bugs logged against this release
7. The goal is too fix as many bugs as possible, regardless of severity or risk.
   • The QA Lead publishes a daily status update on the Early Releases Discussion forum.
   • Questions, doubts and announcements are discussed on the Early Releases Discussion forum.
   • Testers are expected to report all found issues as bugs following the Bug Reporting Guidelines.
   • When a significant number of bugs have been fixed, a new alpha release is produced and the process restarts from step 1. The timing of this decision is dictated by the QA Lead:
   • Need to keep the testing environment as stable as possible in order to attempt as many test cases as possible before restarting (the goal is to identify as many issues as possible before a new installer is built)
   • Need to quickly get to the final installer.

Exit Criteria

• Successful installation and operation on all combinations on platforms, databases and browsers
• 100% execution of the all test cases
• 100% of fixed bugs are verified
• out of all defects logged within 2 weeks before the release date
  • 0 open defects are critical
  • 0 open defects are severe
  • less than 100 open defects are minor or trivial

Expected Duration

• We aim at completing this process in 4 weeks.
• The actual duration might vary depending on the speed of stabilization.

Beta Release

Entry Criteria

1. A successful completion of the Alpha Release Public Testing

Objectives

1. Prove that the product can be successfully deployed in a production environment

Process Description

1. The Release Manager creates a public tag named after the release with the label beta X, where X is a progressive number starting from one (example: 2.40 beta 1, 2.40 beta 2) and prepares the Univeral Installer based on that tag.
2. The QA Team executes an automated acceptance test and verifies all bugs that have been fixed since the previous build.
3. The release is posted on SourceForge in the Early Releases package and an announcement is posted to the Community to advice of its availability.

Exit Criteria

1. Successful smoke test
2. Completed testing of existing features following the extended test cases
3. Where historically no test plans were available tests have been carried out by experience of the QA team
4. New feature were tested according to supplied test plans
5. Fixed defects have been verified

Production Release

Entry Criteria

• 1 customer running Beta in production for at least 2 weeks

Objectives

1. To release the finished product into the market in good quality.

Process Description

• to be defined

Exit Criteria

• to be defined

The Iterative Nature of the Publication Process

Please notice that the Publication Process is inherently iterative as failure to achieve the exit criteria of every step (acceptance tests, public alpha and public beta) will result in a new release being created and the current phase to be restarted.

As such, it is likely that there will be more than one alpha release (alpha 1, alpha 2, etc.) and more than one beta release (beta 1, beta 2, ...).

It is also important to notice that the last release of one phase is exactly the same as the first release of the next phase:

• Alpha N (the release that successfully completes the public alpha cycle) is the same build as Beta 1
• Beta N (the release that successfully completes the public beta cycle) is the same build as the production release.