DevelopmentProcess/Launch

Overview

The Release Launch phase covers two aspects of the development process:

• Publication
• Collateral Development

Publication

The purpose of the publication process is to stabilize a new release, as produced by the Construction process, to take its quality and robustness to production levels.

The publication process consists of two major cycles, alpha testing and beta testing, each preceded by acceptance test.

Alpha Release

The purpose of an alpha phase is to provide testers with a full featured build of the release for evaluation purposes.

During the alpha phase, both the backlog of bugs coming from previous releases and bugs reported by alpha testers are fixed. The goal is to take the product to a level of quality that testers deemed sufficient for production deployment.

The alpha release is intended for testing purposes only and as such no upgrade path is provided either to or from the release.

Alpha Release Acceptance Testing

Entry Criteria

1. A build of the product that includes 100% of the features targeted for the release.
2. The binary installers for all the supported platforms
3. Published Release Notes

Objectives

1. The software can be installed on all major platforms using the Universal Installer using both Oracle and PostgreSQL as databases
2. The software can be installed from sources using both Oracle and PostgreSQL as databases
3. The software is operational on all major platforms using the Universal Installer using both Oracle and PostgreSQL as databases
4. The software can be accessed using all supported browsers

Process Description

1. A week before the expected start of the Acceptance Test an open call for volunteers is sent through:
   • A blog post. See example.
   • A post in the Early Releases Discussion forum. See example.
   • A direct mail to all the partners from the Partner Management Director
   • A direct mail to participants in previous testing exercises from the QA Lead
2. As soon as the installer becomes available, the QA Lead gives volunteers early access to the installer through a private FTP server. Volunteers essentially receive the release at the same time as our QA team.
3. Volunteers are given access to the Acceptance Testing plans so that they have proper functional guidance (but they can test any flow they like as well).
4. All testers, both volunteers and QA staff, execute the test cases
   • During the execution testers are expected to report progress daily to the QA Lead.
   • The QA Lead publishes a daily status update on the Early Releases Discussion forum.
   • Questions, doubts and announcements are discussed on the Early Releases Discussion forum.
   • Testers are expected to report all found issues as bugs following the Bug Reporting Guidelines. Note: guidelines need to be updated.
5. In case critical issues are found, the bugs are corrected and a new installer is published and the process is repeated from step 2.
   • The timing of the publication of the new installer is decided by the QA Lead evaluating competing priorities:
   • Need to keep the testing environment as stable as possible in order to attempt as many test cases as possible before restarting (the goal is to identify as many issues as possible before a new installer is built)
   • Need to quickly get to the final installer.

Exit Criteria

• Successful installation and operation on all combinations on platforms, databases and browsers
• 100% execution of the acceptance test flows
• 0 critical open bugs

Expected Duration

• We aim at completing this process in 2 weeks.
• The actual duration might vary depending on the speed of stabilization.

Alpha Release Public Testing

Entry Criteria

1. A successful completion of the Alpha Release Acceptance Testing

Objectives

1. Stabilize the product and bring it to a quality level where it is possible to use it for production purposes.
2. Validate 100% of the functionality.

Process Description

1. The Release Manager creates a public SVN tag named after the release with the label alpha X, where X is a progressive number starting from one (example: 2.40 alpha 1, 2.40 alpha 2) and prepares the Univeral Installer based on that tag.
2. The QA Team executes an automated acceptance test and verifies all bugs that have been fixed since the previous build.
3. The release is posted on SourceForge in the Early Releases package and an announcement is posted to the Community to advice of its availability.
4. The QA Team executes the full set of test cases, exercising 100% of the functionality.
5. The QA Team also verifies all bugs fixes that have been previously verified.
6. The Engineering team focuses on fixing all classes of bugs:
   • Backlogs of bugs logged against previous releases
   • New bugs logged against this release
7. The goal is too fix as many bugs as possible, regardless of severity or risk.
   • The QA Lead publishes a daily status update on the Early Releases Discussion forum.
   • Questions, doubts and announcements are discussed on the Early Releases Discussion forum.
   • Testers are expected to report all found issues as bugs following the Bug Reporting Guidelines.
   • When a significant number of bugs have been fixed, a new alpha release is produced and the process restarts from step 1. The timing of this decision is dictated by the QA Lead:
   • Need to keep the testing environment as stable as possible in order to attempt as many test cases as possible before restarting (the goal is to identify as many issues as possible before a new installer is built)
   • Need to quickly get to the final installer.

Exit Criteria

• Successful installation and operation on all combinations on platforms, databases and browsers
• 100% execution of the all test cases
• 100% of fixed bugs verified
• 0 open critical bugs
• 0 open severe bugs
• less than 50 open minor bugs
• less than 50 open trivial bugs

Expected Duration

• We aim at completing this process in 4 weeks.
• The actual duration might vary depending on the speed of stabilization.

Beta Release

Entry Criteria

1. A successful completion of the Alpha Release Public Testing
2. Availability of the Upgrader from the previous production release

Objectives

1. Prove that the product can be successfully deployed in a production environment

Process Description

1. The Release Manager creates a public SVN tag named after the release with the label beta X, where X is a progressive number starting from one (example: 2.40 beta 1, 2.40 beta 2) and prepares the Univeral Installer based on that tag.
2. The QA Team executes an automated acceptance test and verifies all bugs that have been fixed since the previous build.
3. The release is posted on SourceForge in the Early Releases package and an announcement is posted to the Community to advice of its availability.
4. To be completed...

Production Release

To be completed...

The Iterative Nature of the Publication Process

Please notice that the Publication Process is inherently iterative as failure to achieve the exit criteria of every step (acceptance tests, public alpha and public beta) will result in a new release being created and the current phase to be restarted.

As such, it is likely that there will be more than one alpha release (alpha 1, alpha 2, etc.) and more than one beta release (beta 1, beta 2, ...).

It is also important to notice that the last release of one phase is exactly the same as the first release of the next phase:

• Alpha N (the release that successfully completes the public alpha cycle) is the same build as Beta 1
• Beta N (the release that successfully completes the public beta cycle) is the same build as the production release.

Collateral Development

At the same time as the release goes through stabilization, a number of collaterals need to be produced in order to make available all the information required for a successful deployment to the Community.

In particular, the following documentation needs to be updated:

• On line help
• User Documentation
• Functional Documentation
• Installation Guide
• ER Diagrams

Additionally, the following software artifacts need to be developed:

• Upgrader: the tool that allows users of the previous release to move to the current one. The Upgrader needs to be developed before the completion of the Alpha cycle in order to allow existing customers to start the beta cycle.

Finally, the following dissemination artifacts need to be developed:

• Demo scripts for the most important new functionality
• Demo recordings (viewlets) of the most important new functionality (optional)