MozillaZine

Testing pre-release versions

From MozillaZine Knowledge Base

Mozilla applications rely on volunteers to test them and to report problems. It is important for as many people as possible to test applications before they are released, using many different computing environments and combinations of preference settings, so that released versions will have fewer bugs.

This document is an overview of how you can participate.

Contents

[edit]

Applications and versions

A Mozilla Application is a software product. For example, Firefox is an application and Thunderbird is an application. Each application is built independently, even though applications may share code. Different versions of each application are built for different purposes:

Nightly (TB Daily) 
Snapshot versions mainly for developers working on the code. Although many nightlies work quite well, some of them might not work at all. They might have serious bugs that destroy your data or crash your computer, or both.
Aurora (TB Earlybird) 
Snapshot versions for initial public testing and usually produced nightly. Basic features certainly work, but some features might not work. Serious bugs are a possibility.
Beta 
Versions for wider public testing, released about weekly. Most features probably work, and serious bugs are less likely. Beta releases close to a scheduled release date can be considered release candidates (i.e., the last beta is close to or identical to the final release).
Release 
Final versions for general use, released every six weeks or so. If necessary, quick follow-up releases are posted to fix urgent security or stability issues with an initial release.
Extended Support Release (ESR) 
Final versions primarily intended for enterprise users. They are scheduled with every 7th regular release and feature-frozen with security updates for about a year.

Download test versions from these servers (which also support FTP):

Alphas and subsequent versions  http://releases.mozilla.org/
Nightlieshttp://ftp.mozilla.org/

To get the latest nightly, choose the application, then nightly, then scroll down past all the dates to the latest-trunk directory at the bottom.

Your choice of which versions to test depends on how involved you are with the code and on how much risk you are willing to take. If you are a developer, then you will almost certainly want to test nightlies. If you are not a developer, but you are comfortable backing up and restoring your data, then you can test alphas and subsequent versions. To receive notification of test builds of Thunderbird releases, subscribe to https://mail.mozilla.org/listinfo/thunderbird-testers

However, to contribute most to the quality of the final release, you should start testing early in the development process. For example, if you can test the code while developers are still working on it, then you have the best chance of influencing the design. But if you only test the last release candidate, then you might have to wait for the next release to see the changes you have suggested.

Notes: 

  • Nightlies and pre-release versions might use a code name instead of the normal application name. For information about Mozilla's code names, see: Internal project names
  • Nightlies may have fake version numbers that look like release version numbers.
[edit]

Installing multiple versions

You can participate in testing without affecting your normal day-to-day use of the application. To do this, you keep the latest final release version that you use day-to-day, and you install test versions separately.

To install a separate version of an application, perform a custom install specifying a different installation directory for each test version. You can leave the latest release version of the application installed in its normal location, so that you can continue to use it for day-to-day tasks.

Do not allow the installer to run the application automatically, unless you are certain that it will not try to use your day-to-day profile, possibly corrupting important data.

If you use a firewall program, then you must permit each new application version that you install to connect to the Internet. For more information, see: Firewalls

You can run more than one version of an application at the same time by adding the switch -no-remote to the command that you use to start the application (possibly in the properties of a shortcut icon). But doing this can be confusing, because it is easy to lose track of which version you are working with.

You can allow test versions to auto-update, so that you always have the latest test version. But if you are testing nightlies, then sometimes an update will introduce worse bugs than the previous nightly, so you might have to install an older version or give up testing until the bugs are fixed.

[edit]

Using profiles to manage risk

A pre-release version could corrupt or destroy data in your profile, delete messages on the mail server that it isn't supposed to, or corrupt a remote folder (IMAP account). Don't use a nightly or alpha build with important data. The safest approach is to use the Profile Manager to create a new profile that uses email accounts that don't have any important data, that you use just for testing.

Create a separate icon that launches the pre-release version of Thunderbird with the specified profile so that you don't accidentally use the pre-release version with your day-to-day profile.

If you want to use it with your normal accounts (not recommended) use a copy of your profile . If you decide to stop testing and want to migrate some of your data back to your normal profile, see:

[edit]

Using add-ons

You can install add-ons in your test profile, but using add-ons complicates testing, because some problems that you find might be caused by broken add-ons. Even so, it is useful to test any add-on that is important to you, so that you can report problems to the author of the add-on.

Mozilla provides add-on compatibility reporters for Firefox and Thunderbird .

See also: Updating add-ons

[edit]

Identifying and reporting bugs

Use Mozilla's Bugzilla database to work with bug reports. The bug reports there include enhancement requests in addition to reports about errors.

To contribute to Bugzilla, you must register there with a valid e-mail address.

If you find a possible bug:

  1. Search Bugzilla for an existing report. If you find an existing report, do not create a new report. Do not comment in an existing report unless you have new technical information that will help developers to resolve the bug.
  2. Ensure that you can reproduce the bug in a new profile with no add-ons. This is because the problem might be caused by some data in your profile, or by a broken add-on. If the problem is caused by a broken add-on, check the add-on's web page to find out whether the add-on is intended to support the version you are testing. If necessary, report the problem to the add-on's author. Do not report add-on bugs in Mozilla's Bugzilla database.
  3. Ensure that you can reproduce the bug in the latest version of the type that you are testing. For example, if you find a bug in a nightly, ensure that it is the latest nightly; if you find a bug in an alpha, beta or release candidate, ensure that it is the latest alpha, beta or release candidate.
  4. Identify the exact version that you are using, so that you can provide this information in the bug report. To find the version identifier, choose: Help – About ...
  5. Create a new bug report, specifying clearly how to reproduce the problem. If appropriate, attach sample data or screenshots to the bug report so that developers can understand exactly what goes wrong. Developers can only fix bugs that they can see for themselves.
  6. Watch for e-mail notification of changes to your bug report. If developers ask questions about the problem, reply if you can by adding a comment in Bugzilla (not by e-mail).

If you are unsure about any of these steps, see the next section for ways to get help.

[edit]

Getting help

For help with testing, and for general discussion about test versions, use the appropriate builds forum:

For other applications, use the appropriate general forum.

Some applications have web pages where you can find more information about testing, and some organize special times when you can participate together with other testers. For more information about the development and testing of each application, see the Mozilla Wiki and QMO.

[edit]

See also

[edit]

External links

  • Mozilla Wiki Mozilla project information
  • QA Quality Assurance at the mozilla developer center
  • QMO Mozilla quality assurance site
  • Bugzilla Mozilla's bug tracking database
Retrieved from "http://kb.mozillazine.org/Testing_pre-release_versions"