Apache Flex® - Unit 16 - Allowing Your Tests to Function with Continuous Integration

Download Unit Project Files

Continuous integration testing is another necessity of the complex application development environment. Generally, when many people and external forces interact with a deployed or actively developed application, quality control measures must be in place for the application's full functionality. Continuous integration systems should run application tests automatically, and some can be configured to do so.

Objectives:

After completing this lesson, you should be able to:

Setup FlexUnit project builds with Apache Ant
Run FlexUnit tests within the Hudson continuous integration environment

Topics

In this unit, you will learn about the following topics:

Understanding continuous integration
Examining a build script
Hudson
Understanding listeners
Failure and Success reporting options

Understanding continuous integration

Continuous Integration is a software development practice where members of a development team integrate their source code changes frequently; typically each developer integrates their changes at least once per day. This practice leads to multiple integrations throughout the day and with every integration the changes are verified by an automated build (which includes testing) to detect integration errors as quickly as possible.

Many tools exist to support a CI environment and make the process automated. Some of the common ones include:

Hudson
Cruise Control
Continuum

CI systems generally include a source repository, an automated build including build script, self testing code, joint commit by developers, an integration machine and, frequently, automated deployment.

CI builds commonly use the following process:

Developers run local builds and tests on code changes.
Developers check in code into the main repository.
An automated build server monitors the repository.
New code is checked into a project build and continuously integrated and tested.
Test results are available to all developers on the project and reported in real time.

Understanding listeners

Listeners act as a communication between the test runners and a reporting device. If a runner acts as the worker, the listener acts as the foreman, reporting the results of the workers labors.

All listeners implement the IRunListener interface. FlexUnit 4 may use any number of listeners, each will be called as a test completes. It is also possibly to write custom listeners further extending the framework.

There are several base listeners included in FlexUnit.

UIListener

A separate library which allows visual reporting of results
Accepts user interaction
Gives detailed information about test run times, passing, failing, ignores and failure messages

CIListener

Separate library listener project
Creates a socket connection to communicate results over a connection
As part of CI, communicates and reports results to the FlexUnit Ant task

XMLListener

Creates an XML report for tests
Primarily used by the Flashbuilder FlexUnit plugin
Can be customized to report XML across different sockets

Walkthrough 1: Using the CIListener

In this walkthrough you will perform the following tasks:

Edit the application to use the CIListener

Adding the CIListener

Import the FlexUnit4Training.fxp project from the Unit 16/Start folder. Please refer to Unit 2: Walkthrough 1 for instructions on importing a Flash Builder project.
Open the FlexUnit4Training.mxml file.
Find and remove the FlexUnitTestRunnerUI component with id testRunner from the application file.

You are removing this section of code as you will no longer be using the UIRunner.
~~<flexui:FlexUnitTestRunnerUI id="testRunner"/>~~
Continuous integration servers are autonomous. In addition to creating unnecessary overhead, a UIRunner prevents FlexUnit from running in a headless environment.
Within the <fx:Script> block, there is a method named onCreationComplete(). Remove the body of this method, leaving just the empty shell of the method.
```
private function onCreationComplete():void {
}
```
Within the <fx:Script> block add a new public variable named core of type FlexUnitCore.
```
public var core:FlexUnitCore;
```
If you did not use code-completion, add the import statement for and org.flexunit.runner.FlexUnitCore at this time.

The core acts as our runner; we will be adding the CIListener to the core listeners.
In the onCreationComplete() method instantiate the core variable to a new FlexUnitCore(). Call the core.addListener() method on the next line, passing it an argument of new CIListener(). On the next line call the core.run( currentRunTestSuite() ).
```
private function onCreationComplete():void {
	core = new FlexUnitCore();
	core.addListener( new CIListener() );
}
```
On the following line, call core.run() passing currentRunTestSuite() as its argument.
```
private function onCreationComplete():void {
	core = new FlexUnitCore();
	core.addListener( new CIListener() );
	core.run( currentRunTestSuite() );
}
```
If you did not use code-completion, add the import statement for org.flexunit.listeners.CIListener at this time.
Save this mxml file. Do not run it at this time.

Examining an Ant build script

A build script is an XML configuration file to run compiling and, in some cases, testing of the app. The build file acts as an instruction manual to the CI server on what files to include and what tests to run. It may also contain additional behaviors.

Scripts are made up of a project, optional properties, optional task definitions and a series of 'targets' which must include at least one default target.

Project tag

Acts as the build header
name - name of the project
default - the default target, or command, to run when the build is run
basedir - Base directory for all work. If none is specified, Ant uses the build file directory as base directory
Project tag structure

<project name=<i>"project_name"</i> basedir=<i>"project_base"</i> default=<i>"default_target"</i> >

Property tag

Frequently used to create shorthand variables for directory locations
Also used to specify directories of task targets
Property tag structure

<property name=<i>"property_name"</i> location=<i>"property_location"</i> />

Taskdef tag

Tasks define the custom behavior to be used with the build
Frequently built into a jar file
The taskdef tag defines the location of a task within the jar
Task definition tag structure

<taskdef resource=<i>"task_location_within_jar"</i> classpath=<i>"task_jar_location"</i> />

Target tag

Creates possible start points for where the build should begin
Targets may depend on other targets
The compile target, common in most Flex builds, will depend on directory construction
In the case of testing, the test target will depend on the application being compiled
If a target has dependencies, the dependencies will be run when the target is run
Target tag structure

<target name=<i>"target_name"</i> depends=<i>"target_dependencies"</i>>

Task

Each task is a job or file to be run as part of the build
A task is a custom piece of code to be executed during the build
There are many common tasks included with Ant
We are also concerned with the compile task, FlexUnit task and delete task
Each task may have its own unique properties
Common task tag structure

<taskname id=<i>"task_id"</i> attribute1=<i>"..."</i> attribute2=<i>"..."</i> ... />

For building projects with Hudson we will be using 6 tasks, Flex Ant task, FlexUnit task, JUnit report task, copy task, zip task and delete task
The flex ant task consists of the following tasks

mxmlc - Application compiler for SWFs
compc - Component compiler for SWFs and RSLs
html-wrapper - Generates wrapper and supporting files
asdoc - Generates ASDoc output

The flexunit task which consists solely of the flexunit task
The junitreport task, which allows viewing of JUnit style reports
The delete task, which will be used as a clean target
Copy task, for copying files
Zip task, for zipping files

Hudson

A popular, open-source, continuous integration server, Hudson is simple to set up, configure and use. It can monitor multiple jobs, and through use of the FlexUnitAntTask, can make automating testing virtually a painless process.

In order to use Hudson, it requires that Ant and/or Maven be installed on the CI system. It may be deployed through the .war file to an Apache, Tomcat, Glassfish or other HTTP server project. Hudson may also be used by way of the Windows service on a Windows machine, or integrated directly with many Linux distributions.

For more information about Hudson and its capabilities, visit:

https://hudson-ci.org

Failure and Success reporting options

The FlexUnit Ant Task allows several options for test reporting

haltonfailure=true|false

If true, when a failure or error is encountered will complete all currently executing tests then cancel the run

verbose=true|false

If true, reports the complete failure message for each failed test
If false, reports number of tests passed and failed only

port=<port results will be reported on, default is CIListener port>
timeout=<length of time to wait for runner to send results before considering test a failure>
failureproperty=<property name to set to true if any tests fail>

Walkthrough 2: Running your tests from Hudson test launch

In this walkthrough you will perform the following tasks:

Prepare a debug projector
Prepare an Ant build file
Install Ant and Hudson
Setup your environment to run tests from Hudson

Import the CI Project

Import the FlexUnitTraining_wt2.fxp contained in Unit 16/Start folder.

Please refer to Unit 2: Walkthrough 1 for instructions on importing a Flash Builder project.

Preparing the debug projector
Navigate to Unit 16/FlexUnitCI and find the file flashplayer_sa_win.exe (Windows users) or flashplayer_10_sa_debug.app.zip (Mac users). This is the debug version of the standalone FlashPlayer. Copy it to your local file system somewhere you will remember.

In this walkthrough, you will associate swf files with the standalone, debug player. Failure to associate SWF files with a standalone FP will cause Ant builds to fail. This requires administrator privileges.
Windows Users: Return to the FlexUnitCI folder. Right-click on the TestSwf.swf and select Open with.... If an expanding menu appears, instead click choose program. This should open the following screen:

Depending on your version, the dialog may appear slightly different. Click on Browse... and select the standalone player you saved earlier. Click OK. The file will attempt to open and throw a SecurityError. This is normal. Click Dismiss and close the player.

Mac Users: Return to the FlexUnitCI folder. Right-click, or if using a single mouse button control-click, on the file TestSwf.swf and select Get Info. In this dialogue you will see a section titled Open With.

Select the drop down menu and click Other... When the browser box opens, navigate to the location you saved the debug player and select that file.

Prepare the Ant build file

Continuous integration projects require a build file. Since you will be using Ant along with Hudson, you need a build.xml file.
Open the build.xml.

You will need to make some modifications to this file for it to work with your project build. You will see a heading that reads: Setup paths for build.

If you changed the organization of your project, make sure these paths reflect those changes.

You will also see a heading . Here you will need to modify the path locations of your tasks as well as set the FLEX_HOME.
First, navigate to your Flex SDK. In most cases, your Flex SDK should be located at root/Program Files/Adobe/Adobe Flash Builder 4/sdks.
```
<property name="FLEX_HOME" 
 location="rootpath:/Program Files/Adobe/Adobe Flash Builder 4/sdks/4.1.0/" />
```
It is highly recommended you use Flex 4.1 SDK. Keep every directory in this walkthrough relative to the FLEX_HOME directory, this way the build file is guaranteed to reference these correctly.

Install Ant and Hudson
Navigate to Unit 16/FlexUnitCI. Copy this entire directory to the root drive.

You may copy it to another location; however this guide assumes you have copied it to this location. If you do not, be sure to adjust all directory references to your installed location.
Navigate to the URL: https://hudson.dev.java.net/hudson.jnlp
Choose to download and run the hudson.jnlp file.

After the Java Web Start has completed you should see a startup dialog:

This is the Hudson test drive. For our purposes this is sufficient. However, if you are running Hudson through your CI machine you will want to install the windows service or console app depending on your OS.
Open your browser and navigate to the http://localhost:8080/

You should see the Hudson start page.
On the left is a menu of options. Click on Manage Hudson and navigate to Configure System. Here, you will need to specify the Ant location.
Go to the sub heading Ant and click the Add Ant button. For the name, specify 'FlexUnit Intro Ant'. Uncheck Install automatically.
In the ANT_HOME field, specify the install directory of your Ant installation. If you used the walkthroughs location, specify 'root/FlexUnitCI/Ant'.
Navigate to the bottom of the page and click Save.
On the left hand menu, click New Job.
Give the job a name of FlexUnitCI Intro. Select Build a free-style software project. This version will allow you to build directly from an Ant script.

You will now see the new project page. You may at this time assign a description to the project. Take note of the Source Code Management settings. The SCM is what will turn our build into a continually integrating project. Normally we would specify our SCM and pass the location of the SCM. Hudson would then be pushed any changes to the repository and immediately execute a build. However, since we do not have any SCM for this project we will be executing manual builds.
Leave this option as None.
Under the heading Build, add a new build step. Click on the drop down and select Invoke Ant. For the Ant Version select the FlexUnit Intro Ant. Under Targets enter 'test'. This will cause Hudson to run the test target every time a build is run.
Click Save.
You should now be taken to the project homepage. Right now this project will do nothing. You need to specify a workspace. To do so, select Workspace from the main screen. This can also be accessed from the menu on the left.

You will receive an error:

This is completely expected. Hudson needs to execute a new build to configure the workspace.
Select Run a build.

You should see new Build History menu on the left, it will update with the new build. At completion, this build should fail (red dot indicates failure). The build will fail because the current target does not exist. However, the workspace setup is now complete.
Click on the workspace link in the left-hand menu again.

You should now see:

Hudson is expecting the base directory of your project which should also contain the build.xml. Unfortunately, since we are not using a SCM, this directory will not populate correctly. We need to add the files directly to the workspace.
Windows: Navigate to: C:/Documents and Settings/<username>/.hudson/jobs/FlexUnitCI Intro/workspace

Mac: Navigate to: Users/<username>/.hudson/jobs/FlexUnitCI Intro/workspace
This is the default location of the test launch workspace. Copy all files contained within the FlexUnit4Training project directory into this directory.
Refresh the Hudson page in your browser.
Hudson should now show the current contents of your project.

The project is now ready to be built. Since we are running this from the Test Launch and do not have a SCM specified, we will need to run manual builds.
Select Build Now from the menu on the left.

Build now will force an immediate build on any files currently in the workspace. This should return a success (blue dot on the left under build history.)

During test execution, you will see the standalone player load with a blank screen. This is normal and will close automatically. Do not close this manually or your build may fail.
Click on build #2.

This is the details view of the build. Since we have verbose set to true in the build file, we can view the details of each run.
Click on Console Output. Here will print detail of each test. Failures will also appear here.

Since JUnit reports were created for the test run, you may also view these.
Click on Back to Project in the left hand menu. Then click on Workspace. This will open the current workspace setup. Click on target then report then html. Select index.html to see the detailed JUnit report.
Congratulations, you have now completed your first Hudson test run.

Hudson will not auto build when new changes are made to the main project. Because we had to copy the build files manually any new changes will also need to be manually copied. This is not necessary in a server version where a SCM tool has been specified for the project. Hudson will pull any new changes and create a new build automatically.