Unit 16 - Allowing Your Tests to Function with Continuous Integration

Download Download Unit Project Files

Continuous integration testing is another necessity of the complex application development environment. Generally, when many people and external forces interact with a deployed or actively developed application, quality control measures must be in place for the application's full functionality. Continuous integration systems should run application tests automatically, and some can be configured to do so.

Objectives:

After completing this lesson, you should be able to:

Topics

In this unit, you will learn about the following topics:

Understanding continuous integration

Continuous Integration is a software development practice where members of a development team integrate their source code changes frequently; typically each developer integrates their changes at least once per day. This practice leads to multiple integrations throughout the day and with every integration the changes are verified by an automated build (which includes testing) to detect integration errors as quickly as possible.

Many tools exist to support a CI environment and make the process automated. Some of the common ones include:

CI systems generally include a source repository, an automated build including build script, self testing code, joint commit by developers, an integration machine and, frequently, automated deployment.

CI builds commonly use the following process:

Understanding listeners

Listeners act as a communication between the test runners and a reporting device. If a runner acts as the worker, the listener acts as the foreman, reporting the results of the workers labors.

All listeners implement the IRunListener interface. FlexUnit 4 may use any number of listeners, each will be called as a test completes. It is also possibly to write custom listeners further extending the framework.

There are several base listeners included in FlexUnit.

UIListener

CIListener

XMLListener

Walkthrough 1: Using the CIListener

In this walkthrough you will perform the following tasks:

Adding the CIListener

  1. Import the FlexUnit4Training.fxp project from the Unit 16/Start folder. Please refer to Unit 2: Walkthrough 1 for instructions on importing a Flash Builder project.

  2. Open the FlexUnit4Training.mxml file.

  3. Find and remove the FlexUnitTestRunnerUI component with id testRunner from the application file.

    You are removing this section of code as you will no longer be using the UIRunner.

    <flexui:FlexUnitTestRunnerUI id="testRunner"/>

    Continuous integration servers are autonomous. In addition to creating unnecessary overhead, a UIRunner prevents FlexUnit from running in a headless environment.

  4. Within the <fx:Script> block, there is a method named onCreationComplete(). Remove the body of this method, leaving just the empty shell of the method.

    private function onCreationComplete():void {
    }
  5. Within the <fx:Script> block add a new public variable named core of type FlexUnitCore.

    public var core:FlexUnitCore;

    If you did not use code-completion, add the import statement for and org.flexunit.runner.FlexUnitCore at this time.

    The core acts as our runner; we will be adding the CIListener to the core listeners.

  6. In the onCreationComplete() method instantiate the core variable to a new FlexUnitCore(). Call the core.addListener() method on the next line, passing it an argument of new CIListener(). On the next line call the core.run( currentRunTestSuite() ).

    private function onCreationComplete():void {
        core = new FlexUnitCore();
        core.addListener( new CIListener() );
    }
  7. On the following line, call core.run() passing currentRunTestSuite() as its argument.

    private function onCreationComplete():void {
        core = new FlexUnitCore();
        core.addListener( new CIListener() );
        core.run( currentRunTestSuite() );
    }

    If you did not use code-completion, add the import statement for org.flexunit.listeners.CIListener at this time.

  8. Save this mxml file. Do not run it at this time.

Examining an Ant build script

A build script is an XML configuration file to run compiling and, in some cases, testing of the app. The build file acts as an instruction manual to the CI server on what files to include and what tests to run. It may also contain additional behaviors.

Scripts are made up of a project, optional properties, optional task definitions and a series of 'targets' which must include at least one default target.

Project tag

<project name="project_name" basedir="project_base" default="default_target" >

Property tag

<property name="property_name" location="property_location" />

Taskdef tag

<taskdef resource="task_location_within_jar" classpath="task_jar_location" />

Target tag

<target name="target_name" depends="target_dependencies">

Task

<taskname id="task_id" attribute1="..." attribute2="..." ... />

Hudson

A popular, open-source, continuous integration server, Hudson is simple to set up, configure and use. It can monitor multiple jobs, and through use of the FlexUnitAntTask, can make automating testing virtually a painless process.

In order to use Hudson, it requires that Ant and/or Maven be installed on the CI system. It may be deployed through the .war file to an Apache, Tomcat, Glassfish or other HTTP server project. Hudson may also be used by way of the Windows service on a Windows machine, or integrated directly with many Linux distributions.

For more information about Hudson and its capabilities, visit:

http://hudson-ci.org

Failure and Success reporting options

The FlexUnit Ant Task allows several options for test reporting

Walkthrough 2: Running your tests from Hudson test launch

In this walkthrough you will perform the following tasks:

Import the CI Project

  1. Import the FlexUnitTraining_wt2.fxp contained in Unit 16/Start folder.

    Please refer to Unit 2: Walkthrough 1 for instructions on importing a Flash Builder project.


    Preparing the debug projector

  2. Navigate to Unit 16/FlexUnitCI and find the file flashplayer_sa_win.exe (Windows users) or flashplayer_10_sa_debug.app.zip (Mac users). This is the debug version of the standalone FlashPlayer. Copy it to your local file system somewhere you will remember.

    In this walkthrough, you will associate swf files with the standalone, debug player. Failure to associate SWF files with a standalone FP will cause Ant builds to fail. This requires administrator privileges.

  3. Windows Users: Return to the FlexUnitCI folder. Right-click on the TestSwf.swf and select Open with.... If an expanding menu appears, instead click choose program. This should open the following screen:

    WindowsOpenWith

    Depending on your version, the dialog may appear slightly different. Click on Browse... and select the standalone player you saved earlier. Click OK. The file will attempt to open and throw a SecurityError. This is normal. Click Dismiss and close the player.

    Mac Users: Return to the FlexUnitCI folder. Right-click, or if using a single mouse button control-click, on the file TestSwf.swf and select Get Info. In this dialogue you will see a section titled Open With.

    MacOpenWith

    Select the drop down menu and click Other... When the browser box opens, navigate to the location you saved the debug player and select that file.


    Prepare the Ant build file

    Continuous integration projects require a build file. Since you will be using Ant along with Hudson, you need a build.xml file.

  4. Open the build.xml.

    You will need to make some modifications to this file for it to work with your project build. You will see a heading that reads: Setup paths for build.

    If you changed the organization of your project, make sure these paths reflect those changes.

    BuildPath

    You will also see a heading <!--Setup Flex and FlexUnit ant tasks -->. Here you will need to modify the path locations of your tasks as well as set the FLEX_HOME.

  5. First, navigate to your Flex SDK. In most cases, your Flex SDK should be located at root/Program Files/Adobe/Adobe Flash Builder 4/sdks.

    <property name="FLEX_HOME" 
     location="rootpath:/Program Files/Adobe/Adobe Flash Builder 4/sdks/4.1.0/" />

    It is highly recommended you use Flex 4.1 SDK. Keep every directory in this walkthrough relative to the FLEX_HOME directory, this way the build file is guaranteed to reference these correctly.


    Install Ant and Hudson

  6. Navigate to Unit 16/FlexUnitCI. Copy this entire directory to the root drive.

    You may copy it to another location; however this guide assumes you have copied it to this location. If you do not, be sure to adjust all directory references to your installed location.

  7. Navigate to the URL: https://hudson.dev.java.net/hudson.jnlp

  8. Choose to download and run the hudson.jnlp file.

    After the Java Web Start has completed you should see a startup dialog:

    HudsonConsole

    This is the Hudson test drive. For our purposes this is sufficient. However, if you are running Hudson through your CI machine you will want to install the windows service or console app depending on your OS.

  9. Open your browser and navigate to the http://localhost:8080/

    You should see the Hudson start page.

  10. On the left is a menu of options. Click on Manage Hudson and navigate to Configure System. Here, you will need to specify the Ant location.

  11. Go to the sub heading Ant and click the Add Ant button. For the name, specify 'FlexUnit Intro Ant'. Uncheck Install automatically.

  12. In the ANT_HOME field, specify the install directory of your Ant installation. If you used the walkthroughs location, specify 'root/FlexUnitCI/Ant'.

  13. Navigate to the bottom of the page and click Save.

  14. On the left hand menu, click New Job.

  15. Give the job a name of FlexUnitCI Intro. Select Build a free-style software project. This version will allow you to build directly from an Ant script.

    NewProjectPage

    You will now see the new project page. You may at this time assign a description to the project. Take note of the Source Code Management settings. The SCM is what will turn our build into a continually integrating project. Normally we would specify our SCM and pass the location of the SCM. Hudson would then be pushed any changes to the repository and immediately execute a build. However, since we do not have any SCM for this project we will be executing manual builds.

  16. Leave this option as None.

    None
  17. Under the heading Build, add a new build step. Click on the drop down and select Invoke Ant. For the Ant Version select the FlexUnit Intro Ant. Under Targets enter 'test'. This will cause Hudson to run the test target every time a build is run.

  18. Click Save.

  19. You should now be taken to the project homepage. Right now this project will do nothing. You need to specify a workspace. To do so, select Workspace from the main screen. This can also be accessed from the menu on the left.

    Workspace

    You will receive an error:

    Error

    This is completely expected. Hudson needs to execute a new build to configure the workspace.

  20. Select Run a build.

    You should see new Build History menu on the left, it will update with the new build. At completion, this build should fail (red dot indicates failure). The build will fail because the current target does not exist. However, the workspace setup is now complete.

  21. Click on the workspace link in the left-hand menu again.

    You should now see:

    NoFiles

    Hudson is expecting the base directory of your project which should also contain the build.xml. Unfortunately, since we are not using a SCM, this directory will not populate correctly. We need to add the files directly to the workspace.

  22. Windows: Navigate to: C:/Documents and Settings/<username>/.hudson/jobs/FlexUnitCI Intro/workspace

    Mac: Navigate to: Users/<username>/.hudson/jobs/FlexUnitCI Intro/workspace

  23. This is the default location of the test launch workspace. Copy all files contained within the FlexUnit4Training project directory into this directory.

  24. Refresh the Hudson page in your browser.

  25. Hudson should now show the current contents of your project.

    CurrentContents

    The project is now ready to be built. Since we are running this from the Test Launch and do not have a SCM specified, we will need to run manual builds.

  26. Select Build Now from the menu on the left.

    Build now will force an immediate build on any files currently in the workspace. This should return a success (blue dot on the left under build history.)

    BuildHistory

    During test execution, you will see the standalone player load with a blank screen. This is normal and will close automatically. Do not close this manually or your build may fail.

  27. Click on build #2.

    This is the details view of the build. Since we have verbose set to true in the build file, we can view the details of each run.

  28. Click on Console Output. Here will print detail of each test. Failures will also appear here.

    Since JUnit reports were created for the test run, you may also view these.

  29. Click on Back to Project in the left hand menu. Then click on Workspace. This will open the current workspace setup. Click on target then report then html. Select index.html to see the detailed JUnit report.

  30. Congratulations, you have now completed your first Hudson test run.

    Hudson will not auto build when new changes are made to the main project. Because we had to copy the build files manually any new changes will also need to be manually copied. This is not necessary in a server version where a SCM tool has been specified for the project. Hudson will pull any new changes and create a new build automatically.

Navigation