Remote and parallel use cases

Building multi-platform products

product1 comprises of component1 and component2. component1 should be built on Windows, while component2 should be built on Linux. After these components have been built, they should be collected together on Windows platform and packaged into product1.

[Warning]Warning

In order to make remote build work, you should keep time of different build machines in sync (Normally a slight difference within 5 seconds is tolerable).

  1. Install QuickBuild on Windows and Linux. Create configuration root.product1 in QuickBuild system running on Windows (refer as root.product1@Windows later), and create configuration root.component2 in QuickBuild system running on Linux (refer as root.component2@Linux later). root.product1@Windows will be used to build component1 and package component1 and component2 into product1, while root.component2@Linux will be used to build component2.

  2. In root.component2@Linux, create appropriate repository, builder and steps so that it can check out source code of component2 from CVS, build component2, and create label on component2 in CVS if build is successful. Artifacts of component2 will be published into some directory of generated build. Do a test on this configuration to make sure it works.

    [Note]Note

    root.component2@Linux will maintain separate build versions from root.product1@Windows. This is reasonable, cause it's the common case that a product comprises of multiple components of different and independent version numbers.

  3. In configuration root.product1@Windows:

    • Create below repositories:

      repository1

      This is a CVS repository which is used to check out source codes of component1 from CVS.

      repository2

      This is a QuickBuild repository which is used to check out build artifacts of component2 from latest build of configuration root.component2@Linux.

    • Create two builders: builder1 and builder2. builder1 is used to build component1, and builder2 is used to package artifacts of component1 and component2 into product1.

    • Create below steps:

      check out component1 from CVS

      This step uses repository1 to check out code of component1 from CVS

      build component1

      This step uses builder1 to build component1

      retrieve component2 artifacts

      This step uses repository2 to check out artifacts from latest build of component2 at Linux box.

      [Note]Note

      By defining this step, root.product1@Windows is considered to be dependent on root.component2@Linux, and root.component2@Linux will be triggered automatically during the dependency resolving phase, which is happened before retrieving component2 artifacts.

      package artifacts of component1 and component2

      This step uses builder2 to package artifacts of component1 and component2 into product1

      create label

      This step does the following things:

      • Creates a label on source code of component1 in CVS.

      • Creates a label on root.component2@Linux to mark the build number of component2 whose artifacts are packaged into this version of product1.

      default

      This step is a serial composition step that runs the above five steps serially.

  4. In this way, product1 can be built easily. Further more, If product1 contains component3 which should be built on Solaris, you can install another QuickBuild system on a Solaris machine, configure it to be able to build component3, and add corresponding QuickBuild repository to check out artifacts from component3 at Windows side.

  5. A live demo is available through QuickBuild's demo site http://livedemo.pmease.com:8081/. Within this live demo:

    • root.remote-builds.product1 stands for root.product1@Windows we talked about.

    • root.remote-builds.LinuxBox.component2 simulates the configuration root.component2@Linux.

Working with parallel builds

My product contains multiple components, which can be built independently. Is there any way to build them simultaneously on multiple machines to speed up the build process?

  1. In order to demonstrate this ability, we will take the example cited in use case Building multi-platform products, but try to build component1 and component2 simultaneously. Nothing needs to be changed at root.component2@Linux side. At root.product1@Windows side, make the following changes:

    • Add a repository say repository3. This repository is defined so that it contains source path of both component1 and component2.

    • Change build necessary condition to be repository["repository3"].modified at basic settings tab of current configuration. The reason for not using effectingRepositoriesModified is that: By using effectingRepositoriesModified, when decides whether or not a new build is necessary, QuickBuild will examine contents of every repository referenced in step definitions to see if they have been modified. During this phase, root.component2@Linux specified in repository2 may be triggered and built (and contents of repository2 are considered to be modified if latest build of component2 happens after latest build of product1), which is not we want. By using repository["repository3"].modified instead, we ignores repository2 during build necessary condition evaluation phase of product1, but still be able to detect changes in both component1 and component2.

    • Create the following steps:

      Check out component1 from CVS

      This step uses repository1 to check out code of component1 from CVS.

      Build component1

      This step uses builder1 to build component1.

      Check out and build component1

      This step is created to execute the above two steps serially.

      Retrieve component2 artifacts

      This step uses repository2 to check out artifacts from latest build of component2 at Linux box. This step will cause component2 been built if necessary (depends on build necessary condition of root.component2@Linux).

      Prepare artifacts of component1 and component2

      This step is a parallel composition of step "checkout and build component1" and "retrieve component2 artifacts". By using this step, component1 and component2 will be built simultaneously.

      Package component1 and component2 into product1

      This step packages prepared artifacts of component1 and component2 into product1.

      Create label

      This step does the following things:

      • Creates a label on source code of component1 in CVS.

      • Creates a label on root.component2@Linux to mark the build number of component2 whose artifacts are packaged into this version of product1.

      default

      This step is a serial composition of step "prepare artifacts of component1 and component2", "Package component1 and component2 into product1", and "create label".

  2. A live demo is available through QuickBuild's demo site http://livedemo.pmease.com:8081/. Within this live demo:

    • root.remote-builds.product1-parallel stands for root.product1@Windows we talked about in this use case.

    • root.remote-builds.LinuxBox.component2 simulates the configuration root.component2@Linux.

Performing automation/smoking tests on a machine other than build machine

After my product has been built, it should be sent to another machine to run smoking/automation tests. If tests pass, mark current build as successful; otherwise, mark it as failed.

  1. Take "product2" as example, we create a configuration root.product2 at machine1 to build this product. In this configuration, set up the following things:

    • Create two repositories:

      repository1

      This repository is created to check out source code of product2 from CVS.

      repository2

      This repository is created to retrieve test log from latest build of configuration root.test-product2@machine2. In this repository, we define a module with source path be "..", and file name pattern be "build_log.txt". In this way, test log will be retrieved (Note that source path is relative to artifacts directory, so we use ".." to change to parent directory of artifacts, which includes the build log file).

    • Create two builders:

      builder1

      This builder is used to build product2.

      builder2

      This builder is used to publish test log into some sub directory of artifacts directory, so that it can be accessed from web interface.

    • Create five steps:

      check out from CVS

      This step uses repository1 to check out source code of product2.

      build with Ant

      This step uses builder1 to build product2.

      retrieve test results

      This step uses repository2 to check out test log of product2. At this point, you may curious about how build result of product2 is sent to machine2. The trick is at root.test-product2@machine2 side. In that configuration, we will define steps to pull latest build result of product2.

      publish test results

      This step uses builder2 to publish test log. Step necessary condition property of this step should be set to true in order to publish the test results for review even the step "retrieve test results" fails.

      default

      This step executes the above four steps one by one.

  2. Another configuration root.test-product2 needs to be created at machine2 to run test against product2. In this configuration, set up the following:

    • Set value of property build necessary condition as true, which means new build will always be generated upon triggering. This is needed because this configuration will be dependently triggered by root.product2@machine1, and for each dependent triggering, we want new build of root.test-product2 been generated so that tests can be run.

    • Create two repositories:

      repository1

      This repository is created to check out build result of product2 from latest build of configuration root.product2@machine1.

      repository2

      This repository is created to retrieve test script from CVS.

    • Create a builder builder1, which will be used to run test script.

    • Create four steps as below:

      check out test scripts

      This step uses repository2 to check out test scripts from CVS.

      check out build results of product2

      This step uses repository1 to check out build results of product2.

      run test against product2

      This step uses builder1 to run test.

      default

      This step executes the above steps one by one.

  3. In this way, build result of product1 will be tested on machine2, and test results will be collected back to machine1. If test fails (that is, build of root.test-product2@machine2 fails), step "retrieve test results" at root.product2@machine1 side will also fail, which causes build of product2 failed.

    [Note]Note

    As you may noticed, root.product2@machine1 depends on root.test-product2@machine2 to retrieve test results, and root.test-product2@machine2 depends on root.product2@machine1 to retrieve product2 build results. Obviously there is a dependency loop here. QuickBuild is clever enough to handle this correctly:

    • When a configuration is triggered actively (that is, triggered by user or schedule, instead of dependency resolver), other configurations detected in a dependency loop is considered as subordinate configurations which will not be taken into account when evaluating build necessary condition of the active configuration. In our case, root.product2@machine1 is triggered actively, and root.test-product2@machine2 plays the role as subordinate configuration.

    • At root.product2@machine1 side, when step "retrieve test results" executes, it will wait until the newly generated build at root.test-product2@machine2 side completes (either successful or failed), and then downloads test results from that build.

    • At root.test-product2@machine2 side, when step "check out build results of product2" executes, it knows that it is a subordinate configuration, and will download build results of product2 right away from root.product2@machine1 without waiting for build of root.product2@machine1 been finished. Otherwise, deadlock will occur. Consequently, you should make sure build results of product2 are available for retrieving before step "retrieve test results" gets running in configuration root.product2@machine1.

    [Warning]Warning

    You should never actively trigger a subordinate configuration. Otherwise, deadlock may occur.

  4. A live demo is available through QuickBuild's demo site http://livedemo.pmease.com:8081/. Within this live demo:

    • root.remote-builds.product2 stands for configuration root.product2@machine1 we talked about.

    • root.remote-builds.LinuxBox.test-product2 simulates configuration root.test-product2@machine2.