TS 67 CDevContinuousIntegration

Connected Developer "Cotinuous Integration" Test Specification

Author: Petr Dvorak
Version: 6.7
Last update: 2009/04/14
Introduction: This document is test specification for Continuous Integration.

To make Hudson running on your localhost, just download the hudson.war file and run java -jar hudson.war


Comments:So far this test specification is related to Hudson support in NetBeans IDE.

Contents


Test suite: New Hudson instance

Purpose: To create a new instance of the Hudson CI in IDE.
Setup: Start IDE with a fresh userdir.

  1. Create a new Hudson instance
    1. Open the Services view (Ctrl+5)
    2. Right-click the Hudson node and invoke Add Hudson Instance
    3. Close the dialog by clicking Cancel
    4. Right-click the Hudson node and invoke Add Hudson Instance again
    5. Fill in an arbitrary name, for example "My Hudson", in the Name field (this will be used as an alias for the instance)
    6. Specify the URL of a Hudson instance
    7. Click Finish
    8. Expand the "My Hudson" node and its subnodes
  2. Create another Hudson instance
    1. Open the Services view (Ctrl+5)
    2. Right-click the Hudson node and invoke Add Hudson Instance
    3. Fill in an arbitrary name, for example "My Hudson - other", in the Name field (this will be used as an alias for the instance)
    4. Specify the URL of some other Hudson instance than in the 1st case
    5. Click Finish
    6. Expand the "My Hudson - other" node and its subnodes
    • EXPECTED RESULT: A new Hudson instance is added in the services view (under the Hudson node). Jobs that are running on a given Hudson instance are visible in the view.


Test suite: Workflow - Hudson instance

Purpose: To test a workflow with the Hudson instance.
Setup: Start IDE with a fresh userdir, add a Hudson instance in the services view.

  1. Check the Hudson instance popup menu
    1. Right-click the Hudson instance (for example "My Hudson")
    • EXPECTED RESULT: Menu contains following five items: New Build, Synchronize, Persist, Open in Browser, Delete, Properties.
  2. Hudson instance - Synchronize
    1. Add some Hudson instance in the IDE, if you didn't do it yet
    2. Create a new job "TestJob" via the web interface on the Hudson instance (so that the Hudson instance contains a job that is not visible in IDE)
    3. Right-click the Hudson instance (for example "My Hudson")
    4. Invoke Synchronize from the context menu
    • EXPECTED RESULT: A new job called "TestJob" should show up in the tree with the other Hudson jobs.
  3. Hudson instance - Open in Browser
    1. Right-click the Hudson instance (for example "My Hudson")
    2. Invoke Open in Browser from the context menu
    • EXPECTED RESULT: A Hudson's dashboard opens in a web browser.
  4. Hudson instance - Delete
    1. Right-click the Hudson instance (for example "My Hudson")
    2. Invoke Delete from the context menu
    3. Add a new Hudson instance so that you can continue with other tests
    • EXPECTED RESULT: A Hudson instance is removed.
  5. Hudson instance - Properties
    1. Right-click the Hudson instance (for example "My Hudson")
    2. Invoke Properties from the context menu
    • EXPECTED RESULT: A dialog with Hudson instance properties shows up. Only the Autosynchronization time field is editable (Name and URL are read-only).


Test suite: Workflow - Hudson Jobs and Builds

Purpose: To test a workflow with jobs (starting, displaying results, ...)
Setup: Start IDE with a fresh userdir, add a Hudson instance in the services view.

  1. Displaying job details in web browser
    1. Right-click any job under Hudson instance node
    2. Invoke Open in Browser from the context menu
    • EXPECTED RESULT: A web browser is opened and it shows detail of the proper job.
  2. Displaying job properties
    1. Right-click any job under Hudson instance node
    2. Invoke Properties from the context menu
    • EXPECTED RESULT: Properties of a job are opened. Only "Watched" checkbox should be editable.
  3. Displaying config.xml
    1. Right-click any job under Hudson instance node
    2. Invoke View config.xml from the context menu
    • EXPECTED RESULT: Config.xml file is opened in editor.
  4. Displaying build details in web browser
    1. Right-click any build (a number, like "#123") of a job under Hudson instance node
    2. Invoke Open in Browser from the context menu
    • EXPECTED RESULT: A web browser is opened and it shows detail of the build.
  5. Showing build console
    1. Right-click any build of a job under Hudson instance node
    2. Invoke Show Console from the context menu
    • EXPECTED RESULT: A console output for last (potentially running) job of the given project is opened in the Output view.
  6. Show changes
    1. Right-click any build of a job under Hudson instance node
    2. Invoke Show Changes from the context menu
    • EXPECTED RESULT: Changes from the previous build (commits to files) are shown in the Output window.


Test suite: Creating Hudson jobs

Purpose: To verify it is possible to create a job on Hudson
Setup: Start IDE with a fresh userdir, add a hudson instance in the services view.

  1. Create a job - simple
    1. Create a new "Hello World" Java Application, and make it versioned in SVN
    2. Right-click the Hudson instance and choose New Build...
    3. Select the project versioned in subversion from the combobox
    4. Change the default name to "My Job - 1"
    5. Leave only the first checkbox ("Build JAR") checked
    6. Confirm the dialog
    • EXPECTED RESULT: A new Hudson job is created, it must be visible in the services view.
  2. Create a job - complete
    1. Create a new "Hello World" Java Application, and make it versioned in SVN
    2. Create some JUnit tests (right-click the main class, invoke Tools>Create JUnit Tests, ...)
    3. Right-click the Hudson instance and choose New Build...
    4. Make the libraries sharable (JUnit needs to be included) - click the Copy Libraries button and go through the wizard (click Next, click Finish) - this will close the New Build wizard
    5. Commit all the changes that were made to sources
    6. Right-click the Hudson instance and choose New Build...
    7. Select the project versioned in subversion from the combobox
    8. Change the default name to "My Job - 2"
    9. Leave all checkboxes checked
    10. Confirm the dialog
    • EXPECTED RESULT: A new Hudson job is created, it must be visible in the services view. The job compiles only if you made the libraries sharable, otherwise the build fails.
  3. Create a job - no SCM
    1. Create a new "Hello World" Java Application, and DO NOT make it versioned in SVN
    2. Right-click the Hudson instance and choose Create Job...
    3. Select some project that is not versioned from the combobox
    • EXPECTED RESULT: An error should be displayed saying that the project uses no source code management.



Test suite: Running Hudson job

Purpose: To verify it is possible to run a Hudson job.
Setup: You will need a Hudson instance with some job that can be finished in a time around 10 seconds, for example a job for the "Hello World" Java Application - follow previous test cases if you don't have this setup.

  1. Run Hudson Job - success
    1. Make sure the application can be built (commit the changes, if any)
    2. Right-click the Hudson job node under Hudson instance node in the Services view
    3. Choose Start Job from the popup menu
    4. Right-click the Hudson instance and choose Synchronize from the popup menu to see that the job is being built (you will see a little badge by the node)
    5. Wait for a while (15seconds)
    6. Right-click the Hudson instance and choose Synchronize from the popup menu
    • EXPECTED RESULT: Successful build is indicated by the blue balloon. If you committed some changes to the source code before the build, they are visible after you invoke "Show Changes" from the build popup menu. The build console can be displayed by "Show Console" command from the build popup menu.
  2. Run Hudson Job - failure
    1. Introduce some error in the source code (and commit the changes)
    2. Right-click the Hudson job node under Hudson instance node in the Services view
    3. Choose Start Job from the popup menu
    4. Right-click the Hudson instance and choose Synchronize from the popup menu to see that the job is being built
    5. Wait for a while (15seconds)
    6. Right-click the Hudson instance and choose Synchronize from the popup menu
    • EXPECTED RESULT: A bubble should notify you that the build has failed. The job failure is indicated by the red balloon. IIf you committed some changes to the source code before the build, they are visible after you invoke "Show Changes" from the build popup menu. The build console can be displayed by "Show Console" command from the build popup menu.


Test suite: Misc tests

  1. Test results in IDE
    1. Have a project with test (passing, some are failing)
    2. Have registered local Hudson instance
    3. Add new build job from this project, run it
    4. You get a notification that test in this job fails
    • EXPECTED RESULT: You can view failures right in IDE in Test results window.


  1. Login
    1. Add new Hudson instance, that requires login to view jobs (or setup such locally)
    2. Invoke popup on this instance, choose "Login"
    • EXPECTED RESULT: Unless you log in properly, you are not able to see nodes


  1. Git support
    1. Create or checkout project that is versioned in Git
    2. Team|Create build job from this project on local Hudson
    • EXPECTED RESULT: Build job can be created, Git is recognized as VCS
  1. Quick search for job
    1. Have registered some Hudson instance that has some job
    2. Type job name into IDE quicksearch
    • EXPECTED RESULT: Job is found by quicksearch and can be jumped to
Not logged in. Log in, Register

By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo