TestNG

(Difference between revisions)

Revision as of 02:40, 8 February 2012

TestNG is a new generation testing framework developed due to stagnation of JUnit at version 3.8.x.

TestNG support

Adds support for creating, running, debugging and displaying results for TestNG tests in Ant and Maven based projects in NetBeans 6.5 and/or later except of NetBeans module projects.

IMPORTANT: TestNG binary bundled and used by this plugin differs from the official distribution from testng.org. Read more

This is work in progress so don't hesistate to file any issues to Bugzilla (contrib/testng)

Instructions for the Maven part of the plugin are yet TBD

Author: Lukas Jungmann
Last Update: February 8, 2012

1 TestNG support
2 Getting Started
3 Related stuff

Issues

Plugin Sources

Generic support (defines TestNG library): http://hg.netbeans.org/main/contrib/file/tip/testng/
Ant support: http://hg.netbeans.org/main/contrib/file/tip/testng.ant/
Maven support: http://hg.netbeans.org/main/contrib/file/tip/testng.maven/

Plugin Binaries

Development version:

Development update centre: http://deadlock.netbeans.org/hudson/job/nbms-and-javadoc/lastSuccessfulBuild/artifact/nbbuild/nbms/updates.xml.gz
Downloadable development binaries:
- Generic support: o.n.m.testng.nbm
- Ant support: o.n.m.testng-ant.nbm
- Maven support: o.n.m.testng-maven.nbm

Stable version:

available on the Plugin portal, for NetBeans 7.x

Getting Started

Installation

Open Tools -> Plugins
Now depending on your preference and current IDE:
- IDE - Development version:
  - switch to Available Plugins tab
  - search for testng
  - select module(s) you want to install
  - hit Install button and follow the steps
- IDE - Released version:
  - either set up Development Update Center in Settings tab if you want to and follow steps in previous section Note: this may entice you to update IDE or some of its internals to the latest, greatest version - you've been warned :-)
  - or switch to Downloaded tab
  - use Add Plugins... button to add nbm files you downloaded
  - hit Install button and follow the steps

Figure 1: Installation from development update centre

Project setup

IMPORTANT: Compile on Save feature has not yet been implemented for TestNG tests, so as the very first step after creating a project, disable this feature in Project Properties -> Compiling -> Compile on Save to avoid unexpected behaviour, please. This is known issue.

For already existing projects with some version of TestNG on its classpath no setup is needed. In this case the plugin just assumes that project has either already been setup by this plugin or you have already some targets running TestNG tests implemented in the Ant's build script or Maven's pom.

New projects and existing projects without any TestNG library on the classpath will be setup by this plugin as soon as you create at least one test or test suite using one of provided wizards. This will add bundled TestNG library to the project and, in Ant based projects, creates new build script, nbproject/testng-impl.xml, which is used for running and debugging tests. If your project messes up by some accident, you can remove the library, delete the created build script and go through the wizard to get correct setup supported by this plugin.

Creating Tests

Plugin comes with two wizards - one to help you to create a simple test and the second to create xml suite

Figure 2: New File wizard with TestNG file templates

New TestNG Test Case wizard will help you to create regular valid TestNG java test file with sample content, while New TestNG Test Suite wizard will help you to create TestNG suite descriptor file in XML format, again with some sample content. You can customize both templates through Tools -> Templates.

Running Tests

Test project (ALT+F6), Run File (Shift+F6) and Test File (CTRL+F6) actions defined in the IDE do work for TestNG tests in the same way as they do for JUnit tests.

The only difference here is Run Focused Test action. This action has its equivalent in TestNG -> Run Test Method action.

Debugging Tests

To debug your TestNG tests, you can't currently use IDE wide actions you're used to from JUnit support by default. These actions are implemented as TestNG -> Debug Test Class and TestNG -> Debug actions which work in the same way as Debug Test File and Debug Focused Test do.

We're working hard on merging TestNG -> Debug Test Class and Debug Test File (CTRL+Shift+F6) actions together and until this is resolved, we provided workaround for this. All you need to do in order to get the shortcut working is to change the build.xml file of your project to start with:

<project name="JavaApplication1" default="default" basedir=".">
   <description>Builds, tests, and runs the project JavaApplication1.</description>
   <import file="nbproject/testng-impl.xml"/>
   <import file="nbproject/build-impl.xml"/>
   ...

Running/Debugging Tests in NetBeans 6.5

Following table gives an overview about supported ways to run/debug tests:

Action	Ant projects - disabled compile on save	Ant projects - enabled compile on save	Maven projects
Run all tests in a project	Test (Alt+F6)	Test (Alt+F6)	Test (Alt+F6)
Run test	Run File (Shift+F6)	TestNG -> Run Test Class	Run File (Shift+F6)
Run test method	TestNG -> Run Test Method	TestNG -> Run Test Method	TestNG -> Run Test Method
Re-run failed test(s)	TestNG -> Re-run Failed Tests	TestNG -> Re-run Failed Tests	TestNG -> Re-run Failed Tests
Debug test	TestNG -> Debug Test Class	TestNG -> Debug Test Class	TestNG -> Debug Test Class
Debug test method	TestNG -> Debug Test Method	TestNG -> Debug Test Method	TestNG -> Debug Test Method

Note: TestNG here refers to TestNG action group in editor's context menu

testng.xml Support

Navigation from testng.xml to particular test, test method, listener and other Java files as well as to other suite files imported from given file is supported through hyperlinks (CTRL+click)

Running and debugging of testng.xml files is supported from file's context menu

Code completion is supported based on testng.dtd

TestNG Results Window

Results window provides the same set of features which are you used to from JUnit's test results window - you can Go To Source, View Difference, Run one particular test method again or immediatelly Debug it. You can also filter results according to its state and Jump from one failure to the next or previous one.

If your project uses TestNG v. 6.3.1 or newer you will see test result to be reported incrementally (we submitted patch to TestNG allowing us to implement this and it has been accepted in v. 6.3.1). For older versions of TestNG, test results remain displayed based on the testng-results.xml file which is produced by TestNG after all tests are run.

Related stuff

There are plans to improve the TestNG support in year 2012
Mixed Mode for TestNG
UnitTests
http://testng.org/

Retrieved from "http://wiki.netbeans.org/TestNG"