Front Page

User FAQ
Developer FAQ
Community Docs
Development
Quality
Dream Team
Popular Projects
Bug DashBoard
Hudson
Sources
Nightly
Wiki Etiquette
Formatting Rules
Terms Of Use
Unlinked Pages
Undefined Pages
Page Index
Recent Changes
Find Page

[RSS]

Table of Contents

Unit Tests

There are multiple reasons why unit tests simplify the developers life. Especially some of them ;
  • speed up the development process (fast way to test new functionality)
  • give the developer confidence that code is correct
  • prevent from regressions
Unit tests are especially important for any kind of API we develop : either Internal or Friend or Stable.

I've put together some useful links/guides how to write unit tests: - describes how to write unit tests for multiple scenarios (file objects, data objects)
- how to set up test dependencies

As the unit tests are concentrated primarily on individual classes and methods it's mostly the responsibility of individual developers to write unit tests !!!

List of Existing Unit Tests for WS Functionality

  • websvc.axis2 /* Milan */
  • websvc.clientapi /* Lukas */
  • websvc.customization /* Rico */
  • websvc.design /* Milan */
  • websvc.editor.hints
  • websvc.jaxwsapi /* Lukas */
  • websvc.jaxwsmodel /* Milan */
  • websvc.manager
  • websvc.projectapi /* Rico */
  • websvc.rest
  • websvc.saas.api
  • websvc.saas.codegen
  • websvc.saas.services.delicious /* Lukas */
  • websvc.saas.services.facebook /* Lukas */
  • websvc.saas.services.flickr /* Lukas */
  • websvc.saas.services.google /* Lukas */
  • websvc.saas.services.twitter /* Lukas */
  • websvc.saas.services.weatherbug /* Lukas */
  • websvc.saas.services.yahoo /* Lukas */
  • websvc.saas.services.zillow /* Lukas */
  • websvc.saas.services.zvents /* Lukas */
  • websvc.utilities
  • websvc.websvcapi /* Lukas */
  • websvc.wsitconf /* Martin Grebac */
  • websvc.wsitmodelext /* Martin Grebac */
  • websvc.wsstackapi /* Milan */

Note : most of the WS Functionality is covered very poorely (with unit tests).

Functional Tests

  • Test Automation
  • Test Guidelines
  • properties can be set on the command line, in nbbuild/user.build.properties or in somekit/nbproject/private/private.properties

Overall summary

Kit nameTest countTest duration (min)Class CoverageMethod Coverage
websvc.kit65 27 75%59%
websvc.restkit 37 18 90%73%

Note: Tests were run on Toshiba TecraM9 - Intel Pentium M 2GHz, 3GB RAM, Ubuntu 8.10 against NetBeans 6.5 RC2

Running Tests

  • Prerequisites:
    • have a local installation of some supported server (GlassFish, Apache Tomcat, JBoss)
    • add test-qa-functional-sys-prop.glassfish.home=/space/to/glassfish-v2ur2 to nbbuild/user.build.properties
  • Running tests from the IDE:
    • open a test in the IDE
    • Run File
  • Running tests from command line: 
cd $NB_ALL/websvc.kit
//will run "stable" tests (by default against GlassFish) in websvc.kit module
//will use default timeout (= 10mins) - can by overriden by -Dtest.timeout
//will overwrite test-qa-functional-sys-prop.glassfish.home from nbbuild/user.build.properties
ant test-qa-functional -Dtest.config=stable -Dtest-qa-functional-sys-prop.glassfish.home=/space/to/glassfish-v2ur2

Web Services core (JAX-WS Support)

Test sources are in websvc.kit module

  • Available test configurations:
Test ConfigurationTest Suite(s)Duration (mins)
stableHintsTests, WebServiceDesignerTest 5
validationHintsTests, WebServiceDesignerTest, WsValidation 13
allFullWsValidation(*), FromWSDLTSuite, HintsTests, WebServiceDesignerTest 27
Note: (*) FullWsValidation is a merge of WsValidation, EjbWsValidation, AppClientWsValidation and JavaSEValidation

Covered scenarios

  • WsValidation
    • Create new web service from Java in Web project
    • Add new web service operation using editor's context menu and action on the web service node
    • Set SOAP 1.2 on the web service
    • Set SOAP 1.1 on the web service
    • Create message handlers
    • Configure message handlers on created service (add, move, remove)
    • Deploy project
    • Call Test Web Service action on the web service
    • Call Generate SOAP-over-HTTP Wrapper action on the web service
    • Call Generate and Copy WSDL action on the web service
    • Redeploy project
    • Create new web service client in Web project
    • Call web service operation in servlet
    • Call web service operation in JSP
    • Call web service operation in regular Java class
    • Create message handlers
    • Configure message handlers on created client (add, move, remove)
    • Refresh Client
    • Deploy project
  • EjbWsValidation
    • Create new web service from Java in EJB module project
    • Add new web service operation using editor's context menu and action on the web service node
    • Set SOAP 1.2 on the web service
    • Set SOAP 1.1 on the web service
    • Call Generate and Copy WSDL action on the web service
    • Create message handlers
    • Configure message handlers on created service (add, move, remove)
    • Deploy project
    • Call Test Web Service action on the web service
    • Create new web service client in EJB module project
    • Call web service operation in session EJB
    • Call web service operation in regular Java class
    • Create new web service from session EJB
    • Create message handlers
    • Configure message handlers on created client (add, move, remove)
    • Refresh Client
    • Deploy project
  • AppClientWsValidation
    • Create new web service client in AppClient module project
    • Call web service operation in Java Main Class
    • Call web service operation in regular Java class
    • Create message handlers
    • Configure message handlers on created client (add, move, remove)
    • Refresh Client
    • Run project
  • JavaSEWsValidation
    • Create new web service client in JavaSE module project
    • Call web service operation in Java Class
    • Create message handlers
    • Configure message handlers on created client (add, move, remove)
    • Refresh Client
    • Run project
  • FromWSDLTSuite
    • Create new web service from WSDL in Web project
    • Refresh service from WSDL
  • WebServiceDesignerTest
    • Add web service operation to ws from Java
    • Add web service operation to ws from Java which uses @WebService.endpointInterface
    • Call Show Sample Input action
    • Call Show Sample Output action
    • Call Switch to list view action
    • Call expand/collapse buttons
    • Call Go To Source action
    • Remove web service operation to ws from Java
    • Remove web service operation to ws from Java which uses @WebService.endpointInterface
  • HintsTests
    • Remove endpointInterface attribute
    • Remove Oneway annotation
    • Remove HandlerChain annotation
    • Remove SOAPMessageHandlers annotation
    • Remove mode attribute
    • Remove serviceName attribute


Identity Support

Test sources are in identity.kit module

  • Prerequisites:
    • have a local installation of JavaEE 5 SDK Update 4 (App Platform variant) with Access Manager 7.1 Patch1 add-on at /space/Sun/SDK or e:\space\SDK (can be overridden in build-qa-functional.xml)
    • after you have installed SDK Glassfish, you should start it once to let it to automatically configure Access Manager
  • Available test configurations:
Test ConfigurationTest Suite(s)
stable <empty>
validationIdentityValidation,IdentityValidationInEJB

Covered Scenarios

  • IdentityValidation
    • Add JavaEE 5 SDK Glassfish into IDE, start it and check Access Manager installed correctly
    • Edit Access Manager Profiles
    • Set AM Security in Web Service Provider in Web Application
    • Set AM Security in Web Service Client in Web Application
    • Deploy AM secured Web Applications and authenticate client to provider
  • IdentityValidationInEJB
    • Set AM Security in Web Service Provider in EJB Module
    • Set AM Security in Web Service Client in EJB Module
    • Deploy AM secured EJB Modules and authenticate client to provider
  • IdentitySamplesTest
    • testStockQuoteService
    • testStockQuoteClient

RESTful Web Services support

Test sources are in websvc.restkit module

  • Available Properties:
Property NameDefault settingProperty Description
plugins.jmaki.skip true skip jMaki related test
plugins.jmaki.nbm ${java.io.tmpdir}/org-netbeans-modules-sun-jmaki.nbm taken into account if plugins.jmaki.skip=false, allows one to skip downloading of nbm from http://ajax.dev.java.net
Note: these properties have to be prefixed by test-qa-functional-sys-prop.


  • Available test configurations:
Test ConfigurationTest Suite(s)Duration (mins)
stableRestSamplesTest, StableTests(*) 18
validationRestSamplesTest, StableTests(*) 18
Note: (*) StableTests is a merge of FromDBTest, CRUDTest, PatternsTest, CStubsTest, RestNodeTest

Scenarios covered by test suites

  • FromDBTest
    • Create new RESTful web services from Database
    • Deploy created application
  • CRUDTest
    • Create new entity classes from sample DB
    • Create new RESTful web services from entity classes
    • Create new entity class with property based access
    • Create new RESTful web service from property based entity class (also tests the functionality of the wizard itself)
    • Call Test RESTful Web Services action
    • Deploy created application
  • PatternsTest
    • Create new RESTful web service from patterns using Singleton pattern
    • Create new RESTful web service from patterns using Container-Item pattern
    • Create new RESTful web service from patterns using Client-Controlled Container-Item pattern
  • RestSamplesTest
    • Create and deploy HelloWorld sample
    • Create and deploy Customer Database sample
    • Create and deploy Customer Database on Spring Sample
  • CStubsTest
    • Create new RESTful web services client stub for existing project (created by CRUDTest)
    • Create new RESTful web services client stub from local WADL file
  • RestNodeTest
    • Check RESTful web services node visibility after opening a project
    • Check "Open" action on the resource node and its subnodes
    • Check dynamic adding/removing RESTful web service operation node to/from project logical view
  • JMakiTest
    • same as CStubsTSuite but with use jMaki option in the wizard turned on

To Be Done

Following table shows features for which we want to write functional tests.
Note: tests will be implemented in no particular order

TaskOwnerNotes
WS Designer, Add operation - from WSDL casesLukas
WS CustomizationLukasfor now covered by TS_60_CustomizationEditor
WS Client CustomizationJardafor now covered by TS_60_CustomizationEditor, Planned for 2008
WSIT stuffTBD
Use custom keystore for IdentityJarda Planned for 2008
Check AM Security vs WSIT behaviourJarda Planned for 2008
Axis2 support
New ws client - dispatch
New ws from wsdl - provider
REST Spring support

Features not planned to be covered by functional tests

Following features will not be covered by automated qa-functional tests anytime soon:

  • Test web service action on web service node
  • JAX-RPC support

Features to be evaluated

  • REST test client - httpunit cannot be used for this because its support for JavaScript is quite limited (it even cannot do simple GET request on the test client correctly), some other possibilities/tools to evaluate - use Selenium, Groovy