CND69UnitTests

Contents

CND 69 Unit Tests

CppUnit: A C++ unit testing framework that started as a port of JUnit to C++.

Team

Overview

Supporting C/C++ unit testing is a feature requested by many C/C++ CND developers. Target user for this feature is a C/C++ developer using the IDE for code development (Leon & Steven) but may also be somewhat useful for sustaining tasks (Antony). This feature is aiming to make the developer more productive by providing seamless unit testing in the IDE. Overview:

  1. Support unit testing at the same or similar level as JUnit is supported by Java projects.
  2. Support both managed (defining) and unmanaged (target) projects.
  3. Support a plugable test harness infrastructure.
  4. Implement support for CppUnit and simple C and C++ tests.
  5. User builds and installs CppUnit libraries and header files seperately. CppUnit is not distributed with NetBeans/CND.
  6. Test and verify with CppUnit 1.12.1 available at Sourceforge http://sourceforge.net/projects/cppunit/files/cppunit/1.12.1/cppunit-1.12.1.tar.gz
  7. Implement plugable test infrastructure in projects module and support for different test types as separate modules.

Prototype

A very rough prototype has been committed to cnd-main. To run it, please build from recent sources and follow the directions below.

Managed Projects

Managed projects come with a top folder called Test Files. You want to manage unit tests under this top folder. You can create (named) tests directly under this top folder or organize them in logical sub folders. A test is just a special folder with special actions. When a test is built, it will exclude the main file from the project and otherwise use all other project settings. A test is empty when created and you will need to add files to the test.Use the standard New->XXX to add a file. Here are templates for CppUnit tests, CUnit tests, and Simple tests. Perhaps we could change New Test... to a wizard that also adds test files?

Step-by-step example of how to create and run a unit test.

  1. Create new C++ Application project
  2. Right-click Test Files folder and select New Test... Give the test a name and click OK.
  3. Right-click the newly created (empty) test and select New->C++->CppUnit Test. Give the class a name and click OK.
  4. Right-click the newly created test and select New->C++->CppUnit Runner, and click OK.
  5. Right-click the test and select Test. The test will now be compiled and executed. Output will be parsed and failures perlinked to test (THIS PART DOESN'T YET WORK). Perhaps we will have a GUI also similar to JUnit tests to show progess and test results?
  6. You can also right-click the top test folder Test Files and select Test action. This will run all tests.
  7. Other actions should be fairly self explanatory and should work.
  8. All tests are integrated into the makefiles and can be built and executed using a new make target: make test.

Tests can be maintained using usual project actions. You may need to configure the test (include files) in it's properties for it to compile and for the code model to work correctly (THIS PART DOESN'T YET WORK). If you use CppUnit tests, you will need to configure tests only one place, that is in the folder containing the tests. Perhaps if we implement a wizard to add a test, it could also configure the test automatically? Any kind of test is supported as long as it has a main function but hyperlinks and GUI will only work if they follow certain yet to be determined specifications.


Unmanaged Projects

Extended unmanaged projects with the notion of Test roots. A Test root is a folder that contains tests and is visible in the project tree but marked specially and showing binaries (they are usually not visible). A test is usually a binary (already being built by the makefile) or a script. Both will be visible in the project tree under test folders and either can be runned (kind of work today) and debugged (doesn't work today) directly. Tests can be managed using standard project actions

Step-by-step example of using tests in a unmanaged project:

  1. Create a project from existing sources. Use Custom mode. In the panel that asks for source roots, there is also a table where you can add test roots. Add a folder as test folder. It will look for standard test folders (test, tests, ...) an automatically add them. This is doesn't yet work. It only look for a folder called tests in top directory. Running in auto mode should add test folder automatically but not implemented yet. Pleas euse Custom mode.
  2. Once the project is created, navigate to the test folder you added. It should have been marked as a test folder and now show all files including binaries. Binaries can be executed or debugged directly from their context menus (debugging doesn't yet work).
  3. Open project properties. Test roots are manageable in the same panel as source roots.

That's it for unmanaged projects. Basically the only support is a special view of test folders and possibility of executing and debugging tests directly from the project view.

User View

Project Logical View, actions and properties

Managed Projects

Managed projects will have actions to add a test harness and also run the tests:

 Project ---------------------> Test
   Header Files
   Resource Files
   Source Files
   Tests ---------------------> Add Tests (CppUnit Tests, CUnit Tests, C++ Tests, C Tests, ...)
   Important Files

Available test harnesses depends on actual implementation. Only one test harness can be added.

After adding a test harness, actions to add, run, and configure tests is a added. It is up the the implementation of the specific harness to implement the actual actions, view and properties. Example of a CppUnit harness:

 Project
   Header Files
   Resource Files 
   Source Files                 New CppUnit Test
   Tests - CppUnit -----------> Test
     Runner                     Properties ------> CppUnit install info, additional compiler/linker options
       Main.cpp               
     A -----------------------> Run
       A.cpp                    Debug
       A.h                      Build
     B                          Delete
       B.cpp                    Properties ------> Additional compiler/linker options
       B.h                    

CppUnit tests are using a common runner (under Runner node) and each test can be individually runned, built, and debugged using actions from the test's context menu. Additional compiler/linker options can be specified in properties. The test will otherwise use the properties from the project.


Or a simple C harness:

 Project
   Header Files
   Resource Files
   Source Files                 New C Test
   Tests - C -----------------> Test
     A                          Properties ------> Additional compiler/linker options
       A.c
       A.h                      Run
     B -----------------------> Debug
       B.c                      Build
       B.h                      Delete
                                Properties ------> Additional compiler/linker options   

Test action in the project context menu will run all the tests in the harness. Individual tests can be runned or debugged from their own context menus. Additional compiler/linker options can be specified in properties. The test will otherwise use the properties from the project.

Unmanaged Projects

Unmanaged projects support only simple tests. A simple test is just a references to a binary plus some information about how detect Success or Failure when runned. The test may or may not be buildable. Example:

 Project ---------------------> Test
   src                          New Test
   Tests ---------------------> Test
     D1                         Properties ------------> Success or Failure, ???
     D2
       D3                       Run
         T1 ------------------> Debug
         T2                     Delete              Executable Path
       D4                       Properties -------> Build command
         T3                                         Success/Failure
    Important Files

The C/C++ Project from Existing Sources wizard will have a new section called Test File Folders where the user can specify folders that contain tests (and not project sources). The wizard will scan specified test folders for binaries and create simple test based on these binaries.

Project Physical View

Physical view will show all the unit test files.

CppUnit Test Wizard

A simple file wizard that creates working CppUnit stubs. It creates a C++ header file and a C++ source file similar to the ones below and place them in the test logical folder.

Source file:

#include "NewClass.h"

CPPUNIT_TEST_SUITE_REGISTRATION(NewClass);

NewClass::NewClass() {
}

NewClass::~NewClass() {
}

void NewClass::setUp() {
   this->example = new int(1);
}

void NewClass::tearDown() {
   delete this->example;
}

void NewClass::testMethod() {
   CPPUNIT_ASSERT(*example == 1);
}

void NewClass::testFailedMethod() {
   CPPUNIT_ASSERT(++*example == 1);
}

Header file:

#ifndef _NEWCLASS_H
#define	_NEWCLASS_H

#include <cppunit/extensions/HelperMacros.h>

class NewClass : public CPPUNIT_NS::TestFixture {
   CPPUNIT_TEST_SUITE(NewClass);
   CPPUNIT_TEST(testMethod);
   CPPUNIT_TEST(testFailedMethod);
   CPPUNIT_TEST_SUITE_END();
 
public:
   NewClass();
   virtual ~NewClass();
   void setUp();
   void tearDown();

private:
   int *example;
   void testMethod();
   void testFailedMethod();
};


Questions:

  1. Should we have advanced choices as Create setUp() and Create tearDown() in wizard? Subclass? Other choices?
  2. Files should not be added to the normal project build but they should be added to code model? Will this raise any problems?

Test Progress and Result Visualizer

Something very similar to what NetBeans has for JUnit: File:unittestprogress.jpg

Command Line

Building and running the tests from the command is supported with the make test command. It will build and run all tests. Running individual tests from the command line is not supported.

Use Cases

Add Tests to Existing (managed) Project

User is asked to add a (CppUnit) unit test to an existing managed project.

  1. He downloads, builds, and install CppUnit test infrastructure on his system
  2. He opens the existing project in the IDE
  3. He right-clicks the Tests node in the project and chooses Add Tests and selects CppUnit Tests in the wizard.
  4. He right-clicks the CppUnit Tests node in the project view and selects New CppUnit Test action and type in the name (A) of the test in the wizard.
  5. He opens A.cpp file in the editor and 'fills in the blanks'.
  6. He right-clicks the project and selects Test. It builds and runs the newly added test.
Running Existing Tests in Managed Project

User is asked to make a code change to an existing project and verify nothing is broken by running existing unit tests.

  1. He makes the code changes
  2. He right-click the project and select Test action or type make test from the command line.


Adding existing test when creating a unmanaged project

User wants to import existing project (ACE) and add existing tests to the project.

  1. He builds the project (ACE) from command line including all tests
  2. He goes through the C/C++ Project with Existing Sources wizard and add the test (test for Ace) folders to list of test folders. Wizard will scan all the test folders for binaries and add them to Test folder as simple tests. Tests may need to be manually configured. For instance the regular Success or Failure pattern may need to be adjusted.

Schedule

Milestone Date Content
1 1/11/10 User View Ready
2 2/22/10 Feature useful
3 NB 6.9 M1 4/07/10
4 NB 6.9 Beta 4/15/10 Feature complete
5 NB 6.9 Code Freeze 5/10/10 Done, all P1/2 bugs fixed.

Risks and Dependencies

Resources

BZ 112571 support for CPPUnit or CUnit https://netbeans.org/bugzilla/show_bug.cgi?id=112571

Some resources of CppUni:

What is CppUnit: http://sourceforge.net/apps/mediawiki/cppunit/index.php?title=Main_Page

CppUnit Sources http://sourceforge.net/projects/cppunit CppUnit Sources

CppUnit Wiki http://de.wikipedia.org/wiki/CppUnit

Existing runners:

http://www.softwaredevelopment.ca/wxtestrunner.shtml

http://sourceforge.net/apps/mediawiki/cppunit/index.php?title=QtTestRunner

http://sourceforge.net/apps/mediawiki/cppunit/index.php?title=MfcTestRunner

http://sourceforge.net/apps/mediawiki/cppunit/index.php?title=CursesTestRunner

Competitor:

http://beans.seartipy.com/category/easyeclipse/

Not logged in. Log in, Register

By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo