TS 71 Profiler

Profiler Test Specification

Author: Emanuel Hucka, Alexander Kouznetsov, Matus Dekanek, Petr Cyhelsky
Version: 1.9 Last update: 2011/08/22

Introduction: This document describes test specification of NetBeans Profiler on one platform. The most used platforms by users and for testing is Windows XP 32-bit and Ubuntu Linux 32-bit with JDK 1.6.26 or jdk 1.7.0. JDK 1.5.0_04 or higher can be used for all test cases if there is not any specified jdk version. Comment: This test specification does not describe validation of measured results.

Test Specification

To Add

  • Support for Maven project

Sanity Test Specification

Installation

# Test Case Name
1.3 Calibration


J2SE Project and Unit Tests

# Test Case Name
2.2 Monitoring
2.3 Analyze Performance
2.6 Analyze Performance - Snapshots
2.7 Analyze Performance - Root Methods
2.10 Analyze Memory Usage
2.12 Analyze Memory Usage - Snapshots
2.15 Analyze Memory Usage - Class History


Test suite 5: NB Module

# Test Case Name
5.1 Integration
5.2 Analyze Performance
5.3 Analyze Memory Usage


Test suite 6: Web Project

# Test Case Name
6.1 Monitoring


Test suite 7: EJB - Glassfish

# Test Case Name
7.1 Integration
7.2 Integration Server
7.3 Profile Mode
7.7 Glassfish v3


Test suite 10: Attach Profiler

# Test Case Name
10.2 Local Direct Application


Test suite 12: HeapWalker

# Test Case Name
12.2 Clases
12.3 Instances
12.5 OQL console


Test suite 13: Profiling Points

# Test Case Name
13.1 Stopwatch and Reset Results

Test suite 1: Installation

Purpose: install and prepare profiler and test data.

# Test Case Description Expected Results
1.1 Install NetBeans 1. Install current version of NetBeans IDE with Java IDE, GlassFish and Tomcat options enabled. Use a folder containing spaces to verify fixes of SPACE_IN_PATH issues.
2. Start NetBeans IDE and check if the profiler is installed.
NetBeans ide should contain Profile menu, Tools -> Options dialog should contain Profiler tab in Miscellaneous section. Profiler version is available in IDE messages.log file in a form of: org.netbeans.modules.profiler/1
1.2 Test Data and Software 1. Download and extract projects: applet, freeform, web, jboss test and order system.
2. Use latest builds of JBoss j2ee server.
3. Configure NetBeans to start with a JDK version. Use e.g. JDK 1.6.0.
NetBeans Profiler is ready to be tested.
1.3 Calibration
[[[sanity | [sanity]]
SETUP: Stop all background processes which will not be running during future profiling.
1. Invoke Profiler -> Advanced Commands -> Run Profiler Calibration.
2. Select main Java platform.
Calibration will pass without any errors and exceptions. There should be opened Progress dialog with a message calibration is running during calibration. Calibration is finished by Information dialog about measured times. The values are usually about 1-100 ms (depends on test machine speed).


Test suite 2: J2SE Project and Unit Tests

Setup: Create and open a sample J2SE project AnagramGame - File -> New Project -> Samples -> Java -> Anagram Game. The project should be set as main project.

# Test Case Description Expected Results
2.1 Integration 1. Select opened project and invoke popup menu (in Projects view).
2. Popup menu should contain enabled Profile Project menu item.
3. Set the project as main project (if it is not) and check if menu action Profile -> Profile Main Project is enabled.
4. Profile Main Project action in Profile toolbar should be enabled too.
5. Press Profile Main Project action in Profile toolbar and confirm opened Question dialog (This is the first time this project ...). Press Cancel button in opened Profile AnagramGame dialog.
Project's build script should be updated - there should be added line: <import file="nbproject/profiler-build-impl.xml"/> Origin build script should be backed up under name build-before-profiler.xml. Project's nbproject folder should contain profiler-build-impl.xml file.
2.2 Monitoring
[[[sanity | [sanity]]
1. Press Profile Main Project... in Profile toolbar. Confirm build script modification dialog.
2. Select Monitor Application in Profile WebApplication dialog, check Enable Threads Monitoring option and press Run button.
3. Check if Threads and VM Telemetry Overview windows are opened. Check if both windows are scrolling automatically their contents.
4. There should not be any thread in threads window with 'Profiler' or 'JFluid' in its name.
5. Select a thread in Threads view and invoke Thread details action in its popup menu.
6. Check if Threads (Details) tab in Threads window is selected and if it contains thread's name, 'pie chart' with statistics and line graph with colored thread's states.
7. Select one thread's state section in the line graph. Check if Details sub tab with textual states information is opened and appropriate state is selected.
8. Invoke VM Telemetry button in Profiler Control Panel.
9. Press Zoom Out button (-) until whole Memory (Heap) graph panel is visible. Its horizontal scrollbar should be disabled.
10. Check if the zoomed out graph is alive.
11. Switch to Memory (GC) graph panel and press Scale to Fit button. Whole graph should be scaled into graphs' boundaries. Wait a moment - graph should be alive and it should be shrinked to the panel boundaries - horizontal scrollbar should be disabled. The Scale to Fit button should be replaced by Fixed Scale button.
12. Switch to Threads (Statistics) graph panel, select a part of the graph - press left mouse button somewhere on left side of the part and drag it to the right side of the part. The selected part should be fit into the graph's boundaries.
13. Open Threads window. Go To Profiled Application and close it.
14. Go back to profiler; there should be shown Information dialog 'The profiled application is finished ...' Wait a few seconds - Threads window on backgroud should continue in showing new states of threads.
15. Confirm the dialog and check the Threads window. There should not be any thread with 'Profiler' or 'JFluid' in its name or any new thread created during finishing of application.
16. Profiled application should be stopped. All graphs should be stopped too.
17. Invoke Save Current View button in any graph panel. There should be opened a Select File or Directory with file system structure, preview of the view and PNG in Files of Type. Write a filename and confirm the dialog. Snapshot of selected view should be stored on disk as png file.
18. Export view - Export current view to all possible formats using the export button - check tha the created files are in correct format.
19. Close Profiling Control Panel. Invoke Window -> Profiling -> Profiling Control Panel. Profiling Control Panel should be opened again.
Monitoring is verified.
2.3 Analyze Performance
[[[sanity | [sanity]]
1. Invoke Profile -> Profile Main Project... action.
2. Select CPU, Analyze Performance task, set Entire application and Filter: Profile Only Project Classes. Press Run button.
3. Wait until the application is started. Open Live Results window.
4. There should be a table with measured methods in the Live Results window.
5. Click DrillDown button to open DrillDown window. It should be bound to the right side of live results panel.
6. Invoke Profile -> Advanced Commands -> View Command-line Arguments. JVM and Main class Command-line Arguments window should be opened and contain corresponding information. Close Command-line Arguments window.
7. Invoke Stop (Terminate) the Profilled Application action.
The Live Results window should stay opened with its latest content.
2.4 Analyze Performance - Live Results 1. Invoke Profile -> Profile Main Project... action. Select Analyze Performance task, set Entire application and Filter: Profile All Classes. Press Run button. Wait until the application is started. Open Live Results window. (Check: Then close and invoke Window -> Profiling -> Live Results)
2. Go to Method Name Filter_ (bottom of live results panel) and select Starts with filter type. Write com.toy and press Enter. Live results table should contain only com.toy.* methods.
3. Uncheck Update Results Automatically toggle button in Live Results tool bar and make some actions in the application (write a word). The results table should not change. Let it several seconds to ensure that table doesn't change.
4. Press Update Results Now button - results table should change than. Check Update Results Automatically toggle button to 'enabled' state.
5. Go to Method Name Filter and press Clear current filter button in the right - table should contain all methods again. Try to select e.g. Regular expression filter type - write .*Map.* to filter textfield and press Apply new filter button - there should be shown mostly methods from java.util.*.
6. Press Reset Collected Results Buffer button in Live Results toolbar - results table should be empty, there should be gray message - No profiling results available yet. Make some action in the profiled application - results table should be fillfulled again.
7. Invoke Save Current View button in Live Results toolbar. There should be opened a Select File or Directory with file system structure, preview of the view and PNG in Files of Type. Write a filename and confirm the dialog. Snapshot of selected view should be stored on disk as png file.
8. Invoke Take Snapshot in Live Results toolbar - the button should be enabled. After invoking of the action a new CPU snapshot should be opened. Switch back to Live Results window.
9. Check sorting of results table by each column - press the column title bar.
10. Sort table by Self Time. Press Show or hide results columns button in the right-top corner, uncheck Self Time, Invocations, Filter items. There should remain only two columns in Live Results table sorted by Self time %. Press the button again and enable all items in the menu. All columns should be shown as before.
11. Select a method in results table and invoke its popup menu. Press Go To Source menuitem. There should be opened Java editor with source of the class or shown Error dialog: No source found for class ...
12. Choose a method with a long name (not fully displayed in results table). Move mouse cursor over the method's name. A tooltip should be shown in place of the method's name containing full name of the method.
13. Export the current view.
2.5 Analyze Performance - DrillDown 1. Invoke Profile -> Rerun Profiling Session. Wait until the application is started. Open Live Results window. Make some actions in started application.
2. DrillDown window should be opened and bound to the right side of Live Results panel. DrillDown toggle button should be placed to the right side of live results toolbar. The DrillDown window should contain Scope part with pie chart and its legend where legend and pie chart items are selectable. After selection of an item Live Results should be filtered to focus only on the selected category of methods.
3. Select Generic UI category in the pie chart, then AWT & Swing Painters. Select any method. Method category distribution part of DrillDown window should contain AWT & Swing Painters bar.
4. Select Project category in Scope part of the DrillDown window. Select any method in the Live Results table. Method category distribution part of DrillDown window should contain at least two categories - IO and UI.
5. Try to resize drilldown window and check if it is repainted correctly. Open a form e.g. Anagrams.java, switch between Profiling Results and editor and check if DrillDown window is shown only in Results window. Try to switch off and on the Drilldown window by DrillDown button Results toolbar.
2.6 Analyze Performance - Snapshots
[[[sanity | [sanity]]
1. Invoke Profiler Control Panel -> Rerun Last Profiling Session.
2. Check Take Snapshot button - it should be enabled during starting of the application. Press the button immediatelly after it is enabled.
3. Wait until the application is fully started and invoke its Close Window action.
4. There should appear Application Finished dialog asking if you want to take a snapshot of the finished application. Confirm it by pushing the Yes button.
5. Check if both taken snapshots contain measured data. The second snapshot should contain more measured methods than the first.
6. Go to the first snapshot and invoke Close Window action on it. It should ask if you want to save the snapshot. Press No button.
7. In the remaining snapshot window invoke Save Snapshot to Project, close it and open it again from Saved Snapshots list in Profiler Control Panel.
8. Invoke Find action from IDE's toolbar, write e.g. "show" into opened dialog's textfield and confirm the Find in Results dialog by Find button. Try to jump through all occurences of the word by Find previous occurance and Find next occurance in the snapshot.
9. Try to change View to Classes and Packages - the results tree should be changed by appropriate way. Return back to Methods view.
10. Check sorting of the results tables by each column - press the column title bar.
11. Invoke Save Current View button in the snapshot's toolbar. There should be opened a Select File or Directory with file system structure, preview of the view and 'PNG' in Files of Type combo. Write a filename and confirm the dialog. Snapshot of selected view should be stored on disk as png file.
12. Go to e.g. java.awt.Component.show(boolean) method, invoke popup menu on it and press Show Back Traces action. It should switch to Back Traces for: show tab and show back traced methods of the selected method.
13. Select Combined tab of the snapshot window, try to select methods in both parts of splited panel and check if there is always selected equivalent method in the other panel.
14. Use Method Name Filter in the bottom panel to find com.toy.anagrams.ui.Anagrams.main (e.g. set Starts With filter type, write 'com.toy' to filter line and confirm) and double click on the method. It should jump into Java editor with Anagrams.java to the selected method's code.
15. Browse Info tab panel of the snapshot. It should contain information about profiling settings, e.g. Profiling Type: CPU Profiling (Entire Application) etc.
16. Select the snapshot in Saved Snapshots list of Profiler Control Panel, press Save As... button, choose a path and file name and save the snapshot there.
17. Delete the snapshot by Delete button. Invoke Load... action in Saved Snapshots, select the previous file and confirm file dialog.
The snapshot should be loaded.
2.7 Analyze Performance - Root Methods
[[[sanity | [sanity]]
1. Invoke Profile -> Profile Main Project... action. Select Analyze Performance task, set Entire application and Filter: Profile All Classes. Press Run button. Wait until the application is started. Open Live Results window.
2. Invoke Guess action in the application. Go to live results table, find the com.toy.anagrams.ui.Anagrams.guessedWordActionPerformed(java.awt.event.ActionEvent) method, invoke its popup menu and select Add to Instrumentation Roots.... In the opened Select Settings Configuration dialog select Analyze Application item and confirm the dialog by OK button.
3. Invoke Profile -> Stop action and wait until the profiler is not finished. Invoke Profile -> Profile Main Project... action.
4. In Select Profiling Task dialog choose Part of Application radio button. "edit..." link next the radio button should be enabled and there should be text '1 root method' next to Part of Application radio button.
5. Click "edit..." link. Check if there is com.toy.anagrams.ui.Anagrams.guessedWordActionPerformed(java.awt.event.ActionEvent) method in Selected Root Methods list in the Specify Root Methods dialog.
6. Invoke Add From Project button. In the opened Select Root Methods dialog open nodes of the Projects tree to AnagramGame -> Sources -> com -> toy -> anagrams -> ui -> Anagrams -> Methods. All these nodes should have shadowed checked checkboxes in front of their names. Check checkbox for nextTrialActionPerformed(java.awt.event.ActionEvent) method and confirm the dialog by OK button. Selected Root Methods list should contain two methods now.
7. Confirm Specify Root Methods dialog by OK button and Select Profiling Task dialog by Run button. Wait until the application is not started and open Live Results window.
8. Browse Live Results window and Basic Telemetry part of Profiler Control Panel (there is Instrumented: label with number of instrumented methods). Press Guess button in the profiled application and watch the Live Results and Basic Telemetry. The number of instrumented methods should increase to about 700, the same for Live Results window. Press New Word button in the application - another methods should be measured/instrumented.
9. Invoke Reset Collected Results action from Profiler Control Panel. Live Results Window should be emptied. Switch to the profiled application window - no methods should be shown in Live Results window.
10. Invoke File -> About action in the profiled application - live results should not be changed.
11.Invoke Profile -> Advanced Commands -> Display Internal Statistics. Internal Statistics of Instrumentation and Hotswapping Operation window should be opened and contain corresponding information. Push Close button.
12. Press Guess button in the profiled application. And finish profiling of the application.
13. Open Java file : Invoke context menu: Profiling -> Add As Profiling Root Method.
14. Open Java file : Invoke context menu: Select Profiling Root Method
There should be 'many' measured methods in Live Results window.
2.8 Analyze Performance - Instrumentation Filters 1. Invoke Profile -> Profile Main Project... action. Select Analyze Performance task, set Entire application and Filter: Profile Only Project Classes.
2. "Show filter value" link should get enabled. Click it - there should be opened Profile Only Project Classes Dialog with list of project's packages - there should be two packages for AnagramGame project. Press To Quick Filter button in the dialog.
3. Set Quick Filter dialog should be opened with preset Inclusive and both packages in the Filter Value text field. Confirm it by OK button. Filter combobox should be changed to 'Quick Filter...' in the Select Profiling Task dialog. There should be 'Edit filter value' link below the combobox to open Set Quick Filter dialog.
4. Select Profile Only Project Classes filter and press Run button. Wait until the application is started. Open Live Results window.
5. Invoke some action in the profiled application. Live results should contain methods from the project's packages only.
6. Stop Profiling and invoke Profile Main Project action. Choose Part of application and click edit link next to it. in the "Specify Root Methods" dialog click on "Add From Project" button. In the "Select Root Methods" dialog choose the whole com.toy.anagrams.lib package and from the com.toy.anagrams.ui package choose onlyAnagrams<init> method (under Anagrams->Constructors). Choose Quick Filter.... Remove com.toy.anagrams.ui. from Filter Value in Set Quick Filter dialog and confirm it by OK button. Start profiling by Run button. Wait until the profiling is started and open Live Results window.
7. Invoke New Word action a few times in the profiled application and check live results. There should be no other method from com.toy.anagrams.ui then com.toy.anagrams.ui.Anagrams.<init>() .
8. Stop Profiling and invoke Profile Main Project action again. Click Edit filter sets link. Press Add button in the Customize Filter Set dialog, write 'Not UI' into the Filter Set Name text field and leave Filter Set Type as Exclusive.
9. Press Edit Global Filters, press New button, write 'NonGUI' into the Name column and 'java.awt., javax.swing.' into the Value column. confirm the Edit Global Filters dialog by OK button.
10. In Customize Filter Sets dialog select NonGUI filter in Global filters list (check its checkbox). Confirm the dialog by OK button. The 'Not UI' filter should be selected in Filter combobox. Start profiling by Run button. And wait for results of profiling in Live Results.
There should not be any methods from java.awt or javax.swing package.
2.9 Window options 1. Window -> Profiling -> VM Telemetry.
2. Window -> Profiling -> Threads.
3. Window -> Profiling -> Telemetry Overview.
2.10 Analyze Memory Usage
[[[sanity | [sanity]]
1. Invoke Profile -> Modify Profiling Session. Select Memory button.
2. Leave default settings - Record object creation only, Track every 10 object allocations, disabled Record Stack Trace for Allocations. Confirm dialog by Run button.
3. Press Live Results button and wait until the appliacation is started. Live Results should be full of measured objects.
4. Press Reset Collected Results button - Live Results should be empty. Switch to the profiled application window and make some action. Stop profiling.
Another measured objects should be in Live Results window.
2.11 Analyze Memory Usage - Live Results 1. Invoke Profile -> Profile Main Project... action. Select Analyze Memory Usage task. Leave default settings - Record object creation only, Track every 10 object allocations, disabled Record Stack Trace for Allocations. Confirm dialog by Run button. Wait until the application is started. Open Live Results window.
2. Uncheck Update Results Automatically toggle button in Live Results tool bar and make some actions in the application (write a word). The results table should not change.
3. Press Update Results Now button - results table should change than. Check Update Results Automatically toggle button to 'enabled' state.
4. Press Reset Collected Results Buffer button in Live Results toolbar - results table should be empty, there should be gray message - No profiling results available yet. Make some action in profiled application - results table should be fillfulled again.
5. Invoke Save Current View button in Live Results toolbar. There should be opened a Select File or Directory with file system structure, preview of the view and PNG in Files of Type. Write a filename and confirm the dialog. Snapshot of selected view should be stored on disk as png file.
6. Invoke Take Snapshot in Live Results toolbar - the button should be enabled. After invoking of the action a new Memory snapshot should be opened. Switch back to Live Results.
7. Check sorting of results table by each column - press the column title bar.
8. Press Show or hide results column button in the right-top corner, uncheck Bytes Allocated, Objects Allocated, Filter items. There should remain only two columns in Live Results table. Press the button again and enable all items in the menu. All columns should be shown as before.
9. Go to Class Name Filter (bottom of live results panel) and select Ends with filter type. Write Buffer and press Enter. Live results table should contain only 'Buffers'. Press Clear current filter - button with red cross on the filter line. Table should contain all classes again. Try to select e.g. Regular expression filter type - write .*Map to filter textfield and press Apply new filter button - there should be shown all 'Maps'. Press Clear current filter button.
10. Select some class from com.toy.anagrams.* subpackages in results table and invoke its popup menu. Try to play with application if there are no classes from specified subpackages. Press Go To Source menuitem. There should be opened Java editor with source of the class. Go to live results and try to invoke Go to Source action on 'char[[ | ]]' class. There should be shown an Warning message: Cannot show source for primitive array.
11. Choose a class with a long name (not fully displayed in results table) or resize Class name column to the left. Move mouse cursor over the class names. A tooltip should be shown in place of the class name containing full name of the class.
2.12 Analyze Memory Usage - Snapshots
[[[sanity | [sanity]]
1. Invoke Profile -> Profile Main Project... action. Select Analyze Memory Usage task. Leave default settings - Record object creation only, Track every 10 object allocations, enabled Record Stack Trace for Allocations. Confirm dialog by Run button.
2. Check Take Snapshot - button it should be enabled during starting of the application. Press the button immediatelly after it is enabled. Memory snapshot should be opened. Switch back to Live Results window.
3. Choose e.g. java.util.HashMap in the Live Results panel and invoke Take Snapshot and Show Allocation Stack Traces action from its popup menu. A new snapshot should be opened with selected Allocation Stack Traces tab panel.
4. Wait until the application is fully started and invoke Close Window action of the profiled application.
5. There should appear Application Finished dialog asking if you wan to take snapshot of finished application. Confirm it by Yes button.
6. Check if all taken snapshots contain measured data.
7. Go to the first and second snapshot and invoke Close Window action on them. It should ask if you want to save the snapshot. Press No button.
8. In the remaining snapshot window invoke Save Snapshot to Project, close it and open it again from Saved Snapshots list in Profiler Control Panel by Open button or double click into the snapshots list.
9. Press Ctrl+F, write e.g. "Long" into opened dialog's textfield and confirm the Find in Results dialog by Find button. Try to jump through all occurences of the word by Find previous occurance and Find next occurance in the snapshot.
10. Try to switch to Allocation Stack Traces tab of the snapshot window. It should be empty now. Switch back to Memory Results panel, select e.g. java.lang.String class, invoke its popup menu and choose Show Allocation Stack Traces action. It should be switched to Allocation Stack Traces tab with all stacktraces of the class.
11. Press Show or hide results column button in the right-top corner of Memory Results panel, uncheck Bytes Allocated, Objects Allocated, Filter items. There should remain only two columns in Live Results table. Press the button again and enable all items in the menu. All columns should be shown as before.
12. Invoke Save Current View button in the snapshot's toolbar. There should be opened a Select File or Directory with file system structure, preview of the view and 'PNG' in Files of Type combo. Write a filename and confirm the dialog. Snapshot of selected view should be stored on disk as png file.
13. Browse Info tab panel of the snapshot. It should contain information about profiling settings, e.g. Profiling Type: Memory (Allocations Only) etc.
14. Select the snapshot in Saved Snapshots list of Profiler Control Panel, press Save As... button, choose a path and file name and save the snapshot there.
15. Delete the snapshot by Delete button. Invoke Load... action in Saved Snapshots, select the previous file and confirm file dialog.
16. Invoke Profile -> Load Snapshot.
17. Profile -> Compare Memory Snapshots
The snapshot should be loaded.
2.13 Analyze Memory Usage - Allocations Count, Log Class History, Compare Memory Snapshots 1. Invoke Profile -> Profile Main Project... action. Select Analyze Memory Usage task. Leave current settings - Record object creation only, Track every 10 object allocations, enabled Record Stack Trace for Allocations. Confirm dialog by Run button.
2. Open Live Results window. Select char[[ | ]] object, right click it and select Log Class History.
3. Take a snapshot.
4. Play with application, try to invoke Profile -> Run GC action several times. All values (allocated bytes and objects) should be unchanging or increasing.
5. Open "History of char[[ | ]]" tab in Live Results window. All values should be unchanging or increasing.
6. Take a snapshot. Compare it with previously created snapshot.
All differences should be zero or negative showing that all values were increasing between these two snapshots.
2.14 Analyze Memory Usage - Object Liveness 1. Invoke Profile -> Profile Main Project... action. Select Analyze Memory Usage task. Set Record both object creation and garbage collection, Track every 10 object allocations, enabled Record Stack Trace for Allocations. Confirm dialog by Run button.
2. Wait until the application is started and make some action in it.
3. Open Live Results window, select a class and invoke Take Snapshot and Show Allocation Stack Traces action from its popup menu. Save opened snapshot and close it.
4. Go to Live Results window panel, sort results by Class Name ascending. Scroll to the first class from package 'sun.awt' invoke its popup menu and press Stop Profiling Classes bellow this line action. Invoke its popup menu again and invoke Stop Profiling this Class action.
5. Make some actions in the profiled application.
There should not be any class from sun.awt package in live results.
2.15 Analyze Memory Usage - Class History
[[[sanity | [sanity]]
1. Invoke Profile -> Profile Main Project... action. Select Analyze Memory Usage task. Leave current settings - Record both object creation and garbage collection, Track every 10 object allocations, enabled Record Stack Trace for Allocations. Confirm dialog by Run button.
2. Open Live Results window, select class 'char[[ | ]]' and invoke Log Class History action from its popup menu.
3. There should be enabled History of char[[ | ]] tab panel in live results window. Go to the tab and press Scale to Fit button.
4. Invoke some actions in profiled application. Both history graphs should be increased. Try to invoke Run GC action from Profiler Control Panel several times. The Live Objects graph should be decreased.
5. Invoke Save Current View button in Live Results toolbar. There should be opened a Select File or Directory with file system structure, preview of the view and PNG in Files of Type. Write a filename and confirm the dialog. Snapshot of selected view should be stored on disk as png file.
6. Go back to Live Results tab, select 'int[[ | ]]' class and invoke Log Class History on it. The History tab name should be changed to History of int[[ | ]] , values in the graph should be changed too.
2.16 Unit Tests 1. Go to Projects view and open AnagramGame -> Test Packages -> com.toy.anagrams.lib -> WordLibraryTest.java file.
2. Be sure the ui focus is in the opened editor and call menu action Profile -> Profile Other -> Profile "WordLibraryTest.java".
3. In Select Profiling Task go to Analyze Performance and select Entire Application, Profile All Classes. Confirm the dialog by Run button.
4. Wait until the profiling is finished and confirm Application Finished dialog by Yes button. A non empty cpu snapshot should be taken.
5. Go to Projects view and open AnagramGame -> Source Packages -> com.toy.anagrams.lib -> WordLibrary.java file.
6. Call menu action Profile -> Profile Other -> Profile Test for "WordLibraryTest.java", in Select Profiling Task go to Analyze Memory Usage and select Record both object creation and garbage collection, enable Record Stack Trace for Allocations. Confirm the dialog by Run button.
7. Wait until the profiling is finished and confirm Application Finished dialog by Yes button. A non empty memory snapshot should be taken.

Test suite 3: Applet

Setup: Unpack and open AppletTest project from applet_ProfilerTestSpecification.zip archive.

# Test Case Description Expected Results
3.1 Profile Applet - AppletViewer SETUP: Open applet/TestApplet.java source file in editor.
1. Change ui focus to an editor with opened TestApplet.java file. Invoke Profile -> Profile Other -> Profile "TestApplet.java" from main menu. Confirm dialog about first time profiling. Select Analyze Performance, Entire Application, Profile All Classes and press Run button in the Select Profiling Task dialog.
2. Wait until the AppletViewer is opened with running application. Press Increase button in the applet a few times. Invoke Live Results button in Profiler Control Panel. There should be applet.TestApplet.make() at the first position of the live results panel. Self Time of the method should be about 500ms * number of invocations.
3. Select Generic UI category in DrillDown window. Find applet.TestApplet.addBActionPerformed(java.awt.event.ActionEvent) method, invoke its popup menu and select Add to Instrumentation Roots action. In the opened dialog choose Analyze Performance and confirm the dialog.
4. Stop Profiling, go to TestApplet.java file and invoke Profile -> Profile Other -> Profile TestApplet.java ... again. Choose Part of Application and confirm the dialog.
5. Wait until the AppletViewer is opened with running application, browse live results and try to invoke Decrease button in the applet. Live results should not be changed. But the Increase action should add many methods to live results table.
6. Invoke Modify Profiling action in Profiler Control Panel. Select Analyze Memory Usage in Modify Profiling Task dialog, select Record both object creation and garbage collection, enable Record Stack Trace for Allocations. Confirm the dialog by OK button.
7. Live Results window should be emptied, Take snapshot button disabled. After an ui action (repaint) in Applet Browser there should appear new memory results in Live Results window.
8. Invoke Reset Collected Results action and make some action in AppletViewer again. Live Results should be empty and after the actions in viewer fillfulled again.
9. Close the AppletViewer window and confirm Application Finished dialog.
A memory snapshot should be taken.
3.2 Profile Applet - Browser SETUP: Prepare a web browser allowing to start Java applets. Go to Tools -> Options -> General and set the browser by Web Browser combo. Go to <$JAVA_HOME>/jre/bin/ControlPanel (javacpl.exe in windows) and in Java -> Java Applet Runtime Settings by View... check if there is set JDK 1.6. On windows it should be modified by editing registers: *HKEY_LOCAL_MACHINE\SOFTWARE\JavaSoft\Java Runtime Environment] "CurrentVersion"="1.6"*
1. Open Files view and go to AppletTest -> build -> AppletTest.html - if the file is not there invoke Run action on TestApplet.java file and close invoked AppletViewer. Invoke popup menu and View action on the html file. A browser should be opened with TestApplet started. Check the java version in the footer of running applet.
2. Invoke Attach Profiler Button from ide toolbar. Press Attach wizard button. In Attach Wizard select Applet in Target Type combo, Local Attach method and Dynamic (JDK 1.6) Attach invocation. Confirm the wizard panel by Next button. Review settings and invoke Next button. Finish the wizard by Finish button.
3. In Select Profiling Task choose Analyze Performance, Part of Application - there should be set 1 Method from previous case. Set Filter to Profile All Classes and confirm the dialog by Attach button.
4. In Select Process select PID of running Java in browser - usually there is not a main class shown. Confirm the dialog by OK button.
5. Wait until Live results window is filled by profiling results. Invoke Take Snapshot and check taken CPU snapshot.
6. Press Threads button and enable threads monitoring. Check monitored threads. Press VM Telemetry button and check Memory (Heap) graph, invoke Run GC button several times and check behaviour of memory graph. Press Profile -> Detach action and confirm Question dialog by No button.
Applet should stay running in the browser. Profiler should be finished.


Test suite 4: Free Form

Setup: Unpack and open test project from freeform_ProfilerTestSpecification.zip archive.

# Test Case Description Expected Results
4.1 Integration 1. Open build.xml of the project.
2. According to Profiler Help - Additional Notes -> Profiling Freeform Projects insert:
<target name="profile" depends="dist" description="Profile Project">
<fail unless="netbeans.home">This target can only run inside the NetBeans IDE.</fail>
<nbprofiledirect>
<classpath> <pathelement location="${dist}/test.jar"/> </classpath>
</nbprofiledirect>
<java classname="${mainclass}" fork="true" dir=".">
<jvmarg value="${profiler.info.jvmargs.agent}"/>
<classpath> <pathelement location="${dist}/test.jar"/> </classpath>
</java>
</target>
to the project tag in build.xml.
3. Go to the Projects view and invoke popup menu of the project. Perform Profile Project action.
4. In Select Profiling Task dialog select Monitor Application and press Run button.
5. In Select Task for Profiling dialog select task 'profile' and press OK.
6. Monitoring of the application should start. Check it in Threads window. Stop Profiling.
7. Insert target:
<target name="profile-file" depends="dist" description="Profile File">
<fail unless="netbeans.home">This target can only run inside the NetBeans IDE.</fail>
<nbprofiledirect>
<classpath> <pathelement location="${dist}/test.jar"/> </classpath>
</nbprofiledirect>
<java classname="${profile.class}" fork="true" dir=".">
<jvmarg value="${profiler.info.jvmargs.agent}"/>
<classpath> <pathelement location="${dist}/test.jar"/> </classpath>
</java>
</target>
to the project build.xml file.
8. Open com/toy/anagrams/Anagrams.java file in editor. Invoke Profile -> Profile Other -> Profile "Anagrams.java" menu action.
9. In Select Profiling Task dialog select Monitor Application and press Run button.
10. In Select Task for Profiling dialog select task 'profile-file' and press OK.
Monitoring of the application should start. Check it in Threads window. Stop Profiling.
4.2 Profiling 1. Invoke Profile Main Project action. Select Analyze Performance, Entire Application, Profile Only Project Classes. Press Run button.
2. Open Live Results window. Make some actions in the application. Return back to ide an Take Snapshot. Check that all results are non-empty.
3. Invoke Modify Profiling. Select Analyze Memory Usage, Record both object creation and garbage collection, select Record Stack Trace for Allocations. Press OK button.
4. Go to the application and perform some actions. Return to ide, open Live Results window and Take Snapshot.
TEARDOWN: Stop profiling.
All results should be non-empty.


Test suite 5: NB Module

Setup: Create and open a sample NB project Paint Application - File -> New Project -> Samples -> NetBeans Plug-in Modules -> Paint Application. The project should be set as main project. Try to start project by Run action and check if all works.

# Test Case Description Expected Results
5.1 Integration
[[[sanity | [sanity]]
1. Select opened project and invoke popup menu (in Projects view).
2. Popup menu should contain enabled Profile Project menu item.
3. Set the project as main project (if it is not) and check if menu action Profile -> Profile Main Project is enabled.
4. Profile Main Project action in Profile toolbar should be enabled too.
5. Press Profile Main Project action in Profile toolbar. Press Cancel button in opened Select Profiling Task dialog.
5.2 Analyze Performance
[[[sanity | [sanity]]
SETUP: Close all documents.
1. Invoke Profile -> Profile Main Project... action. Select CPU and click "Create Custom..." link.
2. In New Custom Configuration (CPU) dialog change its name to 'testconfig' and click OK.
3. Click Advanced Settings link.
4. Select Exact Call Tree, Sampled Timing.
5. Execute profiling by Run button and watch Take Snapshot button. Immediatelly after the button is enabled, press it. It should take non empty CPU snapshot - close it without saving. Wait until whole application is started.
6. In the started application open New Canvas and paint something there. Close the application and confirm Application finished dialog in profiler by Yes button.
7. In the taken snapshot go to Hot Spots, find method org.netbeans.paint.PaintCanvas.mouseMoved, invoke its popup menu and select Add to Instrumentation Roots. In Select Settings Configuration dialog select 'testconfig' configuration and confirm it by OK button.
8. Invoke Profile -> Profile Main Project... action. Leave 'testconfig' custom profiling selected, choose Part of Application. Click "Edit" link next to "1 root method" label.
9. In Specify Root Methods select the method (mouseMoved) and press Remove button. Press Add From Project button. In Select Root Methods dialog open nodes Paint Application -> Sources -> org -> netbeans -> paint -> PaintCanvas -> Methods and select mouseDragged(java.awt.event.MouseEvent) method. Confirm dialogs Select Root Methods and Specify Root Methods by OK button - Specify Root Methods should contain one selected method in Root Methods list.
10. Confirm Select Profiling Task by Run button. Wait until the application is started.
11. Open Live Results window - it should be empty. Go to the profiled application, create new canvas and try to draw something. The first movements of stroke could be slow - instrumentation is running on background. There should be measured methods in Live Results window.
12. Select method org.netbeans.paint.PaintCanvas.mouseDragged(java.awt.event.MouseEvent), invoke its popup menu and press Go To Source action. There should be opened editor with the selected class and its cursor should be moved to the selected method.
13. Invoke Take Snapshot action.
14. Invoke Profile -> Stop Profiling Session.
Profiled application should be finished.
5.3 Analyze Memory Usage
[[[sanity | [sanity]]
1. Invoke Profile -> Profile Main Project... action. Select Monitor Application task, enable Threads Monitoring and press Run button.
2. Wait until the application is started and open New Canvas there.
3. Check Threads window if it is working. Invoke Modify Profiling action, select Analyze Memory Usage, Record object creation only and enable Record Stack Trace for Allocations. Confirm the Modify Profiling Task dialog by OK button.
4. Play with application to produce some profiling results.
5. Open Live Results window and check memory results. Select java.awt.event.MouseEvent class, invoke its popup menu and press Log Class History.
6. Switch to the enabled tab of Live Results 'History of java.awt.event.MouseEvent'. Scale to Fit the view.
7. Go to the profiled application and draw something.
8. Return back to history graph - both graphs should be increased evidently.
9. Stop profiling.


Test suite 6: Web Project

Setup: Open WebApplication project from web_ProfilerTestSpecification.zip. Use default configured IDE with default Tomcat server. Check configuration by invoking action Run Project on the project - project should be executed without any problem.

# Test Case Description Expected Results
6.1 Monitoring
[[[sanity | [sanity]]
1. Press Profile Main Project... in Profile toolbar. Confirm build script modification dialog.
2. Select Monitor Application in Profile WebApplication dialog, check Enable Threads Monitoring option and press Run button.
3. Check if Threads and VM Telemetry Overview windows are opened. Check if both windows are scrolling automatically their contents.
4. There should not be any thread in threads window with 'Profiler' or 'JFluid' in its name.
5. Select a thread in Threads view and invoke Thread details action in its popup menu.
6. Check if Threads (Details) tab in Threads window is selected and if it contains thread's name, 'pie chart' with statistics and line graph with colored thread's states.
7. Select one thread's state section in line graph. Check if Details sub tab with textual states information is opened and appropriate state is selected.
8. Invoke VM Telemetry button in Profiler Control Panel.
9. Press Zoom Out button (-) until whole Memory (Heap) graph panel is visible. Its horizontal scrollbar should be disabled.
10. Check if the zoomed out graph is alive.
11. Switch to Memory (GC) graph panel and press Scale to Fit button. Whole graph should be scaled into graphs' boundaries. Wait a moment - graph should be alive and it should be shrinked to the panel boundaries - horizontal scrollbar should be disabled. The Scale to Fit button should be replaced by Fixed Scale button.
12. Switch to Threads (Statistics) graph panel, select a part of the graph - press left mouse button somewhere on left side of the part and drag it to the right side of the part. The selected part should be fit into the graph's boundaries.
13. Open Threads window. Go To Profiled Application and close it.
14. Go back to profiler there should be shown Information dialog 'The profiled application is finished ...' Wait a few seconds - Threads window on backgroud should continue in showing new states of threads.
15. Confirm the dialog and check the Threads window. There should not be any thread with 'Profiler' or 'JFluid' in its name or any new thread created during finishing of application.
16. Profiled application should be stopped. All graphs should be stopped too.
17. Invoke Save Current View button in any graph panel. There should be opened a Select File or Directory with file system structure, preview of the view and PNG in Files of Type. Write a filename and confirm the dialog. Snapshot of selected view should be stored on disk as png file.
Monitoring is verified.
6.2 Analyze Performance 1. Invoke Profile -> Profile Main Project... action.
2. Select Analyze Performance task, leave default settings - Entire application and filter: Profile Only Project Classes. Press Run button.
3. Open Live Results window.
4. Switch to opened web browser and invoke application button Test a few times. (Do not close the browser than.)
Check Live Results content. There should be com.test.QueueTest.fullfill(java.util.List, int) at the first position of the list.
6.3 Analyze Performance - Actions SETUP: Running profiling from previous case.
1. Select the com.test.QueueTest.fullfill(java.util.List, int) method and invoke its popup menu. Invoke action Go To source in the menu.
2. QueueTest.java should be opened in java editor and scrolled to the fullFill method.
3. Write com.test into Method Name Filter at bottom of Live Results window and press Enter.
4. There should be displayed only com.test.* methods.
5. Select com.test.QueueTest.fullFill(java.util.List, int) method, invoke its popup menu and select Add To instrumentation Roots ... action.
6. Select Analyze Performance item and confirm dialog Select Settings configuration.
7. Stop Profiling by Profile -> Stop action and invoke Profile Main Project again.
8. Select Part of Application radio button, click edit... link next to "1 root method" label and check if there is the selected method. Press Add From Project button, switch to "Default Package View" and go to WebApplication -> Sources -> com -> test -> QueueTest -> Methods and check mergeEvents(java.util.List) method. Confirm the dialogs to the Profile WebApplication.
9. Select Profile All Classes filter and confirm the dialog by Run button.
10. Invoke Test button of the web appliacation in browser a few times as warmup, return back to profiler and invoke Reset Collected Results button.
11. Return back to browser and invoke Test action once again.
12. Go to Live Results window of profiler and check if DrillDown shows appropriate information.
6.4 Analyze Performance - Snapshots SETUP: Running profiling from previous case.
1. Press Take Snapshot action in Profiler Control Panel.
2. A new Snapshot window should be opened. It should contain methods com.test.QueueTest.mergeEvents and com.test.QueueTest.fullFill invoked in http threads (http-8084-Processor...).
3. Invoke Save Snapshot to Project, close it and open it again from Saved Snapshots list in Profiler Control Panel.
4. Press Ctrl+F, write "fullFill" into opened dialog textfield and confirm the Find in Results dialog. By buttons Find previous occurance and Find next occurance jump through all occurances of the method in snapshot.
5. Invoke popup menu on java.util.LinkedList$ListItr.checkForComodification() method and press Show Back Traces action. It should switch to Back Traces for: tab and show back traced methods of the selected method.
6. Select Combined tab of the snapshot window and double click on e.g. com.test.QueueTest.mergeEvents method in the bottom part of Combined view. It should jump into Java editor to the selected method's code.
7. Select the snapshot in Saved Snapshots list of Profiler Control Panel, press Save As... button, choose a path and file name and save the snapshot there.
8. Delete the snapshot by Delete button. Invoke Load... action in Saved Snapshots, select the previous file and confirm file dialog.
The snapshot should be opened in Snapshot window.
6.5 Analyze Memory Usage 1. Invoke Profile -> Profile Main Project... action. Select Analyze Memory Usage task.
2. Leave default settings - Record object creation only, Track every 10 object allocations, disabled Record Stack Trace for Allocations. Confirm dialog by Run button.
3. Press Live Results button and wait until the appliacation is started. Press Test button a few times in shown web browser and go back to Live Results window.
4. Write java.util into class name filter - at bottom of Live Results window and press enter.
TEARDOWN: Stop profiling.
java.util.LinkedList$Entry should be at the first position of Live Results window with hundred thousands of allocated objects.
6.6 Analyze Memory Usage - Objects Liveness 1. Invoke Profile -> Profile Main Project... action. Select Analyze Memory Usage task.
2. Set Record both object creation and garbage collection, Track every 10 object allocations, disable Record Stack Trace for Allocations. Confirm dialog by Run button.
3. Press Live Results button and wait until the appliacation is started. Press Test button a few times in shown web browser and go back to Live Results window.
4. java.util.LinkedList$Entry is probably at the first position of the list. Press Reset Collected Results button. Live Results window should be empty now.
5. Go to web browser and press Test button a few times. Switch back to IDE and look at the Live Results window. java.util.LinkedList$Entry is again at the first position of the list.
6. Select java.util.LinkedList$Entry and invoke its popup menu. Invoke action Stop Profiling this Class. The class should disappear. Try to reset collected results and press test button in browser and check if the class is not present in new results.
7. Sort classes by names (press header button of results table) and select e.g. the first class its name begins with org. Invoke action Stop Profiling Classes bellow this Line and Stop Profiling this Class from its popup menu. Reset results and press test button again. There should not be almost no classes which names begin with org or sun in Live Results window.
8. Press com.test.QueueTest$Event and invoke Go To Source... action from its popup menu.
TEARDOWN: Stop profiling.
QueueTest.java file is opened in editor window and cursor is scrolled to class Event ...
6.7 Analyze Memory Usage - Take Snapshots 1. Invoke Profile -> Profile Main Project... action. Select Analyze Memory Usage task.
2. Set Record bot object creation and garbage collection, Track every 1 object allocations, enable Record Stack Trace for Allocations. Confirm dialog by Run button.
3. Open Live Results window and invoke Test action of the application a few times. Go to the Profiler Control Panel and invoke Run GC button. Select e.g. java.lang.String and invoke action Take Snapshot and Show Allocation Stack Traces on it.
4. Press Ctrl+F - Find Action, write e.g. service to the find dialog textfield and confirm the dialog. Jump through all occurances of the string.
5. Invoke Save Snapshot to Project action. Select the snapshot in Saved Snapshots list of Profiler Control Panel, press Save As... button, choose a path and file name and save the snapshot there.
6. Delete the snapshot by Delete button. Invoke Load... action in Saved Snapshots, select the previous file and confirm file dialog.
TEARDOWN: Stop profiling.
The snapshot should be opened in Snapshot window.


Test suite 7: EJB - Glassfish

Setup: Open OrderSystem project from order_system_ProfilerTestSpecification.zip with all its required projects. Use IDE with fresh userdir, register latest stable version of glassfish server to Services -> Servers. Check if Java DB is running - Tools -> Java DB Database -> Start Server is disabled. Resolve all confilcts after opening of the project and try to Run the project by Run -> Run Main Project action to check if all settings are correct.

# Test Case Description Expected Results
7.1 Integration
[[[sanity | [sanity]]
1. Select opened project and invoke popup menu (in Projects view).
2. Popup menu should contain enabled Profile Project menu item.
3. Set the project as main project (if it is not) and check if menu action Profile -> Profile Main Project is enabled.
4. Profile Main Project action in Profile toolbar should be enabled too.
5. Press Profile Main Project action in Profile toolbar. Select default Java platform in Select Java Platform for Profiling dialog, check Always use the selected platform for profiling and confirm the dialog.
6. Confirm opened Question dialog (This is the first time this project ...). Press Cancel button in opened Select Profiling Task dialog.
Project's build script should be updated - there should be added line:
<import file="nbproject/profiler-build-impl.xml"/>.
Origin build script should be backed up under name build-before-profiler.xml. Project's nbproject folder should contain profiler-build-impl.xml file.
7.2 Integration Server
[[[sanity | [sanity]]
1. Invoke Profile -> Profile Main Project menu action. Select Monitor Application and confirm the dialog by Run button.
TEARDOWN: Stop Application server by invoking of action Stop on Services -> Servers -> <Glassfish Server name>.
The application server should be started in profile mode. The application should be running, Threads view should show threads on the server.
7.3 Profile Mode
[[[sanity | [sanity]]
1. Invoke the popup menu on Services -> Servers -> <Glassfish Server name> and press Start in Profile Mode.
2. Confirm the Information dialog about used JDK and agent port number. There should be shown log from Application Server with Profiler Agent: Initialized succesfully at the end.
3. Invoke Attach profiler action from ide toolbar. Select Project OrderSystem. Click "change" link next to "Attach mode" label in the bottom to open Attach Wizard, select J2EE Web/App Server target type, server type glassfish, local attach method, direct attach invocation, press next and finish the wizard.
4. Select Monitor Application, enable Threads and confirm the Select Profiling Task dialog by Attach button.
TEARDOWN: Detach the server. Confirm Question dialog 'Do you want to terminate the profiled application upon detach?'
Profiler should attach to application server. Server should be started and threads view should show its threads states.
7.4 Analyze Performance - Project Classes 1. Invoke Profile -> Profile Main Project... action. Select Analyze Performance task, Entire Application, Profile Project & Subproject Classes. Press Run button.
2. Wait until the application on server is not started. Open new customer form in web browser and Save new customer.
3. Invoke Take Snapshot. Browse taken snapshot and Live Results window in ide. There should be measured only methods from org.shop and org.apache.jsp packages.
4. Go to Live Results window and double click on org.shop.OrderBean.save() method. There should be opened java editor of Order.java with cursor on 'save' method.
5. Select method org.shop.OrderBean.save() again, invoke its popup menu and invoke Add to Instrumentation roots action, select Analyze Performance and confirm the Select Settings Configuration dialog by OK button.
6. Invoke Profile Main Project action, confirm question 'Profiling Session is currently in progress'. Select CPU, Analyze Performance task, Part of Application, Profile All Classes. There should be "No root methods" label next to Part of Application option.
7.5 Analyze Performance - Root Methods SETUP: Continue previous test case.
1. Press "define..." link next the Part of Application. Invoke Add From Project button. Check if Web application view is selected. Open tree OrderSystem -> JSPs and check checkbox in front of listOrders.jsp. Confirm Select Root Methods dialog and Specify Root Methods dialog by OK buttons. Press Run button of Profile OrderSystem dialog.
2. Wait until the application is started. Open Live Results window. Go to browser and make some action in the application - press Java Server Faces Welcome Page link.
3. Watch Live Results and in profiler control panel Basic Telemetry - Instrumented Methods. Live Results should be empty and number of Instrumented methods should be small.
4. Fill a new customer data in welcomeJSF.jsp page in browser and confirm it by Save button.
5. Profiler should start profiling of the application. Opened live results are filled with measured methods, Instrumented methods value is non zero.
6. Wait until the Instrumented methods number is constant.
7. Invoke Reset Collected Results action. Live Results window should be emptied than.
8. Return back in browser to new customer form, fill it and confirm again by Save button.
9. Invoke Take Snapshot action. Save taken snapshot to project.
New results should be measured much faster then at first time. The methods are all instrumented. There should be root method of whole call tree named org.apache.jsp.listOrders_jsp._... in the snapshot.
7.6 Analyze Memory Usage 1. Invoke Profile -> Profile Main Project... action. Select Monitor application task and press Run button.
2. Wait until the application is fully started - web browser is opened with the application's web page.
3. Invoke Profile -> Modify Profiling menu action and select Analyze Memory Usage task. Select Record both object creation and garbage collection, Track every 1o object allocations and Record Stack Trace for Allocations. Confirm dialog by OK button.
4. Go to the Save customer form and save a new customer, invoke Reset Collected Results in ide and save another customer in web browser.
5. Invoke Take Snapshot and Save Snapshot to Project.
There should be catched all classes allocated by invoking of the application action in the snapshot. This use case should be used for all memory profiling of big applications - start monitoring and switch to profiling after the server and application is started to partialy eliminate profiling of big amount of server's classes.
7.7 Glassfish v3
[[[sanity | [sanity]]
Setup: Open WebApplication project from web_ProfilerTestSpecification.zip. Open properties of the project and as a server set the Glassfish v3 server.
1. Perform the same testing as for Order system project.
The project should run without any problems.


Test suite 8: EJB - JBoss

Setup: Open JBossTest project from jboss_test_ProfilerTestSpecification.zip with all its required projects. Use IDE with fresh userdir, register latest stable version of jboss server to Servicess -> Servers.. Resolve all confilcts after opening of the project and try to Run the project by Run -> Run Main Project action to check if all settings are correct.

# Test Case Description Expected Results
8.1 Integration 1. Select opened project and invoke popup menu (in Projects view).
2. Popup menu should contain enabled Profile Project menu item.
3. Set the project as main project (if it is not) and check if menu action Profile -> Profile Main Project is enabled.
4. Profile Main Project action in Profile toolbar should be enabled too.
5. Press Profile Main Project action in Profile toolbar.
6. Confirm opened Question dialog (This is the first time this project ...). Press Cancel button in opened Select Profiling Task dialog.
Project's build script should be updated - there should be added line:
<import file="nbproject/profiler-build-impl.xml"/>.
Origin build script should be backed up under name build-before-profiler.xml. Project's nbproject folder should contain profiler-build-impl.xml file.
8.2 Integration Server 1. Start the application by Run -> Run Main Project menu action.
2. Check if the application is running - browser should be opened with application's page.
3. Invoke Profile -> Profile Main Project menu action. Select Monitor Application and confirm the dialog by Run button.
TEARDOWN: Stop Application server by invoking of action Stop on Services -> Servers -> <Glassfish Server name>.
The application server should be stopped and started again in profile mode. The application should be running, Threads view should show threads of the server.
8.3 Profile Mode 1. Invoke the popup menu on Services -> Servers -> <Glassfish Server name> and press Start in Profile Mode.
2. Confirm the Information dialog about used JDK and agent port number. There should be shown log from Application Server with 'Profiler Agent: Waiting for connection on port ...' at the end.
3. Invoke Attach profiler action from ide toolbar. Select Project JBossTest. Open Attach Wizard, select J2EE Web/App Server target type, server type jboss 4.x, local attach method, direct attach invocation, preee next and finish the wizard. Select Monitor Application and confirm the Select Profiling Task dialog by Attach button.
TEARDOWN: Detach the server. Confirm Question dialog 'Do you want to terminate the profiled application upon detach?'
Profiler should attach to application server. Server should be started and threads view should show its threads states.
8.4 Analyze Performance - Project Classes 1. Invoke Profile -> Profile Main Project... action. Select Analyze Performance task, Entire Application, Profile Only Project Classes. Press Run button.
2. Wait until the application on server is not started. Save new id and value in web browser.
3. Invoke Take Snapshot. Browse taken snapshot and Live Results window in ide. There should be measured only methods from web., util. and ejb. packages.
4. Go to Live Results window and double click on web.Save.processRequest method. There should be opened java editor of Save.java with cursor on 'processRequest' method.
5. Select method web.Save.processRequest in snapshot again, invoke its popup menu and invoke Add to Instrumentation roots action, select Preset: Part of Application and confirm the Select Settings Configuration dialog by OK button.
6. Invoke Profile Main Project action, confirm question 'Profiling Session is currently in progress'. Go to analyze Performance, Part of Application - Select. There should be method web.Save.processRequest(...) in the Root Methods list in Specify root Methods dialog. Select and remove the method. Confirm the dialog by OK button.
8.5 Analyze Performance - Root Methods 1. Invoke Profile -> Profile Main Project... action. Select Analyze Performance task, Part of Application, Profile All Classes.
2. Press Select button next the Part of Application. Invoke Add From Project button. Check if The Logical view is selected. Open tree JBossTest -> Web Resources -> Servlets and check checkbox in front of /Save servlet. Confirm Select Root Methods dialog and Specify Root Methods dialog by OK buttons. Press Run button of Select Profiling Task dialog.
3. Wait until the application is started. Open Live Results window. Go to the browser and invoke Show action for some id.
4. Watch Live Results and in profiler control panel Basic Telemetry - Instrumented Methods. They should be empty.
5. Fill a new id and value data and save them by Save button.
6. Profiler should start profiling of the application. Opened live results are filled by measured methods, Instrumented methods value is non zero.
7. Wait until the Instrumented methods number is constant for a time.
8. Invoke Reset Collected Results action. Live Results window should be emptied than.
9. Return back to browser and Save new id and value.
10. Invoke Take Snapshot action. Save taken snapshot to project.
New results should be measured much faster then at first time. The methods are all instrumented. There should be root method of whole call tree named web.Save.doGet(... in the snapshot.
8.6 Analyze Memory Usage 1. Invoke Profile -> Profile Main Project... action. Select Monitor application task and press Run button.
2. Wait until the application is not fully started - web browser is opened with the application's web page.
3. Invoke Profile -> Modify Profiling menu action and select Analyze Memory Usage task. Select Record both object creation and garbage collection, Track every 1o object allocations and Record Stack Trace for Allocations. Confirm dialog by OK button.
4. Save a new id and value in web browser, invoke Reset Collected Results in ide and save a new id again.
5. Invoke Take Snapshot and Save Snapshot to Project.
There should be catched all classes allocated by invoking of the application action in the snapshot.
This use case should be used for all memory profiling of big applications - start monitoring and switch to profiling after the server and application is started to partialy eliminate profiling of big amount of server's classes.


Test suite 9: J2EE Servers Autointegration

# Test Case Description Expected Results
9.1 Autointegration of Tomcat SETUP: Open WebApplication project from web_ProfilerTestSpecification.zip. Copy tomcat server distributed with netbeans to some temporary folder. Set its manager account - add role manager to some user in <TOMCAT_HOME>/conf/tomcat-users.xml and add the server to Services -> Servers node. Open the project's WebApplication properties and set the configured tomcat to its Run -> Server setting.
1. Invoke Run Project to check if all is working and to deploy the application on the server. Invoke Stop on the server in Services -> Servers node.
2. Invoke Attach Profiler action from ide's toolbar. Select Project: WebApplication and task Monitor Application, enable Threads. Click "define..." link in the bottom.
3. Select Target Type J2EE Web/App Server, J2EE Web/App Server Type Tomcat 5.5, Attach Method Local, Direct Attach Invocation. Press Next button two times, select Automatic integration and press Next button. To Tomcat Installation select path to prepared copy of tomcat and press Next button.
4. Review integration steps and press Perform button. Check in prepared tomcat home if the changes mentioned in integration steps were done - copy and configure catalina script. Press Next button. Check Automatically start the server checkbox and Finish the wizard.
5. There should be opened terminal window with 'Profiler Agent: Waiting for connection on port ...' message at the end.
6. Invoke Attach button in Select Profiling Task dialog. Monitoring of the tomcat should start. Check VM Telemetry and Threads windows if there are live data.
7. Open a web browser and go to the URL: http://localhost:8080/test. Perform Test action there.
TEARDOWN: Detach from the Profiled application and finish the server.
E.g. in Memory (Heap) graph the invoked actions should be visible by changes in memory consumption of the tomcat server.
9.2 Autointegration of Glassfish SETUP: Open OrderSystem project from order_system_ProfilerTestSpecification.zip with all its required projects. Use IDE with fresh userdir, register glassfish server to Servicess -> Servers. Check if Java DB is running - Tools -> Java DB Database -> Start Java DB Server is disabled.
1. Invoke Run Project to check if all is working and to deploy the application on the server. Invoke Stop on the server in Services -> Servers node.
2. Invoke Attach Profiler action from ide's toolbar. Select Project: OrderSystem and task Monitor Application, Enable Threads Monitoring. Click "define..." link in the bottom.
3. Select Target Type J2EE Web/App Server, J2EE Web/App Server Type Sun Application Server 9 / GlassFish, Attach Method Local, Direct Attach Invocation. Press Next button two times, select Automatic integration and press Next button. To Server Installation textfield select path to the glassfish server and press Next button.
4. Review integration steps and press Perform button. Check in glassfish home folder if the changes mentioned in integration steps were done - backup and modify asenv.conf and domain.xml. Press Next button. Check Automatically start the server checkbox and Finish the wizard.
5. There should be opened terminal window with 'Profiler Agent: Waiting for connection on port ...' message at the end.
6. Invoke Attach button in Select Profiling Task dialog. Monitoring of the glassfish server should start. Check VM Telemetry and Threads windows if there are live data.
7. Wait for application server startup is completed. Open a web browser and go to the URL: http://localhost:8080/OrderSystem-war. Go to the welcome page and Save some customers there.
TEARDOWN: Detach from the Profiled application and finish the server.
E.g. in Memory (Heap) graph the invoked actions should be visible by changes in memory consumption of the glassfish server.


Test suite 10: Attach Profiler

# Test Case Description Expected Results
10.1 Dynamic Attach JDK1.6 SETUP: Create or open a sample J2SE project AnagramGame - File -> New Project -> Samples -> General -> Anagram Game. The project should be set as main project.
1. Invoke Run Project to start the application.
2. Invoke Attach Profiler action from ide's toolbar. Select Project: AnagramGame and task Analyze Memory Usage - Record both object creation and garbage collection, Record Stack Track for Allocations. Click "define..." link in the bottom.
3. Select Application Target Type, Local Attach method, Dynamic (JDK 1.6) Attach invocation. Press Next button and Finish button (the last step can be skipped).
4. Press Attach button in Select Profiling Taks dialog.
5. Select the java process with com.toy.anagrams.ui.Anagrams class in Select Process dialog. Confirm it by OK button.
6. Open Live Results window and perform some actions in profiled AnagramGame application.
7. Live Results window should contain measured data of memory profiling.
8. Open Threads view and enable Threads monitoring.
9. Perform Detach from the Profiled application action and do not confirm to terminate the profiled application in appeared dialog - press No button.
10. Invoke Attach Profiler action again with the same settings.
11. Select the java process with com.toy.anagrams.ui.Anagrams class in Select Process dialog. Confirm it by OK button.
12. Enable monitoring in Threads window.
13. Perform Detach from the Profiled application action and do confirm to terminate the profiled application in appeared dialog - press Yes button.
The application should run after the first detach. After second attach there should not be some new threads (Profiler or Jfluid) in Threads view. After the second detach both application and profiling should be finished.
10.2 Local - Direct - Application
[[[sanity | [sanity]]
SETUP: Build the sample J2SE project AnagramGame, go to its dist folder outside of ide and create an execution script there - containing simple: java -jar anagrams.jar. Try to execute the application by the script and close the application if the script is working.
1. Invoke Attach Profiler action from ide's toolbar. Select Project: AnagramGame and task Analyze Performance - Entire Application, Profile All Classes. Click "define..." link in the bottom.
2. Select Application Target Type, Local Attach method, Direct Attach invocation. Press Next button two times. Copy the extra argument from Step 1 and paste it to the prepared execution script before '-jar'. Press Finish button in Attach wizard.
3. Start the application by modified script. Go to the ide and perform Attach button in Select Profiling Task dialog.
4. Open Live Results window and check measured methods. Perform some actions in profiled application and check Live Results. Live Results should be non empty with appropriate measured methods.
5. Close the profiled application by its Close window button. Application Finished dialog should be opened in profiler - profiling should be finished.
6. Start the application again, attach to it and wait until the application is running.
7. Perform Detach from the Profiled Application, press No button in appeared dialog.
TEARDOWN: close the profiled application.
The application should run without problem.
10.3 Local - Direct - Applet SETUP: Open AppletTest project from applet_ProfilerTestSpecification.zip archive, open its applet.TestApplet class and execute it - by Run file action. Close the AppletBrowser. AppletTest/build folder should contain generated TestApplet.html file.
1. Invoke Attach Profiler action from ide's toolbar. Select Project: AppletTest and task Analyze Performance - Entire Application, Profile Only Project Classes. Click "define..." link in the bottom.
2. Select Applet Target Type, Local Attach method, Direct Attach invocation. Press Next button two times. Perform first two steps in manual integration panel. Press Finish button in Attach wizard. Cancel the Attach Profiler dialog for now.
3. Go to files view and perform action View on AppletTest -> build -> TestApplet.html file.
4. Web browser with blocked applet should be opened.
5. Invoke Attach Profiler action from ide's toolbar. Confirm the Select Profiling Task dialog by Attach button.
6. The applet should be started, perform some actions there. Open Live Results window and check measured data - there should be measured methods of the applet.
7. Try to close the browser. Application Finished dialog should appear in ide. Confirm it to take snapshot. There should be a new non empty taken snapshot opened.
8. View the html file again, attach to it and wait until the applet started.
9. Perform Detach from the Profiled Application, press No button in appeared dialog.
TEARDOWN: Clear the settings added to the ControlPanel.
The applet should work without change. Profiling should be finished.
10.4 Local - Direct - Tomcat SETUP: Open WebApplication project from web_ProfilerTestSpecification.zip. Copy tomcat server distributed with netbeans to some temporary folder. Set its manager account - add manager role to some user in <TOMCAT_HOME>/conf/tomcat-users.xml and add the server to Services -> Servers node. Open the project's properties and set the configured tomcat to its Run -> Server setting.
1. Invoke Run Project to check if all is working and to deploy the application on the server. Invoke Stop on the server in Services -> Servers node.
2. Invoke Attach Profiler action from ide's toolbar. Select Project: WebApplication and task Monitor Application. Click "define..." link in the bottom.
3. Select Target Type J2EE Web/App Server, J2EE Web/App Server Type Tomcat 5.5, Attach Method Local, Direct Attach Invocation. Press Next button two times, select Manual integration and press Next button. Perform all steps in Manual integration panel and Finish the wizard.
4. Invoke Attach button in Select Profiling Task dialog. Monitoring of the tomcat should start. Check VM Telemetry and Threads windows if there are live data.
5. E.g. in Memory (Heap) graph the invoked actions should be visible by changes in memory consumption of the tomcat server.
6. Detach from the Profiled application and do not terminate the profiled application.
7. Check application in web browser.
The application (and server) should be running.
10.5 Local - Direct - GlassFish SETUP: Open OrderSystem project from order_system_ProfilerTestSpecification.zip with all its required projects. Use IDE with fresh userdir, register glassfish server to Servicess -> Servers. Check if Java DB is running - Tools -> Java DB Database -> Start Java DB Server is disabled.
1. Invoke Run Project to check if all is working and to deploy the application on the server. Invoke Stop on the server in Services -> Servers node.
2. Invoke Attach Profiler action from ide's toolbar. Select Project: OrderSystem, task Monitor Application, enable Threads Monitoring. Click "define..." link in the bottom.
3. Select Target Type J2EE Web/App Server, J2EE Web/App Server Type Sun Application Server 9 / GlassFish, Attach Method Local, Direct Attach Invocation. Press Next button two times, select Manual integration and press Next button. Perform all steps in Manual integration panel and Finish the wizard.
4. Invoke Attach button in Attach Profiler dialog. Monitoring of the glassfish should start. Check VM Telemetry and Threads windows if there are live data.
5. E.g. in Memory (Heap) graph the invoked actions should be visible by changes in memory consumption of the tomcat server.
6. Detach from the Profiled application and do not terminate the profiled application.
7. Check application in web browser.
The application (and server) should be running.
10.6 Remote Attach Application SETUP: Install appropriate remote pack on a remote machine. Setup calibrate script in remote pack's bin folder and start calibration.
1. Build e.g. AnagramGame sample project in ide, pack and send it to the remote machine. Unpack it there and create an execution script in its dist folder - should contain e.g.
java -jar /space/tmp/AnagramGame/dist/anagrams.jar
Use absolute path in the script.
2. In local IDE invoke Attach Profiler ... action, select Project: <External Application>. Click "define..." link in the bottom.
3. Select Application Target Type, Remote Attach Method, Direct Attach Invocation, press Next button. Write remote machine's name and select its OS name, press Next button. Review the attach settings and press Next button.
4. In manual integration instructions go to Step 3 (Step 1 and Step 2 should be done before) and rewrite the command to remote application's start script as JVM option - before '-jar': java -agentpath:... -jar anagrams.jar. Finish the wizard.
5. Start remote application with modified script - there should be messages about successfully started profiler agent in the terminal.
6. Go back to local ide, select Analyze Performance task, Entire Application, Profile All Classes. Click Create Custom link. Select "From existing configuration" option and select "Analyze Performance" item. Confirm New Custom Configuration (CPU) dialog. Click Advanced Setting link and check Profile threads started after entering any of the root methods option. Confirm Attach Profiler dialog by Attach button.
7. The remote application should be started, local profiler should show measured data e.g. in VM Telemetry Overview. Open Live Results window in local profiler. Check if some reuslts are available.
8. Invoke Take Snapshot action, wait until non empty snapshot is opened and store it by Save action. The snapshot should appear in Saved Snapshots part of Profiler Control Panel - project <Global>
9. Invoke Profiler -> Detach... menu action. A Question dialog should appear asking if you want to terminate the profiled application upon detach. Confirm it by Yes button. The remote application should be closed, local profiling finished.
10. Find java.awt.EventDispatchThread.run() method in saved snapshot. Invoke its popup menu and press Add to Instrumentation roots action. In Select Settings Configuration select Analyze Perfomance configuration and confirm the dialog.
11. Start remote application by modified start script.
12. Invoke Attach Profiler ... action, select Project: <External Application>, choose Analyze Performance, Part of Application (check by "edit" link if there is selected the root method) and Profile All classes. Confirm Attach Profiler dialog by Attach button.
13. Remote application should be started, local profiler should show measured data of cpu profiling.
14. Perform Modify Profiling action from Profiler Control Panel. Select Analyze Memory Usage, Record both object creation and garbage collection and Record Stak Trace for allocations. Confirm the Modify Profiling dialog by OK button.
15. Invoke some action in remote application and check if results in local profiler are changed.
16. Open VM Telemetry window, select Memory (Heap) and scroll to the end of graph. Invoke Run GC action from Profiler control Panel and watch the graph. Value of Used Heap memory should be decreased.
17. Invoke Take Snapshot action, wait until the snapshot window is opened and invoke Save Snapshot action. The snapshot should be non empty and successfully stored.
18. Invoke Detach action and press No button in Question dialog asking if the profiled application should be closed.
There should be messages about closing of profiler agent in the log of remote application, e.g. Connection with agent closed. The application should run after the agent is closed.
10.7 Remote Attach Tomcat SETUP: Install appropriate remote pack on a remote machine. Setup calibrate script in remote pack's bin folder and start calibration. Install appropriate Tomcat server on the machine. Use e.g. WebApplication test project from web_ProfilerTestSpecification.zip, build it in ide and send its dist/WebApplication.war archive to the remote server. Put the application archive to <TOMCAT_HOME>/webapps folder there.
1. In local IDE invoke Attach Profiler ... action, select Project: <External Application>. Click "define..." link in the bottom.
2. Select J2EE Web/App Server Target Type, Tomcat 5.5 Server Type, Remote Attach Method, Direct Attach Invocation, press Next button. Write remote machine's name and select its OS name, press Next button. Review the attach settings and press Next button.
3. Go to the Step 3 in the integration instructions list and perform all steps to the end.
4. Finish the Attach wizard.
5. Select Analyze Performance, Entire Application. Click Edit filter sets link.
6. In Customize Filter Set press Add, write 'test' to filter set name, select Inclusive type, invoke Edit Global Filters, press New in Edit Global filters dialog and write e.g. 'Test Classes' as filter name and 'com.test.' as the new filter value. Confirm dialog by OK button. Add the 'Test Classes' filter to 'test' filter and confirm the dialog by OK button.
7. Check if the 'test' filter is selected in Filter combobox. Click Create Custom link. Select "From existing configuration" option and select "Analyze Performance" item. Confirm New Custom Configuration (CPU) dialog. Click Advanced Setting link and check Profile threads started after entering any of the root methods option. Press Attach button.
8. Open web browser with remote application and press Test button a few times.
9. Check if Live Results is changing. Open VM Telemetry and check memory usage of remote server. Open Threads window and enable Threads monitoring.
10. Select com.test.QueueTest.test() method, invoke its popup menu and perform Add to Instrumentation Roots action. Select Analyze Performance and press OK button.
11. Invoke Modify Profiling action and verify that Analyze Performance and Part of Application options are selected. Choose filter Profile All Classes. Press OK button.
12. Go to the web browser and invoke Test action a few times - as 'warmup' to eliminate additional times spent in instrumentation and optimalization of code.
13. Press Rest Collected Results button in profiler and press the Test button in web breowser once more. Wait for results are updated and invoke Take Snapshot.
TEARDOWN: Detach from the Profiled Application and terminate it.
Snapshot now contains detailed and relativelly precise measurement of the test method performance.


Test suite 11: Remote Packs

# Test Case Description Expected Results
11.1 Installation Download a remote pack for used remote os and jdk version to remote machine. Unpack it to a folder. There should be a bin folder with calibrate and profile script and lib folder with jfluid-server*.jar libraries and lib/deployed/<jdk_version>/<os>/<libraries>
11.2 Calibration 1. Edit bin/calibrate.sh(.bat) file. Set JAVA_HOME property to existing JVM home folder.
2. Execute the calibrate.sh script.
There should be messages shown in terminal confirmed successfull calibration - e.g. 'Calibration performed successfully', 'Aproximate time in one ...' Therer should be created folder <USER_HOME>/.nbprofiler with calibration data file - machinedata.<JDK_VERSION>
11.3 Execution 1. Build e.g. AnagramGame sample project, pack and send it to the remote machine. Unpack it there and create an execution script in its dist folder - should contain e.g.
java -jar /space/tmp/AnagramGame/dist/anagrams.jar
Use absolute path in the script.
2. Try to execute the script - application should start without any problem.
3. Edit the start script again - replace 'java' command by path to the remote pack's 'profile' script (use absolute paths). Start the script.
4. There should be messages about successfull initializing of profiler agent and number of tcp port in terminal.
5. Try to remote attach to the profiled application - see Attach Profiler suite.
After attach from profiler client (ide) the remote application should be started and profiling data send back to the profiling ide. After closing of profiled application the ide should recognize it.


Test suite 12: HeapWalker

Setup: Prepare a heapdump. Start <JDK16_HOME>/bin/jconsole, select a PID with org.netbeans.Main name and invoke Connect. Go to MBeans tab, select com.sun.management -> HotSpotDiagnostic -> Operations. In the panel write a file path (e.g. /tmp/nbheapdump) to the p0 textfield next to dumpHeap button and press the dumpHeap button. There should invoke Info dialog about successfully invoked method - close the jconsole.

# Test Case Description Expected Results
12.1 Summary 1. Profile -> Take Heap Dump (Destination - Profiled project, custom directory)
2. Open the heapdump by Profile -> Load Heap Dump menu action.
3. Check Summary information.
12.2 Classes
[[[sanity | [sanity]]
SETUP: Switch to Classes page
1. Check sorting of heapdump table by each column - press the column title bar.
2. Press Show or hide columns small button in the right-top corner of the table, uncheck Instances, Size, Filter items. There should remain only two columns in heapdump table. Press the button again and enable all items in the menu. All columns should be shown as before.
3. Go to Class Name Filter (bottom line of heapdump panel) and select Ends with filter type. Write Buffer and press Enter. Classes table should contain only 'Buffers'. Press Clear current filter_ - button with red cross on the filter line. Table should contain all classes again. Try to select e.g. Regular expression filter type - write .*Map to filter textfield and press Apply new filter button - there should be shown all 'Maps'. Press Clear current filter button.
4. Select e.g. java.lang.String class and press Static Fields button (top-right corner of window). Static Fields panel should be inserted to the right side of heapdump window. There should be shown all static fields of String class (class, serialVersionUID etc.). There should be shown icons legend in the bottom line of the window (Array Type, Object Type etc.)
5. Press Instances button. An empty panel should be shown with No class selected message. Switch back to Classes view.
6. Select e.g. java.lang.String and invoke its popup action Go to Source. It should open source file of java.lang.String class in editor and move cursor to class definition. (It doesn't work on some JDKs on mac os x).
7. Select e.g. java.io.File class in Classes heapdump window, invoke its popup menu and press Show in Instances View action.
It should change view to Instances with three panels Instances, Fields and References of the class java.lang.String.
8. Click "Compare with another heap dump" link at the top right of classes tab. Select another dump for comparison. Check the differences.
12.3 Instances
[[[sanity | [sanity]]
1. Try to close and open again all panels in Instances view. Use 'x' buttons in right-top corner of each panel and buttons with icons on heapdum window toolbar. Try to change size of panels - after closing and opening again the size of panel should be preserved.
2. Check the legend line bellow the toolbar - there should be name of class and some statistics about it.
3. Try to switch on and off all available columns in each panel by Show or hide columns button (top-right corner of each panel).
4. Try to sort instances list table by particular columns - press title of the column.
5. Find a table cell (mostly in Fields or References panels) which`s content is not fully displayed (ends by ...). Move mouse cursor over the cell and check if its full content is shown in appeared tooltip.
6. Select a node of non-primitive type in Fields or References panel. Invoke its popup menu and press Show Instance action. Instances of the class should be displayed in heapdump window and there should be selected the instance in Instances list panel.
7. Select a node of non-primitive type in Fields panel. Invoke its popup menu and press Show in Classes View action. Classes view should be displayed in heapdump wind with the selected class. Return back to Instances.
8. Select a node of non-primitive type in Fields or References panel. Invoke its popup menu and press Go to Source action. The editor with source of the class should be opened in ide.
9. Click on the "retained size" link at the top right - the retained sizes should be computed and displayed in new column.
12.4 OutOfMemory Protection SETUP: Adapt code of opened AnagramGame project. Open Anagrams.java class. Add code ArrayList data = new ArrayList(24); as class field and append the code data.add(new int [500000]); into private void guessedWordActionPerformed(java.awt.event.ActionEvent evt) method. Open Properties of the AnagramGame project, select Run category and insert '-Xmx2m' into VM Options line.
1. Invoke Profile -> Profile Main Project... action. Select Monitor Application task and confirm the dialog by Run button.
2. Open terminal window of ide or messages.log file from userdir and check if there is a message 'Profiler.OutOfMemoryProtection: Enabled' at start of current profiling.
3. Open VM Telemetry window in ide and check select Memory (Heap) view.
4. Switch to the profiled application and press Guess button two times. Used Heap graph in VM Telemetry should be increased about 3.8MB. Check the gray section of the graph starting at 6MB - it should be representation of Max Heap value of JVM.
5. Press the Guess button again in the application (or a few times if the max heap is not on 6MB). There should be shown a Question dialog - Profiled application crashed and generated heap dump... Confirm it by Yes button.
6. A new heap dump should be opened. Check Output window of profiled application - above the OOM exception there should be a log 'Dumping heap to ...' describing file where the heap dump is created. Check it on filesystem.
TEARDOWN: Clean all changes of AnagramGame code, remove -Xmx2m from the project's command line.
12.5 OQL console
[[[sanity | [sanity]]
SETUP: create/open standart Anagram game project.
1. Start profiling the project. Use Memory settings, Record both object creation and garbage collection, check Record stack traces for allocations.
2.Wait until the application starts. Invoke Profile/Take Heap dump(confirm all dialogs)).
3. In open heap dump go to OQL console tab. In Saved queries select one, click on Open button. In Query editor click on Execute.
4. Close the Heap dump and open it again (or create a new one). Repeat the query.
There should be some result in the Query result window, or message that query returned no results.

Test suite 13: Profiling Points

Comment: Profiling points should be tested in each project type - j2se, web, nb project, applet and ejb.

# Test Case Description Expected Results
13.1 Stopwatch and Reset Results
[[[sanity | [sanity]]
1. Open Anagrams.java source file in editor. Move cursor to the first line of method guessedWordActionPerformed (there is if command). Invoke popup menu there and press Profiling -> Insert Profiling Point menuitem.
2. Select Profiling Point Type Stopwatch. Click Next button. Confirm default settings by pressing Finish button.
3. The stopwatch annotation should be added to the line, line should have different background color.
4. Invoke popup menu on the annotation, there should be Profiling Point submenu containing Enabled, Customize, Delete and Go To End Point options. Select Go To End Point. Warning message is shown. Select Profiling Point > Customize menu item.
5. Choose Timestamp and duration radio button. Location (end) controls gets enabled. Move cursor to the last line of method guessedWordActionPerformed (there is command guessedWord.requestFocusInWindow();) in editor. Push Current Line button in the Location (End) section. Click OK button.
6. Invoke popup menu on the end stopwatch annotation, select Profiling Point -> Go To Start Point. Cursor should be moved to the start stopwatch line. Invoke popup menu on the start stopwatch annotation, select Profiling Point -> Go To End Point. Cursor should be moved to the end stopwatch line.
7. Move cursor to the first line of method nextTrialActionPerformed. Press Shift-F9. Reset Results profiling point is inserted and Customize Profiling Point dialog is shown. Confirm it by OK button.
8. Invoke Profile -> Profile Main Project... action. Select Analyze Performance task, set Entire application and Filter: Profile Only Project Classes. Ensure that Use defined Profiling Points option is enabled and Show active Profiling Points link opens list with two Profiling Points. Press Run button.
9. Wait until the application is started and press its Guess button five times. Profiling Points window should show 5 hits for Stopwatch profiling point and no results available for reset results profiling point.
10. Click report link for Stopwatch profiling point. Report window should be shown, for each hit timestamp and duration should be shown.
11. Open Live Results. There should be some measured methods. Click New Word button in profiled application. Live Results should show only two methods from now.
12. Click report link for Reset Results in Profiling Points window. Report window should be shown, for each hit timestamp should be shown.
13. Stop Profiling. Start Profing again having User defined Profiling Points option disabled. Play with application. No results for profiling points should be shown. Stop Profiling.
14. Right-click Reset Results in Profiling Points window and select Show in Source menu item. Cursor should be moved to first line of nextTrialActionPerformed method.
15. Rigth-click its annotation and select Profiling Points > Enabled. Reset Results Profiling point should be shown gray in editor and Profiling Points window.
16. Right-click Stopwatch in Profiling Points window and select Show Start in Source menu item. Cursor should be moved to first line of guessedWordActionPerformed method. Right-click Stopwatch in Profiling Points window and select Show End in Source menu item. Cursor should be moved to last line of guessedWordActionPerformed method.
17. Rigth-click its annotation and select Profiling Points > Delete. Stopwatch Profiling point should be removed from editor and Profiling Points window.
13.2 Take Snapshot 1. Open Anagrams.java source file in editor. Move cursor to the first line of method nextTrialActionPerformed (wordIdx ...). Invoke popup menu there and press Profiling > Insert Profiling Point menuitem.
2. Select Profiling Point Type Take Snapshot, click Next. Enable Reset Results after taking snapshot. Choose Begin for Line. Click Finish.
3. Invoke Profile -> Profile Main Project... action. Select Analyze Performance task, set Entire application and Filter: Profile All Classes. Ensure that Use defined Profiling Points option is enabled and Show active Profiling Points link opens list with two Profiling Points. Press Run button.
4. Wait until the application is started and press its New Word button two times. Close the application. Do not take snapshot.
TEARDOWN: In profiling points window remove both profiling points.
There should be two new snapshots in Saved Snapshots part of Profiler Control Panel. Profiling Points window should show 2 hits, and report for Take Snapshot profiling point should contain links to both snapshots. No results should be available for Reset Results Profiling Point.
13.2 Timed- and Triggered- Profiling Points 1. Open Anagrams.java source file in editor. Push Shift key and press F9 key several times until Timed Take Snapshot is not chosen. Release Shift key. Customize Profiling Point dialog is shown.
2. Choose Heap dump option, To directory option and enter some directory. Choose Take every option, ensure that 1 minute is chosen. Press OK.
3. Press Add Profiling Point button in Profiling Points window. Select Triggered Take Snapshot. Click Next. Select surviving generations in Take when list, enter 5 generation, choose Take every time option. Click Finish.
4. Invoke Profile -> Profile Main Project... action. Select Memory task. Confirm dialog by Run button.
5. When profiled application is started open VM Telemetry Overview window. Verify that surviving generations exceeded 5.
6. Open Profiling Points window. There should be one hit for triggered take snapshot profiling point if surviving generations value exceeded 5 and several hits for timed take snapshot profiling point.
7. Open report for Timed Take Snapshot profiling point. There should be several hits with 1 minute interval between them.
8. Stop profiling.
9. Verify that all taken snapshots and heapdumps are available and are placed in the specified locations. Try to load heap dumps using Profile -> Load Heap Dump menu items.
13.3 Load Generator SETUP: Open Web Application. Resolve missing server problem if arise selecting GlassFish. Use Tools > Plugins to download and install JMeter, Load Generator and Profiler Bridge modules. Create new Load Testing Scripts > JMeter Plan in test folder of Web Application.
1. Open QueueTest.java. Move cursor to the first line of processRequest() method. Use shift-F9 to insert Load Generator profiling point. Customize Profiling Point dialog is shown.
2. Use browse button to enter path for Script. Confirm the dialog.
3. Invoke Profile -> Profile Main Project... action. Select Analyze Performance task. Confirm dialog by Run button.
4. When test button is shown in browser press it two times. Verify that Profiling Points window shows two hits for Load Generator Profiling Point. //Doesn't work currently
5. Open Services > Load Generators > JMeter and verify that there are two nodes for JMeter Plan invocations. //Doesn't work currently
13.4 Profiling Points Window 1. Window -> Profiling -> Profiling Points
2. Invoke Context menu -> Edit.
3. Invoke Context menu -> Delete
4. Invoke Context menu -> Enable/Disable


Test suite 14: JMeter

Setup: Go to Tools -> Plugins, Available Plugins tab. Find Testing Tools category, JMeter Kit and install it into IDE. Open WebApplication project from Web_ProfilerTestSpecification.zip.

# Test Case Description Expected Results
14.1 Integration 1. Go to Services view (Window -> Services) and check Load Generators node - there should be sub node JMeter.
2. Set WebApplication project as main project. Invoke File -> New File, category Load Testing Scripts, JMeter Plan. Go Next and Finish the wizard.
New jmeter script should be created in test folder of the project.
14.2 Run TBD

Test suite 15: SelfSamplingTool

Setup: Have memory toolbar open.

# Test Case Description Expected Results
14.1 Create snapshot 1. Click on the "Profile the IDE" shortcut on the memory toolbar to start profiling. Do some actions in the ide e.g. open project. Click on the ide again to stop the profiling. IDE snapshot should be opened with graphical timeline present.
14.2 Timeline 1. Hover mouse over the timeline and check zooming via mouse wheel.
2. Check tooltips on both UIGestures and stack depth graphs.
3. Select some part of timeline via mouse.
4. Click on "+", "-" and "=" buttons.
5. Click into the graph.
1. Zoom is functioning.
2. Tooltips are present and change depending on where you hover.
3. The call tree is shown only for the selected area.
4. Buttons increase, decrease and reset graph height.
5.Stack traces are shown.
14.3 Edit TBD
14.4 Run TBD




Not logged in. Log in, Register

By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo