TS 74 Profiler

Profiler Test Specification

Author: Emanuel Hucka, Alexander Kouznetsov, Matus Dekanek, Petr Cyhelsky, Jiri Skrivanek
Version: NetBeans 7.4
Last update: 19-th July 2013
Bugzilla: profiler/Base
Introduction: This test specification is focused on testing profiler in NetBeans IDE. Sanity test specification contains selected parts of this document. Previous version is ProfilerTestSpecification73.


Contents


Test suite: Installation

  1. Installation
    1. Install current version of NetBeans IDE with Java EE, GlassFish and Tomcat servers. Use a folder containing spaces to verify fixes of SPACE_IN_PATH issues.
    2. Start NetBeans IDE and check Profiler main menu is available
    3. Check Tools -> Options dialog contains Profiler tab in Java section.
    4. Invoke Profiler -> Advanced Commands -> Run Profiler Calibration
    5. Select default Java platform.
    6. Calibration should be finished by information dialog
    7. Click Show Details and see measured values
    8. Click OK


Test suite: Java SE Project

  1. Monitoring
    1. Open new project wizard and create sample project Samples -> Java -> Anagram Game
    2. Press Profile Project in main window toolbar
    3. Select Monitor in Profile AnagramGame dialog, check Enable Threads Monitoring and Enable lock contention monitoring option and press Run button.
    4. Check if Threads and VM Telemetry Overview windows are opened. Check if both windows are scrolling automatically their contents.
    5. There should not be any thread in threads window with 'Profiler' in its name.
    6. Select a thread in Threads view and invoke Thread details action in its popup menu.
    7. Check if Details tab in Threads window is selected and if it contains thread's name, 'pie chart' with statistics and line graph with colored thread's states.
    8. Select one thread's state section in the line graph. Check if Details sub tab with textual states information is opened and appropriate state is selected.
    9. Invoke VM Telemetry button in Profiler Control Panel.
    10. Press Zoom Out button (-) until whole Memory (Heap) graph panel is visible. Its horizontal scroll bar should be disabled but visible to prevent graph bumping on resize. Check if the zoomed out graph is alive.
    11. Switch to Memory (GC) graph panel and press Scale to Fit button. Whole graph should be scaled into graphs' boundaries. Wait a moment - graph should be alive and it should be shrunk to the panel boundaries - horizontal scrollbar should be disabled. The Scale to Fit button should be replaced by Fixed Scale button.
    12. Switch to Threads / Loaded Classes graph panel and check it is live
    13. Open Threads window. Go To Profiled Application and close it.
    14. Go back to profiler. There should be shown Information dialog 'The profiled application has finished ...'.
    15. Confirm the dialog and check the Threads window. There should not be any thread with 'Profiler' in its name or any new thread created during finishing of application.
    16. Profiled application should be stopped. All graphs should be stopped too.
    17. Invoke Save Current View to Image button in any graph panel. There should be opened a Select File or Directory with file system structure, preview of the view and PNG in Files of Type. Write a filename and confirm the dialog. Snapshot of selected view should be stored on disk as png file.
    18. Switch to VM Telemetry and click Export to.... Save in all possible formats and check that the created files are in correct format.
    19. Close Profiler panel. Invoke Window -> Profiling -> Profiler and Profiler panel should be opened again.
  2. Analyze Performance
    1. Invoke Profile -> Profile Project (AnagramGame) from main menu.
    2. Select CPU, Analyze Performance task, set Advanced (instrumented) and Filter: Profile Only Project Classes. Press Run button.
    3. Wait until the application is started. Open Live Results window.
    4. There should be a table with measured methods in the Live Results window.
    5. Click DrillDown button (top-right corner pie-chart button) to open DrillDown window. It should be bound to the right side of live results panel.
    6. Invoke Profile -> Advanced Commands -> View Command-line Arguments. JVM and Main class Command-line Arguments window should be opened and contain corresponding information. Close Command-line Arguments window.
    7. Invoke Stop (Terminate) the Profiled Application action.
  3. Analyze Performance - Live Results
    1. Invoke Profile -> Profile Project... action. Select Analyze Performance task, set Advanced (instrumented) and Filter: Profile All Classes. Press Run button. Wait until the application is started. Open Live Results window. (Check: Then close and invoke Window -> Profiling -> Live Results)
    2. Go to Method Name Filter_ (bottom of live results panel) and select Contains filter type. Write com.toy and press Enter. Live results table should contain only com.toy.* methods.
    3. Uncheck Update Results Automatically toggle button in Live Results tool bar and write a word in the application and click Guess. The results table should not change. Let it several seconds to ensure that table doesn't change.
    4. Press Update Results Now button - results table should change then.
    5. Go to Method Name Filter and press Clear filter button in the right - table should contain all methods again. Try to select e.g. Regular expression filter type - write .*Map.* to the text field and press Apply filter button - there should be shown mostly methods from java.util.*.
    6. Press Reset Collected Results Buffer button in Live Results toolbar - results table should be empty, there should be gray message - No profiling results available yet. Toggle Update Results Automatically button. Click New Word button in the profiled application - results table should be filled again.
    7. Invoke Save Current View to Image button in Live Results toolbar. There should be opened a Select File or Directory with file system structure, preview of the view and PNG in Files of Type. Write a filename and confirm the dialog. Snapshot of selected view should be stored on disk as png file.
    8. Invoke Take Snapshot in Live Results toolbar. After invoking of the action a new CPU snapshot should be opened. Switch back to Live Results window.
    9. Check sorting of results table by each column - press the column title bar.
    10. Sort table by Self Time [%]. Press Show or hide results columns button in the right-top corner, uncheck Self Time, Total Time, Invocations, Filter items. There should remain only two columns in Live Results table sorted by Self time [%]. Press the button again and enable all items in the menu. All columns should be shown as before.
    11. Select a method in results table and invoke its popup menu. Press Go To Source menu item. There should be opened Java editor with source of the class or shown status message "Can not resolve class..."
    12. Move mouse over a method with a long name (not fully displayed in results table). A tooltip should be shown in place of the method's name containing full name of the method (not available on Mac).
    13. Click Export to... and check it is saved correctly in all formats.
    14. Call Profile -> Stop Profiling Session from main menu.
  4. Analyze Performance - DrillDown
    1. Invoke Profile -> Rerun Profiling Session. Wait until the application is started. Open Live Results window. Make some actions in started application.
    2. Click DrillDown button. DrillDown window should be opened and bound to the right side of Live Results panel. The DrillDown window should contain Scope part with pie chart and its legend where legend and pie chart items are selectable. After selection of an item Live Results should be filtered to focus only on the selected category of methods.
    3. Select UI category in the pie chart, then AWT/Swing. Select e.g. java.awt.Component.getParent() method. Method categories part of DrillDown window should contain "Listeners" and "Painters" bars.
    4. Try to resize drilldown window and check if it is repainted correctly. Open Anagrams.java form, switch between Live Results and editor and check if DrillDown window is shown only with Live Results window. Try to switch off and on the Drilldown window by DrillDown button.
    5. Press Shift+F2 to stop profiling session.
  5. Analyze Performance - Snapshots
    1. Go to Profiler tab and invoke Rerun Last Profiling Session
    2. Check Take Snapshot button - it should be enabled during starting of the application. Press the button immediately after it is enabled.
    3. Wait until the application is fully started and close it.
    4. There should appear "Application Finished" dialog asking if you want to take a snapshot of the finished application. Click Yes button.
    5. Check if both taken snapshots contain measured data. The second snapshot should contain more measured methods than the first.
    6. Go to the first snapshot and try to close it. It should ask if you want to save the snapshot. Press No button.
    7. In the remaining snapshot window invoke Save Snapshot to Project from toolbar, close it and open it again from Saved Snapshots list in Profiler panel.
    8. Press Ctrl+F to open find dialog, write e.g. "show" into it and confirm. Try to jump through all occurrences of the word by pressing F3.
    9. Try to change View from Methods to Classes or Packages - the results tree should be changed by appropriate way. Return back to Methods view.
    10. Check sorting of the results tables by each column - press the column title bar.
    11. Invoke Save Current View to Image button in the snapshot's toolbar. There should be opened a Select File or Directory with file system structure, preview of the view and 'PNG' in Files of Type combo. Write a filename and confirm the dialog. Snapshot of selected view should be stored on disk as png file.
    12. Go to e.g. "java.awt.Component.show(boolean)", invoke popup menu on it and press Show Back Traces action. It should switch to "Back Traces for: show" tab.
    13. Select Combined tab of the snapshot window, try to select methods in both parts of split panel and check if there is always selected counterpart method in the other panel.
    14. Go to Hot Spots tab, find "com.toy.anagrams.ui.Anagrams.main" and double click on the method. It should jump into Java editor with Anagrams.java to the selected method's code.
    15. Browse Info tab panel of the snapshot. Try editing user comments, see if they persist.
    16. Close snapshot.
    17. Select the snapshot in Saved Snapshots list of Profiler panel, rename the snapshot using the Rename button, press Save As... button, choose a path and file name and save the snapshot there.
    18. Delete the snapshot by Delete button.
    19. Invoke Profile -> Load Snapshot... action from main menu, select the previous file and confirm file dialog. The snapshot should be loaded.
  6. Analyze Performance - Root Methods
    1. Right-click AnagramGame project node and choose Profile. Select CPU -> Analyze Performance, set Advanced (instrumented) and Filter: Profile All Classes. Press Run button. Wait until the application is started. Open Live Results window.
    2. Click Guess button in the application. Go to live results table, find the "com.toy.anagrams.ui.Anagrams.guessedWordActionPerformed(java.awt.event.ActionEvent)" method, invoke its popup menu and select Add to Instrumentation Roots.... In the opened "Select Settings Configuration" dialog select Analyze Performance item and click OK button.
    3. Invoke Profile -> Stop action and wait until the profiler is finished. Invoke Profile -> Profile Project... action.
    4. Choose Advanced (instrumented). Next to the radio button should be text "custom profiling roots" and "edit..." link.
    5. Click edit... link. Check if there is "com.toy.anagrams.ui.Anagrams.guessedWordActionPerformed(java.awt.event.ActionEvent)" method selected in "Edit Profiling Roots" dialog.
    6. Check check box for "nextTrialActionPerformed(java.awt.event.ActionEvent)" method and click OK button.
    7. Click Run button. Wait until the application is started and open Live Results window.
    8. Look at "Basic Telemetry" part of Profiler panel and remember number of instrumented methods. Press Guess button in the profiled application and watch the Live Results and Basic Telemetry. The number of instrumented methods should increase. Press New Word button in the application - another methods should be measured/instrumented
    9. Invoke Reset Collected Results. Live Results window should be emptied. Switch to the profiled application window - no methods should be shown in Live Results window.
    10. Invoke File -> About action in the profiled application - live results should not be changed.
    11. Invoke Profile -> Advanced Commands -> Display Internal Statistics. "Internal Statistics of Instrumentation and Hotswapping Operations" window should be opened and contain corresponding information. Push Close button.
    12. Open StaticWordLibrary.java, right-click getWord method and choose Profiling -> Add As Profiling Root Method.... Click OK button.
    13. Right-click again getWord method and choose Profiling -> Select Profiling Root Methods.... Click OK button to select configuration and check that getWord method is selected. Click Cancel.
    14. Click Modify Profiling Session... and then edit... link.
    15. Click Advanced... button and remove all methods. Click OK in both dialogs and check "default profiling roots" label is displayed in remaining dialog.
    16. Click Run and when updated stop profiling.
  7. Analyze Performance - Instrumentation Filters
    1. Invoke Profile -> Profile Project action. Select CPU -> Analyze Performance task, set Advanced (Instrumented) and Filter: Profile Only Project Classes.
    2. Click Show filter value link - there should be opened dialog with two packages for AnagramGame project. Press To Quick Filter button in the dialog.
    3. "Set Quick Filter" dialog should be opened with preset Inclusive and both packages in the Filter Value text field. Confirm it by OK button. Filter combo box should be changed to "Quick Filter...".
    4. Press Run button. Wait until the application is started.
    5. Open Live Results window and check that it contains just methods from the project's packages only.
    6. Stop profiling and press Alt+F2 to profile project. Choose CPU -> Analyze Performance -> Advanced (Instrumented) and click customize... link next to it. In the "Edit Profiling Roots" dialog uncheck "com.toy.anagrams.ui" and select only constructor under Anagrams class. Click OK.
    7. Select "Quick filter" and remove "com.toy.anagrams.ui.". Start profiling by Run button. Wait until the profiling is started and open Live Results window.
    8. Invoke New Word action a few times in the profiled application and check live results. There should be no other method from com.toy.anagrams.ui then com.toy.anagrams.ui.Anagrams.<init>().
    9. Stop profiling and invoke Profile Project action again. Click Edit filter sets link. Press Add button in the "Customize Filter Set" dialog, write "Not UI" into the "Filter Set Name" text field and leave Filter Set Type as Exclusive.
    10. Press Edit Global Filters, press New button, write "NonGUI" into the Name column and "java.awt., javax.swing." into the Value column. Click OK button.
    11. In "Customize Filter Sets" dialog select "NonGUI" filter in "Global filters" list (check its check box). Confirm the dialog by OK button. The "Not UI" filter should be selected in Filter combo box. Start profiling by Run button.
    12. Check in "Live Results" window there are not any methods from "java.awt" and "javax.swing" package.
    13. Click Modify Profiling Session... and then edit... link.
    14. Click Advanced... button and remove all methods. Click OK in both dialogs and check "default profiling roots" label is displayed in remaining dialog.
    15. Click Run and when updated stop profiling.
  8. Analyze Memory Usage
    1. Invoke Profile -> Profile Project....
    2. Select Memory button and leave default Quick (sampled) mode.
    3. Click Run button, open "Live Results" and wait until the application is started.
    4. Press Reset Collected Results button - Live Results should get empty and refreshed after couple seconds with new sample.
    5. Stop profiling.
  9. Analyze Memory Usage - Live Results
    1. Invoke Profile -> Profile Project... action. Select Memory -> Analyze Memory -> Advanced (Instrumented) task with unchecked "Record full object lifecycle" and "Record stack trace for allocations".
    2. Click Run.
    3. Open "Live Results" and disable Update Results Automatically.
    4. Click "Guess" and check results haven't changed.
    5. Press Reset Collected Results Buffer button and results should be cleared.
    6. Click "Guess" and then Update Results Now and results should change.
    7. Check Update Results Automatically toggle button to enabled state.
    8. Invoke Save Current View to Image button in Live Results toolbar. There should be opened a Select File or Directory with file system structure, preview of the view and PNG in Files of Type. Write a filename and confirm the dialog. Snapshot of selected view should be stored on disk as png file.
    9. Invoke Take Snapshot in Live Results toolbar
    10. Switch back to "Live Results" and check sorting of results table by each column
    11. Press Show or hide results column button in the right-top corner, uncheck "Bytes Allocated", "Objects Allocated", "Filter" items.
    12. Press the button again and enable all items.
    13. Go to "Class Name Filter" (bottom of live results panel). Write String and press Enter. Live results table should contain only 'Strings'. Press Clear filter - button with red cross on the filter line. Table should contain all classes again.
    14. Select Regular expression filter - write .*Map to filter text field and press Apply filter button - there should be shown all 'Maps'. Press Clear filter button.
    15. Invoke File -> About in tested application. Right-click "com.toy.anagrams.ui.About" and choose Go To Source. Editor should be opened.
    16. Go to live results and try to invoke Go to Source action on "char[]" class. There should be shown an warning message: "Cannot show source for primitive array".
    17. Move mouse over a class with a long name (not fully displayed in results table). A tooltip should be shown in place of the class name containing full name of the class (not available on Mac).
    18. Stop profiling.
  10. Analyze Memory Usage - Snapshots
    1. Invoke Profile -> Profile Project... action. Select Memory -> Analyze Memory -> Advanced (Instrumented).
    2. Select Record Stack Trace for Allocations. Confirm dialog by Run button.
    3. Check Take Snapshot is enabled during starting of the application.
    4. Press Take Snapshot immediately after it is enabled.
    5. Click Export to... and save it.
    6. Close snapshot.
    7. Choose "java.util.HashMap" and invoke Take Snapshot and Show Allocation Stack Traces from its popup menu. A new snapshot should be opened with selected "Allocation Stack Traces" tab panel.
    8. Close snapshot without saving it.
    9. Close the profiled application and check that appears "Application Finished" dialog asking if you want to take snapshot of finished application. Click Yes.
    10. Click Save Snapshot to Project, close it and open it again from "Saved Snapshots" list.
    11. Press Ctrl+F, write "String" and click "Find". Find next occurrence (F3).
    12. Right-click "java.lang.String" class and choose Show Allocation Stack Traces.
    13. Press Show or hide results column button in the right-top corner, uncheck Bytes Allocated, Objects Allocated, Filter items. Press the button again and enable all items.
    14. Invoke Save Current View to Image button in the snapshot's toolbar. Write a file name and confirm the dialog.
    15. Switch to Info tab of the snapshot and check its content.
    16. Select the snapshot in "Saved Snapshots", press Save As... button, choose a path and file name and save it.
    17. Delete the snapshot by Delete button.
    18. Invoke main menu Profile -> Load Snapshot, open previously stored snapshot and then close it.
    19. In main menu choose Profile -> Compare Snapshots and browse for two previously saved snapshots.
    20. Check plus and minus values in opened "Allocations Comparison" panel.
    21. Close panel.
  11. Analyze Memory Usage - Object Liveness
    1. Invoke Profile -> Profile Project... action. Select Memory -> Analyze Memory -> Advanced (Instrumented).
    2. Select also "Record full object lifecycle" and click "Run".
    3. Go to Live Results. Sort results by "Class Name" ascending. Scroll to the first class from package "sun.awt", right-click and choose Stop Profiling Classes bellow this line action. Right-click again and choose Stop Profiling this Class.
    4. Check stopped classes dismiss from view.
  12. Analyze Memory Usage - Class History
    1. Right-click "char[]" object and choose Log Class History.
    2. Press Scale to Fit button.
    3. Invoke some actions in profiled application. Try to invoke Run GC action from Profiler Control Panel several times.
    4. Invoke Save Current View to Image button in Live Results toolbar. Select file name and save it.
    5. Go back to "Live Results" tab, select "int[]" class and invoke Log Class History on it. The History tab name should be changed to "History of int[]", values in the graph should be changed too.
    6. Stop profiling.
  13. Unit Tests
    1. Go to Projects view, right-click "AnagramGame -> Test Packages -> com.toy.anagrams.lib -> WordLibraryTest.java" and choose Profile File.
    2. Select CPU -> Analyze Performance -> Advanced (instrumented), Profile All Classes and click Run.
    3. Wait until the profiling is finished and confirm Application Finished dialog by Yes button.
    4. Go to Projects view, right-click AnagramGame -> Source Packages -> com.toy.anagrams.lib -> WordLibrary.java file and choose Profile Test File.
    5. Select Memory -> Analyze Memory -> Advanced (instruments) and select all check boxes. Click Run button.
    6. Wait until the profiling is finished and confirm "Application Finished dialog" by Yes button.
    7. Close all snapshots without saving.
  14. OutOfMemory Protection
    1. Open Anagrams.java class and modify code the following way
      ArrayList data = new ArrayList(24);
      private void guessedWordActionPerformed(java.awt.event.ActionEvent evt) {
      data.add(new int [500000]);
    2. Open properties of the AnagramGame project, select Run category and insert '-Xmx10m' into "VM Options" line.
    3. Invoke Profile -> Profile Project main menu. Select Monitor -> Monitor Application and click Run button.
    4. Open terminal window of IDE or messages.log file from userdir and check if there is a message 'Profiler.OutOfMemoryProtection: Enabled' at start of current profiling.
    5. Open VM Telemetry window and notice horizontal line at level of inserted Max Heap value of JVM.
    6. Press Guess button in profiled application two times. Used Heap graph in VM Telemetry should be increased about 3.8MB.
    7. Press Guess button again a few times until a Question dialog - "Profiled application crashed and generated heap dump." appears. Confirm it by Yes button.
    8. A new heap dump should be opened. Check Output window of profiled application - above the OOM exception there should be a log "Dumping heap to ..." describing file where the heap dump is created.
    9. Stop profiling and clean all changes of AnagramGame code, remove -Xmx from the project's properties.

Test suite: Applet

  1. Profile Applet - AppletViewer
    1. Unpack and open AppletTest project from TS_Profiler_AppletTest.zip archive.
    2. Right-click "applet/TestApplet.java" and choose Profile File.
    3. Select CPU -> Analyze Performance -> Advanced -> Profile only project classes and press Run button.
    4. Wait until the AppletViewer is opened with running application. Press Increase button few times.
    5. Open "Live Results" and check there is applet.TestApplet.make() at the first position. Self Time should be about 500ms * number of invocations.
    6. Open DrillDown panel and select UI category.
    7. Right-click addBActionPerformed method and choose Add to Instrumentation Roots. In the opened dialog choose Analyze Performance and confirm the dialog.
    8. Click Modify Profiling Session, click edit... and check added root.
    9. Click Run and wait until instrumentation is modified.
    10. Click Decrease button. Live results should not be changed. Click Increase and live results should change.
    11. Invoke Modify Profiling Session and select Memory -> Analyze Memory -> Advanced with all check boxes selected.
    12. Click Run and then Increase button.
    13. Check memory results appear in live results panel.
    14. Invoke Reset Collected Results Buffer action and make some action in AppletViewer again. Live Results should be empty and after the actions in viewer filled again.
    15. Close the AppletViewer window and confirm Application Finished dialog.
  2. Profile Applet - Browser
    1. Disable security restrictions on your system. For example on windows open Java from Control Panel, switch to Security tab and decrease security level to Medium. Click OK and restart your browser. Also keep in mind that if applet runs in 32-bit Java in browser, you have to run IDE with 32-bit JDK too.
    2. Switch to Files view and go to "AppletTest -> build -> TestApplet.html". Invoke popup menu and View action on the html file. A browser should be opened with TestApplet started.
    3. Invoke main menu Profile -> Attach Profiler
    4. In "Attach Profiler" dialog select AppletTest project in top most combo box "Attach to:" and check profiling roots are set from previous test case (just one method).
    5. Choose CPU -> Analyze Performance -> Advanced and click Attach.
    6. In "Attach Settings" keep default selection "Target Local", "Connection Dynamic" and click OK.
    7. In "Select Process To Attach" choose applet launcher process and click OK
    8. Click Increase button and wait until "Live results" window is filled by profiling results.
    9. Invoke Take Snapshot and check taken CPU snapshot.
    10. Press Threads button and enable threads monitoring.
    11. Press VM Telemetry button and check Memory (Heap) graph, invoke Run GC button several times and check behaviour of memory graph.
    12. Press Detach from the Profiled Application and click No button. Check applet stays running in the browser and profiler is finished.

Test suite: Free Form

  1. Integration
    1. Unpack and open test project from TS_Profiler_freeform.zip archive.
    2. Right-click project node and choose "Run". Check Anagram game is opened and then close it.
    3. Select opened project in Project view and invoke Profile -> Profile Project from main menu.
    4. Click Generate in confirmation dialog.
    5. Right-click Profile on project root node.
    6. Select Monitor -> Monitor Application and press Run button.
    7. Anagram game should start and threads should be monitored.
    8. Stop profiling.
    9. Right-click "src -> com.toy.anagrams.ui -> Anagrams.java" and choose Profile File.
    10. Click Generate to confirm adding of build target "Profile File".
    11. Right-click "src -> com.toy.anagrams.ui -> Anagrams.java" and choose Profile File again.
    12. Select Monitor -> Monitor Application and press Run button.
    13. Anagram game should start and threads should be monitored.
    14. Stop profiling.
  2. Profiling
    1. Invoke Profile -> Profile Project from main menu.
    2. Select CPU -> Analyze Performance -> Advanced -> Profile Only Project Classes and press Run button.
    3. Open Live Results window and take snapshot.
    4. Invoke Modify Profiling.
    5. Select Memory -> Analyze Memory -> Advanced and check all check boxes. Press Run button.
    6. Go to the application and perform some actions. Return to Ide, open Live Results window and Take Snapshot.
    7. Stop profiling.


Test suite: NB Module

  1. Analyze Performance
    1. Create sample project File -> New Project -> Samples -> NetBeans Modules -> Paint Application. Try to start project by Run action and check if all works.
    2. Invoke main menu Profile -> Profile Project... action.
    3. Select CPU and click Create Custom... link.
    4. In New Custom Configuration (CPU) dialog change its name to "testconfig" and click OK.
    5. Select Advanced (instruments) and click Advanced Settings link.
    6. Select Exact call tree, sampled timing.
    7. Execute profiling by Run button and watch Take Snapshot button. Immediately after the button is enabled, press it. It should take non empty CPU snapshot - close it without saving. Wait until whole application is started.
    8. In the started application open New Canvas and paint something there. Close the application and confirm "Application finished dialog" in profiler by Yes button.
    9. In the taken snapshot go to Hot Spots, find method "org.netbeans.paint.PaintCanvas.mouseDragged", invoke its popup menu and select Add to Instrumentation Roots. In "Select Settings Configuration" dialog select "testconfig" configuration and confirm it by OK button.
    10. Right-click "Paint Application" project node and choose Profile. Leave 'testconfig' custom profiling selected, click Edit link.
    11. In "Edit Profiling Roots" deselect the method mouseDragged and select mouseClicked() method. Click OK.
    12. Click Run button. Wait until the application is started.
    13. Open "Live Results" window - it should be empty. Go to the profiled application, create New Canvas and try to draw something. Method mouseClicked should be displayed in Live Results window.
    14. Right-click org.netbeans.paint.PaintCanvas.mouseClicked(java.awt.event.MouseEvent) method and choose Go To Source. There should be opened editor with the selected class and its cursor should be moved to the selected method.
    15. Invoke Take Snapshot action.
    16. Invoke Profile -> Stop Profiling Session.
  2. Monitor and lock contention
    1. Invoke Profile -> Profile Project (Paint Application) from main menu.
    2. Select Monitor -> Monitor Application, select Enable threads monitoring, Enable lock contention monitoring and press Run button.
    3. Wait until the application is started and open New Canvas there and check Threads window if it is working.
    4. Switch to "Lock Contention" window.
    5. Under each thread is set of objects/locks on which synchronization was done. It may signal bottlenecks when single object is locked many times and with different threads.
    6. Try to export this view as image or other file type.
    7. Select Monitors in "View by:" combo box and you should see list of objects.
    8. Expand several nodes and check threads accessing object are listed underneath.
  3. Analyze Memory Usage
    1. Invoke Modify Profiling Session... action, select Memory -> Analyze Memory -> Advanced, select both Record full object lifecycle and Record stack trace for allocations. Click Run to confirm.
    2. Play with application to produce some profiling results.
    3. Open Live Results window, right-click java.awt.event.MouseEvent class and choose Log Class History.
    4. Click Scale to Fit in opened History of java.awt.event.MouseEvent.
    5. Go to the profiled application and draw something.
    6. Check both graphs increased evidently.
    7. Click Run Garbage Collection in Profiled Application and check "Live Objects" and "Live Bytes" decrease to zero.
    8. Stop profiling.


Test suite: Web Project

  1. Monitoring
    1. Open new project wizard and create "Java Web|Web Application" project (Java EE 7, GlassFish Server).
    2. Right-click project node and choose Profile.
    3. Select Monitor -> Monitor Application in Profile WebApplication dialog, select Enable threads monitoring option and press Run button.
    4. Check if Threads and VM Telemetry Overview windows are opened. Check if both windows are scrolling automatically their contents.
    5. There should not be any thread in threads window with Profiler or JFluid in its name.
    6. Right-click a thread in Threads view and choose Thread Details.
    7. Check if Details tab in Threads window is selected and if it contains thread's name, pie chart with statistics and line graph with colored thread's states.
    8. Select one thread's state section in line graph. Check if Details sub tab with textual states information is opened and appropriate state is selected.
    9. Invoke VM Telemetry button in Profiler control panel.
    10. Press Zoom Out button (-) until whole Memory (Heap) graph panel is visible. Its horizontal scroll bar should be disabled but visible to prevent graph bumping on resize.
    11. Switch to "Memory (GC)" graph panel and press Scale to Fit button. Whole graph should be scaled into graphs' boundaries - horizontal scroll bar should be disabled. The "Scale to Fit" button should be replaced by "Fixed Scale" button.
    12. Switch to "Threads/Loaded Classes" graph panel and use mouse wheel to zoom out and in.
    13. Open Threads window. Go To Services view and stop GlassFish.
    14. Information dialog The profiled application has finished ... is opened. Wait a few seconds - Threads window on background should continue in showing new states of threads.
    15. Confirm the dialog and check the Threads window is stopped.
    16. Invoke Save Current View to Image button in any graph panel and try to save it.
  2. Analyze Performance
    1. Create new servlet in package aaa in web project.
    2. Right-click created servlet and choose Profile File.
    3. 2. Select CPU -> Analyze Performance -> Advanced -> Profile Only Project Classes. Press Run button.
    4. Open Live Results window and wait until browser with servlet is opened.
    5. Refresh browser few times and check that number of invocations in live results has changed.
    6. Right-click processRequest method and choose Go To Source. Check editor is opened.
    7. Right-click processRequest method and choose Add To Instrumentation Roots....
    8. Select "Analyze Performance" item and confirm dialog "Select Settings configuration".
    9. Stop profiling.
    10. Right-click created servlet and choose Profile File again.
    11. Click edit... link and check if there is the selected method.
    12. Change filter to "Profile all clases" and click Run.
    13. Wait until browser with servlet is opened and check in live results there is only processRequest method from aaa package and several methods from java.util and java.lang.
  3. Analyze Performance - Snapshots
    1. Press Take Snapshot action in Profiler control panel.
    2. A new snapshot window should be opened. It should contain method processRequest invoked in http-listener thread.
    3. Invoke Save Snapshot to Project, close it and open it again from "Saved Snapshots" list in Profiler panel.
    4. Right-click processRequest method and choose Show Back Traces.
    5. Select Combined tab of the snapshot window and double click on processRequest method in the bottom part of Combined view. It should jump into Java editor to the selected method's code.
    6. Select the snapshot in Saved Snapshots list of Profiler Control Panel, press Save As...' button, choose a path and file name and save the snapshot there.
    7. Delete the snapshot by Delete button. Invoke Profile -> Load Snapshot from main menu, select the previous file and load it.
    8. Stop profiling.
  4. Analyze Memory Usage
    1. Invoke Profile -> Profile Project. Select Memory -> Analyze Memory -> Advanced, select both Record full object lifecycle, Record Stack Trace for Allocations. Confirm dialog by Run button.
    2. Press Live Results button and wait until the application is started.
    3. Press Reset Collected Results button. Live Results window should be empty for a while and then refreshed.
    4. Right-click char[] and choose Stop Profiling this Class. The class should disappear.
    5. Sort classes by names (press header button of results table) and select e.g. the first class its name begins with org. Invoke action Stop Profiling Classes bellow this Line.
    6. Right-click e.g. java.lang.String and choose Take Snapshot and Show Allocation Stack Traces.
    7. Invoke Save Snapshot to Project action. Select the snapshot in Saved Snapshots list, press Save As... button, choose a path and file name and save the snapshot there.
    8. Delete the snapshot by Delete button. Invoke Profile -> Load Snapshot... main menu, select the previous file and open it.
    9. Stop profiling and close all documents in editor.
  5. Supported Servers
    1. Register other supported application servers in Services tab and repeat the following step for all of them.
    2. Right-click server in Services view and choose Start in Profile Mode.
    3. Wait until message "Profiler Agent: Waiting for connection on port 5140" appears in server log and then stop server.
    4. Open new project wizard and create "Java Web|Web Application" project with desired server.
    5. Right-click project node and choose Profile
    6. Select CPU -> Advanced -> Profile only project classes and click Run.
    7. Open "Live Results" and wait until index page is opened in browser.
    8. Check that live results contains some data.
    9. Stop profiling.
    10. Create new servlet.
    11. Right-click servlet and choose Profile File.
    12. Wait until application is redeployed and servlet opened in browser.
    13. Click Take Snapshot and Dump Heap and check opened views.
    14. Open VM Telemetry Overview, VM Telemetry and Threads and check that all values are refreshed continuously.
    15. Stop profiling and check server is stopped as well.


Test suite: Attach Profiler

  1. Local Dynamic
    1. Run AnagramGame project
    2. Invoke Profile -> Attach Profiler... from main menu.
    3. Select "AnagramGame" in top combo box Attach to:.
    4. Select CPU -> Analyze Performance -> Advanced -> Profile only project classes.
    5. If you are attaching profiler for the first time, "Attach Mode" is not yet defined - click define.... Otherwise click change... at the bottom of "Attach Profiler" dialog and select Local and Dynamic type.
    6. Click Attach.
    7. Select Anagrams process and click OK.
    8. Open "Live Results", click something in profiled application and check methods from AnagramGame project are displayed.
    9. Click Detach from the Profiled Application in Profiler panel.
    10. Click Yes in "Detach Profiler" window and check profiler is stopped and profiled application is terminated.
  2. Local Direct
    1. Invoke Profile -> Attach Profiler... from main menu.
    2. Select "AnagramGame" in top combo box Attach to:.
    3. Select CPU -> Analyze Performance -> Advanced -> Profile only project classes.
    4. Click change... at the bottom of "Attach Profiler" dialog and select Local and Direct type.
    5. Check it is possible to change agentpath parameter for JDK 6 vs. 5 and 32 bit vs. 64 bit by clicking Click link.
    6. Click copy to clipboard and then two times OK to confirm "Attach Settings" window.
    7. Click Cancel in "Attach Profiler" window.
    8. Right-click AnagrameGame project node and choose Properties.
    9. Go to Run category, paste content of clipboard to "VM Options" text field and click OK.
    10. Run AnagramGame project. It should print to console "Profiler Agent: Waiting for connection on port 5140".
    11. Choose Attach Profiler from main toolbar and click Attach.
    12. Check that methods from AnagramGame project are displayed in "Live Results".
    13. Invoke Profile -> Detach... from main menu.
    14. Click Yes in "Detach Profiler" window and check profiler is stopped and profiled application is terminated.
    15. Remove agentpath from project properties.
  3. Remote
    1. Invoke Profile -> Attach Profiler... from main menu.
    2. Click change... at the bottom of "Attach Profiler" dialog and select Remote target.
    3. Type localhost in hostname text field and select appropriate OS and JVM.
    4. Click create a Remote profiling pack, save it to some empty folder and click OK in confirmation dialog.
    5. Click OK in "Attach Settings" dialog.
    6. Click Cancel in "Attach Profiler" window.
    7. Unzip remote pack, go to bin folder and from command-line run calibration.
    8. Copy anagrams.jar from AnagramGame/dist folder to remote pack bin folder (run Clean and Build on project if dist does not exist yet).
    9. Set JAVA_HOME property to valid value.
    10. Run profile-16.* -jar anagrams.jar
    11. It should print to console "Profiler Agent: Waiting for connection on port 5140" and wait.
    12. Return to IDE and invoke Profile -> Attach Profiler... from main menu.
    13. Click Attach and check methods from AnagramGame project are displayed in "Live Results".
    14. Invoke Profile -> Detach... from main menu.
    15. Click Yes in "Detach Profiler" window and check profiler is stopped and profiled application is terminated.


Test suite: HeapWalker

  1. Summary
    1. Right-click project node and choose Profile
    2. Select CPU -> Advanced -> Profile only project classes and click Run.
    3. Open "Live Results" and wait until application is started.
    4. Click Dump Heap, choose destination to "Custom Directory" and click No to not open it in HeapWalker.
    5. Click something several times in profiled application.
    6. Click Dump Heap again, choose destination to "Profiled Project" and click Yes to open it in HeapWalker.
    7. Check Summary view contains overall data. Click Show System Properties and Show Threads. Check links in stack traces are valid.
    8. Click link in "Application windows" and return back to Summary panel.
    9. Click Find to search for biggest objects by retained size. Confirm question dialog and when finished click some links to get to Instances view.
    10. Close heap dump and open it again from "Saved Snapshots".
  2. Classes
    1. Switch to Classes page.
    2. Check sorting of table by each column - press the column title bar.
    3. Press Show or hide columns small button in the right-top corner of the table, uncheck Instances, Size, Filter items. There should remain only two columns in the table. Press the button again and enable all items in the menu.
    4. Go to Class Name Filter (bottom line of panel) and select Contains filter type. Write Buffer and press Enter. Classes table should contain only 'Buffers'. Press Clear filter button with red cross on the filter line. Table should contain all classes again. Try to select e.g. Regular expression filter type - write .*Map to filter text field and press Apply filter button - there should be shown all 'Maps'. Press Clear filter button.
    5. Select java.lang.String class and press Static Fields button (top-right corner of window). Static Fields panel should be inserted to the right side of heapdump window. There should be shown all static fields of String class (class, serialVersionUID etc.). There should be shown icons legend in the bottom line of the window (Array Type, Object Type etc.)
    6. Right-click java.lang.String and choose Go to Source. It should open source file of java.lang.String class in editor and move cursor to class definition. (It doesn't work on some JDKs on mac os x).
    7. Click Compare with another heap dump link at the top right of Classes tab. Browse for heap dump previously stored at custom folder.
    8. Review plus and minus values across table and click reset view.
  3. Instances
    1. Right-click java.lang.String and choose Show in Instances View.
    2. Try to close and open again all panels in Instances view. Use 'x' buttons in right-top corner of each panel and buttons with icons on heap dump window toolbar. Try to change size of panels - after closing and opening again the size of panel should be preserved.
    3. Check the legend line bellow the toolbar - there should be name of class and some statistics about it.
    4. Try to switch on and off all available columns in each panel by Show or hide columns button (top-right corner of each panel).
    5. Try to sort instances list table by particular columns - press title of the column.
    6. Find a table cell (mostly in Fields or References panels) which's content is not fully displayed (ends by ...). Move mouse cursor over the cell and check if its full content is shown in appeared tooltip.
    7. Click on the retained size link at the top right - the retained sizes should be computed and displayed in new column.
    8. Sort instances by retained size and select first item. Check there is value partially shown in description panel. Click Show all to see entire value. Click Save to file and save value to disk.
    9. Right-click a node of non-primitive type in Fields or References panel and choose Show Instance. Instances of the class should be displayed in heap dump window and there should be selected the instance in Instances list panel.
    10. Click Navigate back arrow in toolbar.
    11. Right-click a node of non-primitive type in Fields panel and choose Show in Classes View. Classes view should be displayed in heap dump window with the selected class. Return back to Instances.
    12. Right-click a node of non-primitive type in Fields or References panel and choose Go to Source. The editor with source of the class should be opened.
    13. Right-click this field in References view and choose Show Nearest GC Root.
    14. Right-click select GC root and choose Copy Path From Root. Check that the path is copied into clipboard.
  4. OQL console
    1. Go to OQL console tab.
    2. In "Saved queries" select "List java.io.File Instances" and click Open.
    3. Click Execute in "Query Editor".
    4. Click some instance in "Query Results" and then return back from Instances tab.
    5. Try code completion next to java.io. and replace File by FileInputStream.
    6. Click Save and try to modify properties in "Saved Queries".
    7. Delete created query.
    8. Close heap dump panel.
    9. Stop profiling.


Test suite: Profiling Points

  1. Stopwatch and Reset Results
    1. Open "Anagrams.java in editor. Move cursor to the first line of method guessedWordActionPerformed (there is if command).
    2. Invoke popup menu there and press Profiling -> Insert Profiling Point menu item.
    3. Select "Profiling Point Type" Stopwatch.
    4. Click Next and confirm default settings by pressing Finish button.
    5. The stopwatch annotation should be added to the line, line should have different background color.
    6. Invoke popup menu on the annotation, there should be Stopwatch Profiling Point sub menu containing Enabled, Edit, Delete and Go To End Point options. Select Go To End Point. Warning message is shown.
    7. Again open popup and now select Profiling Point > Edit menu item.
    8. Choose Timestamp and duration radio button. Location (stop) controls gets enabled. Set line number to 274 (last line of method guessedWordActionPerformed). Click OK.
    9. Invoke popup menu on the end stopwatch annotation (line 274), select Profiling Point -> Go To Start Point. Cursor should be moved to the start stopwatch line.
    10. Invoke popup menu on the start stopwatch annotation (line 266), select Profiling Point -> Go To End Point. Cursor should be moved to the end stopwatch line.
    11. Move cursor to the first line of method nextTrialActionPerformed. Press Shift-F9. Reset Results profiling point is inserted and Customize Profiling Point dialog is shown. Confirm it by OK button.
    12. Right-click AnagramGame project node and choose Profile.
    13. Select CPU - > Analyze Performance -> Profile Only Project Classes. Ensure that Use defined Profiling Points option is enabled and Show active Profiling Points link opens list with two Profiling Points.
    14. Press Run button and wait until the application is started.
    15. Press its Guess button five times. "Profiling Points" window should show 5 hits for Stopwatch profiling point and no results available for reset results profiling point.
    16. Click report link for Stopwatch profiling point. Report window should be shown, for each hit timestamp and duration should be shown.
    17. Open Live Results. There should be some measured methods. Click New Word button in profiled application. Live Results should show only two methods from now.
    18. Click report link for Reset Results in Profiling Points window. Report window should be shown, for each hit timestamp should be shown.
    19. Stop Profiling.
    20. Start Profiling again having User defined Profiling Points option disabled. Play with application. No results for profiling points should be shown. Stop Profiling.
    21. Right-click Reset Results in Profiling Points window and select Show in Source menu item. Cursor should be moved to first line of nextTrialActionPerformed method.
    22. Rigth-click its annotation and select Profiling Points > Enabled. Reset Results Profiling point should be shown gray in editor and Profiling Points window.
    23. Right-click Stopwatch in Profiling Points window and select Show Start in Source menu item. Cursor should be moved to first line of guessedWordActionPerformed method. Right-click Stopwatch in Profiling Points window and select Show End in Source menu item. Cursor should be moved to last line of guessedWordActionPerformed method.
    24. Rigth-click its annotation and select Profiling Points > Delete. Stopwatch Profiling point should be removed from editor and Profiling Points window.
  2. Take Snapshot
    1. Open Anagrams.java source file in editor. Move cursor to the first line of method nextTrialActionPerformed (wordIdx ...). Invoke popup menu there and press Profiling > Insert Profiling Point menu item.
    2. Select Profiling Point Type Take Snapshot, click Next. Enable Reset Results after taking snapshot. Choose Begin for Line. Click Finish.
    3. Right-click AnagramGame project node and choose Profile.
    4. Select CPU - > Analyze Performance -> Profile Only Project Classes. Ensure that Use defined Profiling Points option is enabled and Show active Profiling Points link opens list with two Profiling Points.
    5. Press Run button and wait until the application is started.
    6. Press its New Word button two times. Close the application. Do not take snapshot.
    7. There should be two new snapshots in "Saved Snapshots" part of Profiler panel. Profiling Points window should show 2 hits, and report for Take Snapshot profiling point should contain links to both snapshots. No results should be available for Reset Results Profiling Point.
    8. In profiling points window remove both profiling points.
  3. Timed- and Triggered- Profiling Points
    1. Open Anagrams.java source in editor. Push Shift key and press F9 key several times until Timed Take Snapshot is chosen. Release Shift key. Customize Profiling Point dialog is shown.
    2. Choose Heap dump option, To directory option and enter some directory. Choose Take every option, ensure that 1 minute is chosen. Press OK.
    3. Press Add Profiling Point button in "Profiling Points" window. Select Triggered Take Snapshot. Click Next.
    4. Select surviving generations in "Take when" list, enter 2 generations, choose Take every time option. Click Finish.
    5. Right-click AnagramGame project node and choose Profile.
    6. Select Memory - > Analyze Memory -> Advanced.
    7. Press Run button and wait until the application is started.
    8. Open VM Telemetry Overview window and click Run Garbage Collector until surviving generations exceeds 2.
    9. Open "Profiling Points" window. There should be one hit for "triggered take snapshot" profiling point and several hits for "timed take snapshot" profiling point.
    10. Open report for Timed Take Snapshot profiling point. There should be several hits with 1 minute interval between them.
    11. Stop profiling.
    12. Verify that all taken snapshots and heapdumps are available and are placed in the specified locations. Try to load heap dumps using Profile -> Load Heap Dump menu items.

Test suite: Profile the IDE

  1. Create snapshot
    1. Click on the Profile the IDE shortcut on the memory toolbar to start profiling.
    2. Do some actions in the IDE e.g. open project.
    3. Click toolbar button again to stop the profiling.
    4. IDE snapshot should be opened with graphical timeline present.
  2. Timeline
    1. Hover mouse over the timeline and check zooming via mouse wheel when Fixed Scale is selected.
    2. Check tooltips on both "UI Actions" and "Stack depth" graphs.
    3. Click on +, - and = buttons to increase, decrease and reset graph height.
    4. Click into the graph and stack traces should be shown.
    5. Click Select all in toolbar to return to call tree.
  3. Probes timeline
    1. Right-click getNextEvent() sub node of AWT-EventQueue thread and choose Select Intervals.
    2. At Probes timeline you should see dashed line where blue parts indicate time when method is available at stack trace.
    3. Click several times at various places at Probes timeline. Marks should be added.
    4. Click Clear marks button at the top of the graph and marks should disappear.
  4. Export
    1. Click Export IDE Snapshot and save it.
    2. Invoke Profile -> Load IDE Snapshot from main menu, browse for previously saved snapshot and check it is successfully opened in IDE.
    3. Select some part of timeline via mouse and check call tree has changed.
    4. Click Export to... in call tree toolbar and save snapshot.
    5. Choose Profile -> Load Snapshot from main menu and load it.
    6. Check that only selected snapshot is opened.
    7. Close all opened snapshots.
Not logged in. Log in, Register

By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo