Fitness61FAQs

Performance FAQs

People often (will) ask how we managed so radically improve release 6.1 from the performance point of view. Here are the answers:

What are the main performance improvements in NetBeans 6.1?

Startup experience

• Cold startup of full IDE faster by 20% (empty IDE)
• Cold startup of full IDE with a project and files opened faster by 40%

• Projects open lazily after startup, the last used and user selected projects open first
• Editor initialization and opening does not block UI (AWT event queue)

• Memory consumption after startup down by 37% (87MB in 6.1 vs 140MB in 6.0)
• After startup initialization reduced by 50% (about 15s in 6.1 vs 30s in 6.0) (*)
• Scanning java project classpath after startup faster by 40%

I/O related

• Faster file deletion - about 50%
• Expand a folder with 100 java files faster by 40%, on network by 50%
• 1st code completion in java editor opening time reduced by 70%

More details in I/O improvements.

All numbers taken as average from relative improvements measured repeatedly across more platforms, rounded conservatively (e.g. 47% to 40% gain).

(*) Not counting in the up-to-date sources scanning

You claim to understand the real performance problems more in 6.1, how is that?

Well, we decided to use advanced logging capabilities available in the NetBeans IDE to gather and report performance data and especially performance problems. For example we are able to detect that some code blocks AWT for extremely long amount of time and get snapshots of the state of the system. Using regular channels we are able to gather this data and process them.

This way we can really understand real performance problems of our users and fix problems that are most painful for them.

Why is the startup so slow?

Actually it depends on which startup you are referring to. Internally we recognize three types of it:

• Warm - second and subseqent startups of the same IDE, all files needed by the IDE, JDK were already touched once and potentially the operating system already contains their pieces in its caches - this is a kind of startup that users see only in rare situations, like if the IDE crashes, potentially just after downloading and installing it.
• cold - startup right after the system boot up - no IDE nor JDK files in caches, operating system has to really access the disk and read data from it. This is what users would see when they install the IDE, shutdown computer and start the IDE next day.
• real - is our codename for the startup we really see on our computers. After using the IDE for few days or weeks, shutdown the computer while having projects open, go home, return back, start and try to work. Not really optimized in NetBeans 6.0

Now to the question, why is it so slow? There is many aspects to this, however the biggest contributors in the real case include I/O operations:

• verify that caches are correct

• one mark per cluster, needs cooperation with autoupdate and installer
• check of relevant files in userdir (config/Modules/** & modules/**)

• open JAR files (there are many!) on demand and load classes from them (usually less than 10 classes from each JAR, over 100 files have less than seven loads!)
• initialize data not really stored in caches

• state of module system
• menus, shortcuts
• window system

• load all modules, most of them may not even be necessary for the work
• wait for project infrastructure to load all projects (most of them not needed, as people want usually work with just one)
• wait for java infrastructure to verify its caches

• proposal for speedup

Only after this set of operations are users of NetBeans 6.0 able to start to reasonably work with their opened files in their IDE.

Goal for NetBeans 6.1 was to shorten the time needed to reasonably work with an editor (show a code completion) in real startup scenario by 50%.

How did you managed to shorten the cold startup of 'Big IDE'?

• A bit of reduction of work done during startup, plus making the IDE usable sooner (don't block till the projects are fully opened and rescanned).

• Speedup due to cheaper cache checking - e.g. less stat on small files
• Speedup due to linear StartupCache - a cache of supposedly all resources needed during startp

Did you concentrate only on the initial startup or did you also fix the startup during everyday use?

• Showing main window first, and only then loading projects IZ 108120

• Hilarious side effect: People have more time to read welcome screen
• Show all projects, but initialize them lazily

• Use expert heuristics to guess what users want

• Initialize the project user wants to work on as first

• The one that user expands
• The one that has opened file in editor

• Provide immediate results in Goto Type dialog

• Search through not validated database of data, optimistically ignore errors
• Merge the dialog with Goto File, which is able to produce results without waiting for end of scanning

Your press release says that you are now set up for future growth. What exactly did you do?

• We have made our existing regression infrastructure measurements more accurate

• Making first towards regression framework

• Deliver at least one yes/no test for the MercurialBuildInfrastructurePlan
• How will we measure progress of our new improvements before they are integrated?
• Write tests to measure memory leaks
• Write tests comparing various configs (platform, basic IDE, full IDE)

• startup time
• responsiveness
• memory leaks
• number of classes loaded into memory

I have my sources and/or userdir on network drive. Everything is incredibly slow. Did you improved anything that could speed things up?

We especially concentrated on the case where part of the data users of the IDE work with is on network drive. We did a bunch of testing, and can verify that 6.1 version is much better. Also we concentrated on optimizing and minimizing disk operations.

Are there any plans for future releases in the area of regression framework?

Read our testing infrastructure todo.

Did you found some scalability problems? E.g. places that work acceptably in small IDE or platform and were broken in larger configurations?

• Java files switching in editor differs in specific configurations (tested in platform, IDE with java support and full IDE).
• Performance bugs reported in java/editor category

Did you improve something for Web developers?

• Code completion is improved (it has been very I/O intensive and not properly cached sometimes) - few comments from IZ 123370.

A user comment: I repeated the testing procedure with the new module and CC works like a charm! ... For the test project there was no delays at all, and for the normal one - that includes a lot of libraries, CC was slow only the first time invoked. ... One question: can I use this module with NetBeans 6.0? It would improve my daily work a lot.
A user comment: I too would *really* like to see this as a regular Update Center update for NB 6.0.

I/O measurement
Configuration: Full IDE, JDK 6u4, GlassFish v2u1, Linux 2.6.24
Project: New VisualWeb project (Java EE 5), default page Page1 (empty)

The numbers in the following table are calls on java.io.File.lastModified() done from JSP Parser module.

(1) "Please wait" has been displayed

• We have improved JSP/HTML/JS/CSS editor. Look at the table below (all the times are not exact, just estimates).

]] 
|  4 
|  0 
|  0 
|  2
|- 
|- 
| colspan=5| '''JS''' - jquery-1_Fitness61FAQs.2.1.pack.js (11 lines, last line 26890 chars, 26,5 kB)
|- 
|  CC [[

]] 
|  1 
|  0 
|  0 
|  0
|- 
|  scroll 
|  OK 
|  OK 
|  '''KO''' (1) 
|  OK
|- 
|  CC [[[endOfLongLine,Inside"split"Word | [end of long line, inside "split" word]] 
|  N/A 
|  0 
|  N/A 
|  0
|- 
|- 
| colspan=5| '''CSS''' - mediawiki_Fitness61FAQs.css (1639 lines, 31,3 kB)
|- 
|  open 
|  11 
|  1 
|  3 (2) 
|  1
|- 
|  CC 
|  2 (3)(4) 
|  0 
|  0 
|  0
|- 
|- 
| colspan=5| '''HTML''' - handbook-x86_Fitness61FAQs.html (9757 lines, 438,2 kB)
|- 
|  open 
|  3 
|  2 (17) 
|  10 
|  5
|- 
|  ''enter'' at the beginning 
|  1 
|  1 
|  0 
|  3
|- 
|  1st save 
|  1 
|  0 
|  '''35''' (5) 
|  3
|- 
|  add <script> next to <head> 
|  '''N/A''' (5)(6) 
|  1 
|  0 (4) 
|  4 (7)
|- 

|  CC [[

]] 
|  5 (8) 
|  0 (8) 
|  0 
|  '''4''' (9)
|- 

|  CC [[

]] 
|  5 
|  0 
|  0 
|  '''4''' (9)
|- 
|  CC [[[anyAttribute | [any attribute]] 
|  4 
|  4 
|  0 
|  6
|- 
|  undo to the default state 
|  OK 
|  OK 
|  '''KO''' (10)(5) 
|  '''KO''' (10)(5)
|- 
|  close 
|  2 
|  0 
|  0 
|  0
|- 
|- 
| colspan=5| '''JSP''' (mixed) - editcomposedtestrun_Fitness61FAQs.jsp, 316 lines, 24,6 kB (11)
|- 
|  open 
|  2 
|  2 
|  5 
|  4
|- 
|  warnings/errors 
|  '''KO''' (12) 
|  '''KO''' (12) 
|  OK (13) 
|  OK (13)
|- 

|  Java CC [[

]] 
|  1 (14) 
|  0 (18) 
|  0 (15) 
|  0
|- 

|  Java CC [[

]] 
|  0 (9)(16) 
|  0 (18) 
|  0 
|  1
|- 

|  JS CC [[

]] 
|  1 
|  0 
|  0 
|  0
|- 
|  CSS CC 
|  2 (3) 
|  0 
|  0 
|  0
|}


(1) '''missing characters'''<br>
(2) no errors, warnings at all<br>
(3) just element names, no values<br>
(4) from that place, no color highlighting (the rest of file is black)<br>
(5) UI freeze<br>
(6) '''IDE has to be killed'''<br>
(7) continuous parsing, active part of file (the screen) has to priority<br>
(8) incorrect suggestions (<a> etc.)<br>
(9) no suggestions<br>
(10) really slow<br>
(11) some missing imports<br>
(12) no info about missing imports (just one in the status bar), just few HTML warnings (deprecated attributes)<br>
(13) all the missing imports are properly marked as errors<br>
(14) because of missing imports (I guess), CC does not work at all (e.g. [[[Strin\ | [Strin\]] and no suggestions)<br>
(15) although strange behaviour ("java.*", import of java.util.HashMap creates new <%page import)<br>
(16) mark occurences works<br>
(17) editing is not possible for another 30 sec because of navigator probably<br>
(18) few basic CC items, but incorrect ("public", "abstract" etc.)<br>
====Did you do anything with Memory consumption?====
The memory consumption is wide topic, which includes different areas. Once of the most important is to prevent 
memory leaks. Those are usually little bugs, however with huge effect, especially if the system is used for long time - e.g. days or weeks. Here is what we did for 6.1:

 We fixed every leak we found
 We proactively searched for memory leaks in common areas - use timers/counters aka Runtime Watches to watch:

 Leaking project types [http://www.netbeans.org/issues/show_bug.cgi?id=121855 IZ 121855]
 Leaking dialogs
 Leaking components
 Leaking actions
 DOM trees, AST, Lexer structures, Jasper instances, Taglibrary instances


 We wrote regression tests to prevent such memory leaks from being reintroduced

Another issue is to lower the number of needed classes, especially for functionality that is not currently in use. This is longer term effort, but parts of it started even in the short time dedicated to 6.1 release:

 The [[FeatureOnDemand | FeatureOnDemand]] proposal takes a high level approach and allows dramatic decrease in the amount of needed memory in the "Big IDE" configuration
 We compared the [http://jupiter.czech.sun.com/wiki/view/PerformanceQE/LoadedClasses loaded classes] in various configurations

 we were able to lower their numbers, thus decrease the need for permanent memory
 we wrote tests to verify certain classes are not loaded at all (like java.security.cert.*) 


 We built upon the UI proxy concept to remove the need to initialize classes for objects that are not used, or are just partially visible (icon, display name)

 this includes nodes in Services tab
 topcomponents shown, but with hidden content




----
See also the [[Fitness61PressRelease| Press Release]].
{{attachments|

 [[Media:jquery-1_Fitness61FAQs.2.1.js]]
 [[Media:jquery-1_Fitness61FAQs.2.1.pack.js]]
 [[Media:handbook-x86_Fitness61FAQs.html]]
 [[Media:mediawiki_Fitness61FAQs.css]]
 [[Media:editcomposedtestrun_Fitness61FAQs.jsp]]}}

• This page was last modified on 15 February 2010, at 18:03.
• This page has been accessed 3,456 times.