As usual with every release, when finishing NetBeans 6.9 we start thinking about what performance issues to attack in next releases. This page collects performance topics that are actually most important for our users, and ideas how we could improve the IDE performance in various aspects (UI responsiveness, scalability, etc).
Note: This page is not a plan yet, it's an area for gathering ideas, some may be quite speculative. It's also more a list of problems than solutions or goals. If you have any idea or see an important problem, just throw it here. We'll finalize the content, prioritize and select specific topics as part of next release planning. See also previous PostNB68PerfTopics, PostNB67PerfTopics, PostNB65PerfTopics.
Indexing and parsing infrastructure
A note to scale: The issues described here are visible only with large projects, i.e. projects with thousands of files.
Scanning-related issues are still at the top of performance issues reported by our users. No performance improvements were planned for 6.9, there were some regressions solved and a few partial improvements (bug fixes). It is now important to do more in this area again.
The general complaint is still "scanning too long, scanning too often". Especially "too often" is irritating in situations with no visible reason to start scanning. It is so painful because (as says the next complaint) many features are not usable during scanning, either not available, or terribly slow.
We've been always focusing on the "too long/too often" problem, and slowly improving. There's still a lot of work there. As usual, we can continue making it faster, more efficient, fix bugs, but maybe we should also pay attention to the "not usable during scanning" side of the problem. The feeling is that some more effort might be spent to make the IDE work better for the user, be usable even during scanning, with main features always available and responsive, no matter the amount or speed of the scanning running meanwhile.
The user experience is that quite often the scanning does not find any change, or is unrelated to actual user's task (e.g. things like opening a new project, closing a project, or adding a library block navigation or code completion within already opened projects). In most cases all data seems available and the user would appreciate to be able to use the IDE even though the data may not be guaranteed as complete and correct at that moment. The issues this may cause seems minimal compared to what people face now when blocked completely by "scanning in progress". The fact that the data is not up-to-date for a moment should be easy to communicate.
Go to Type/Go to File (navigation)
Responsive navigation is essential for perceived performance of the IDE. Despite various improvements in the past, it is still not good. General requirements:
- fast, not influenced by other work the IDE is doing on background,
- available anytime, even during scanning, even with partial results (e.g. during up-to-date check, scanning another project, etc).
Ad speed: We have bug 177274 to track reports of slow Go to Type. Several fixes have been made already, more are needed. The speed should be "as you type". E.g. there is the possibility to move the index into memory, which would make the access faster.
Ad availability: There is a significant lag when trying Go to Type during up-to-date check (e.g. after startup). Partially it is a raw performance problem (ongoing up-to-date check I/O slows down requests to the index), but it is also because the index for given project is not opened until other projects it depends on are scanned first. If the project is big with many dependencies, it may take a long time till it comes to turn. Would be great if the index availability was independent on state of project scanning, or even opening (assuming the indexes exist already). In case of the initial scan the problem is similar in that the most interesting sources are processed last, but it can't be solved just by making the index available sooner -- the index must be created first.
Code completion, find usages, etc
Code completion, navigator view, finding usages, and other functionality not modifying sources that require java lang. infrastructure have similar general problems as navigation: speed and availability. They should not be blocked by scanning.
Ad speed: There's bug 179727 for tracking reported slow code completion. The amount of complaints and visible problems in CC is rather smaller (compared to Go to... navigation features).
A specific case is the delay in populating the navigator view when switching between files in editor, which makes the switching feel slow. (Changing file selection in Projects explorer has much faster response in Navigator. Perhaps a too aggressive refresh or lack of caching. If the file has not been changed externally, there's hardly a difference in what to show in Navigator when it is visited next time.)
Ad availability: These features (that require some parsing) are often even less available than Go to Type/Go to File. They are not available at all if a scanning is in progress. (E.g. bug 185696 is a good example.) Except the initial scan, it should just work with the latest data available, even if some scanning needs to run at the same moment. There is an idea of "transactional index" that should allow this mode.
- Bug 133943 that collects a lot of user feedback as well as some ideas.
- IDEUsableWhenScanning for list of possibly affected operations.
Some examples of scanning that should not block availability of features:
- when opening/closing a project (while there are other projects still opened),
- when project classpath is modified (added/removed library or project),
- when a file with lots of dependencies is edited by user,
- when "external" change is detected (by external process or VCS),
- after startup, when IDE is doing its long up-to-date check,
Scanning order based on relevance
If scanning could run independently and not block features, then the next thing to consider is the order of indexing/parsing -- to optimize how it delivers updates. In some cases it makes sense to first process files/changes closer to the user's current context (edited file, opened project) to make them sooner available in the index than rest of the less relevant files (other projects).
This could include a rescan after editing a file in editor, or up-to-date check after startup or opening a new project to make files of this project available for Go to File and Go to Type first before processing the project dependencies that take much longer and are less relevant for the user.
C++ projects (CND)
- Indexing infrastructure does not scale to size of real C++ projects (tens of thousands of files).
- CND team developed own infrastructure, specialized for C++ projects. E.g. Go to File/Go to Type is known to be faster and better available than in NB. (It's possible to turn on the NB indexing infrastructure for C++ projects, but it is too slow.)
- CND infrastructure is able to scale with additional HW (memory, CPU cores), e.g. use memory for more caching, CPUs for parallel parsing.
- We should investigate how to make the general indexing infrastructure usable for C++ projects, without affecting the users (in expected performance and scalability).
Other notes to scanning
- Lots of slowness is also due to I/O contention. Ideally all I/O but the one serving immediate user request should be temporarily stopped in the IDE. Introducing "idle I/O" might help... Another idea was to do more in-memory caching: if everything needed is in memory, there is no issue with I/O.
- There are still many reports of scanning that happens too often, in situations when no scanning should be initiated, running unexpectedly long. These are clearly bugs, need to be fixed.
- There are known inefficiencies in the way projects and Java infrastructure work together, how the classpath is composed and updated, how queries work... Besides speed, one manifestation is the inability to fix the infamous "scanning started when project is closed" (related to GlobalPathRegistry).
- Check for up-to-date indexes is needlessly loading all providers (besides it's not working correctly for projects reopened later - bug 180262).
Really big files are required to be editable in the IDE, e.g. C++ or XML. The IDE cannot handle well files bigger than ~1MB.
The new view hierarchy being prepared (e.g. for the word wrapping feature) is also a potentially big risk for editor performance.
- Fast stepping in debugger (bug 186517). Currently stepping is slowed down by updating views after each step (thread call stacks, variables, watches), next step must wait on the updates to finish. Instead, the updates should be quickly canceled when user invokes another step.
- Faster start, less I/O, e.g. bug 186808.
Performance tab in Options
There are various settings and switches that affect performance of the IDE in various ways. People often don't know about them. Would be nice to have a dedicated tab for performance in Options for that.
Some examples what could be configured here:
- automatic checking for external changes (on/off),
- scope of scanning after change in a file (same projects, all opened projects, etc),
- memory settings (e.g. max heap or max perm. gen. size),
- self profiling settings (limits when it starts and when slowness is reported),
- Lucene indexes in memory (on/off),
- some diagnostics (e.g. turning on/off RepositoryUpdater logging)?
Checking for external changes sometimes looks like a heavy task, started anytime the user switches back to the IDE, making the IDE less responsive for a while. Native filesystem listeners might solve the problems with the recursive listeners and detecting external file changes. There's an agreement it would improve performance and reliability, it's also the only way to recognize all changes correctly.
Performance optimizations for Equinox (mainly startup).
Maven 3 - quite some performance problems with maven based projects are attributed to the maven embedder. In Maven 3 it's performance should be improved, with more caching, etc, much more suitable for IDE integration. Maven3Embedder
Changes in self-profiling that made it available for platform (without profiler) introduced some performance problems, e.g. bug 185688, bug 186690. While the sampling itself is now less intrusive, processing the snapshot at the end is somewhat problematic.
More accurate slowness detector
In some areas there are many reports that are not very useful, mainly those close to the 3s limit. They often contain long class loading or very slow I/O, that can't be reasonably dealt with. We need to find a way to create less "garbage" snapshots, maybe also detect suspicious ones and for such have higher time limit for automatic reporting.
Improving performance has always been dependent on the ability to get relevant data or test cases to analyze. The self profiling improved this a lot, in many cases it's enough to have a profiling snapshot from the user and it's not necessary to reproduce the problem (which is often very difficult). This was a real breakthrough, however, there are still many problems that are difficult to reproduce even for the users who face them. If the problem happens randomly there may be no good moment to start self-profiling, or it may not even start when the IDE is blocked on 100% CPU. Also not everyone takes the effort to start over, repeat the problem, create snapshot, send it. It can be seen on the fact that vast majority of snapshots we receive are from the automatic slowness detector (which detects only specific types of problems), compared to snapshots created and sent manually.
It would be much easier for users to provide performance data if they could just create a snapshot retrospectively when they see a performance problem. This would require some kind of permanent sampling running all the time, keeping data for last 5 minutes (for example). So at any moment profiling data would be available for a post-mortem analysis, the user would not have to reproduce the problem again with self-profiling. The sampling method introduced in 6.9 has quite low overhead to make this possible.
For analyzing the snapshots this would require some additional features in profiler. The user would send the complete 5 minutes NPSS snapshot that would be too large to analyze in the classic CPU view with aggregated call trees. The tree view can be used only for certain time period that needs to be identified first. There could be some kind of "time line" view with possibility to select the relevant time interval and see it as the call tree. Possibly also with labels from gesture collector to make it easier to identify the user actions on the time line.
Tasks: Permanent Profiling
- TH: Throw away old data (older than ~5min)
- Modify Self Profile Action to trigger send via UI Gestured Collector
- Ensure people understand the possibility of permanent profiling (UI).
- TH: Share single thread dump between all currently running samplings
- Overlay data with java.util.logging.XMLFormater format
- Special understanding for UI Gestures Collector enhancements
- What data to represent for stacktraces?
- Start with simplest to implement, run usability study then
- Select part of timeline, generate regular snapshot from it
Tasks: Open All
- http://statistics.netbeans.org needs to generate either:
- descriptor with URLs
- one bundle with everything (stacktraces, XML gestures)
- npss file with additional URLs (to gestures XML)
- Maximum acceptable performance regression when instant profiling is on must not exceed 10%