PostNB69PerfTopics

Revision as of 14:24, 16 June 2010 by Tpavek (Talk | contribs)

As usual with every release, when finishing NetBeans 6.9 we start thinking about what performance issues to attack in next releases. This page collects performance topics that are actually most important for our users, and ideas how we could improve the IDE performance in various aspects (UI responsiveness, scalability, etc).

Note: This page is not a plan yet, it's an area for gathering ideas, some may be quite speculative. It's also more a list of problems than solutions or goals. If you have any idea or see an important problem, just throw it here. We'll finalize the content, prioritize and select specific topics as part of next release planning. See also previous PostNB68PerfTopics, PostNB67PerfTopics, PostNB65PerfTopics.

Indexing and parsing infrastructure

Scanning-related issues are still at the top of complaints about performance from our users. No performance improvements were planned for 6.9, there were some regressions solved and a few partial improvements (bug fixes). It is now important to do more in this area again.

As usual, we can try to make things faster, more efficient and scalable, but it becomes clearer with every release that we need to put more effort to make the IDE work better for the user. That is for example making the IDE usable even during scanning, with main features always available and responsive, no matter the amount or speed of the scanning running meanwhile. Very often the scanning does not find any change, or is quite unrelated to actual user's task (e.g. opening a new project or adding a library blocks navigation or CC within the existing project). In most cases all data is available and the user would appreciate it's possible to use the IDE even though not all data are guaranteed to be complete and correct at that moment. The frequency and severity of issues this may cause is minimal compared to what they face now when blocked completely why "scanning in progress".

Go to Type/Go to File (navigation)

Responsive navigation is essential for perceived performance of the IDE. Despite various improvements in the past, it is still not good. General requirements:

  • fast, not influenced by other work the IDE is doing on background,
  • available as soon as possible (after startup, when opening a project), even with partial results.

Ad speed: We have bug 177274 to track reports of slow Go to Type. Several fixes have been made already, more are needed. The speed should be "as you type". E.g. there is the possibility to move the index into memory, which would make the access faster.

Ad availability: There are significant delays when trying Go to Type soon after startup or when opening a project. E.g. after startup all projects must be opened first, then indexing (up-to-date check) goes through the projects. Would be great if the index availability was independent on state of projects opening/scanning. There's an idea of a more independent "mega index" that might allow this.

Another problem is apparent on opening large projects. When a new project is being opened, the data for index must be collected first. But due to project dependencies, the user project is scanned as the last one (except tests), while the user would prefer to scan it first so it is possible to use at least Go to File/Type to somewhat navigate in the project while waiting on scanning to finish.

Code completion, find usages, etc

Code completion, navigator view, finding usages, and other functionality not modifying sources that require java lang. infrastructure have similar general problems as navigation: speed and availability. They should not be blocked by scanning.

Ad speed: There's bug 179727 for tracking reported slow code completion.

Another example is the delay in populating the navigator view when switching between files in editor, which makes the switching feel slow. (Changing file selection in Projects explorer has much faster response in Navigator. Perhaps a too aggressive refresh or lack of caching. If the file has not been changed externally, there's hardly a difference in what to show in Navigator.)

Ad availability: These features (that require some parsing) are often even less available than Go to Type/Go to File. They are not available at all if a scanning is in progress. (E.g. bug 185696 is a good example.) Except the initial scan, it should just work with the latest data available, even if some scanning needs to run at the same moment. There as an idea of "transactional index" that should allow this mode.

See also:

C++ projects (CND)

  • Indexing infrastructure does not scale to size of real C++ projects (tens of thousands of files).
  • CND team developed own infrastructure, specialized for C++ projects. E.g. Go to File/Go to Type is known to be faster and better available than in NB. (It's possible to turn on the NB indexing infrastructure for C++ projects, but it is too slow.)
  • CND infrastructure is able to scale with additional HW (memory, CPU cores), e.g. use memory for more caching, CPUs for parallel parsing.
  • We should investigate how to make the general indexing infrastructure usable for C++ projects, without affecting the users (in expected performance and scalability).

Other notes to scanning

  • Lots of slowness is also due to I/O contention. Ideally all I/O but the one serving immediate user request should be temporarily stopped in the IDE. Introducing "idle I/O" might help...
  • There are known inefficiencies in the way projects and Java infrastructure work together, how the classpath is composed and updated, how queries work... Besides speed, one manifestation is the inability to fix the infamous "scanning started when project is closed" (related to GlobalPathRegistry).
  • Check for up-to-date indexes is needlessly loading all providers (besides it's not working correctly for projects reopened later - bug 180262).

Editor

Really big files are required to be editable in the IDE, e.g. C++ or XML. The IDE cannot handle well files bigger than ~1MB.

The new view hierarchy being prepared (e.g. for the word wrapping feature) is also a potentially big risk for editor performance.

Performance tab in Options

There are various settings and switches that affect performance of the IDE in various ways. People often don't know about them. Would be nice to have a dedicated tab for performance in Options for that.

Some examples what could be configured here:

  • automatic checking for external changes (on/off),
  • scope of scanning after change in a file (same projects, all opened projects, etc),
  • memory settings (e.g. max heap or max perm. gen. size),
  • self profiling settings (limits when it starts and when slowness is reported),
  • Lucene indexes in memory (on/off),
  • some diagnostics (e.g. turning on/off RepositoryUpdater logging)?
  • etc.

Platform

Native filesystem listeners - should solve problems with recursive listeners and detecting external file changes every time the IDE gains focus. There's an agreement it would improve performance and reliability, it's also the only way to recognize all changes correctly.

Performance optimizations for Equinox (mainly startup).

Maven 3 - quite some performance problems with maven based projects are attributed to the maven embedder. In Maven 3 it's performance should be improved, with more caching, etc, much more suitable for IDE integration.

Diagnostics

Self-profiling fixes

Changes in self-profiling that made it available for platform (without profiler) introduced some performance problems, e.g. bug 185688, bug 186690.

Permanent sampling

Improving performance has always been dependent on the ability to get relevant data or test cases to analyze. The self profiling improved this a lot, in many cases it's enough to have a profiling snapshot from the user and it's not necessary to reproduce the problem (which is often very difficult). This was a real breakthrough, however, there are still many problems that are difficult to reproduce even for the users, or they simply don't want to put the effort into repeating the problem, creating and sending a snapshot. It can be seen on the fact that vast majority of snapshots we receive are from the automatic slowness detector, compared to snapshots created and sent manually.

It would be much easier to provide data for solving performance problems if the user could just create a snapshot retrospectively when they see a performance problem. This would require some kind of permanent sampling running all the time, keeping data for last 5 minutes (for example). So at any moment profiling data would be available for post-mortem analysis. The sampling method introduced in 6.9 has quite low overhead to make this possible.

This would require additional features in profiler to be able to select the relevant time interval and create the classic CPU call tree from it. (The user would just send the complete 5 minutes NPSS snapshot.) There could be some kind of time line view, possibly with labels from gesture collector to make it easier to identify the user actions.

Debugger

  • Fast stepping in debugger (bug 186517). Currently stepping is slowed down by updating views after each step (thread call stacks, variables, watches), next step must wait on the updates to finish. Instead, the updates should be quickly canceled when user invokes another step.
  • Faster start, less I/O, e.g. bug 186808.
Not logged in. Log in, Register

By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo