PostNB69PerfTopics

(Difference between revisions)
m
m (Indexing and parsing infrastructure)
Line 8: Line 8:
Scanning-related issues are still at the top of complaints about performance from our users. No performance improvements were planned for 6.9, there were some regressions solved and a few partial improvements (bug fixes). It is now important to do more in this area again.
Scanning-related issues are still at the top of complaints about performance from our users. No performance improvements were planned for 6.9, there were some regressions solved and a few partial improvements (bug fixes). It is now important to do more in this area again.
-
As usual, we can try to make things faster, more efficient and scalable, but it becomes clearer with every release that we need to put more effort to '''make the IDE work better for the user'''. That is for example making the IDE usable even during scanning, with main features always available and responsive, no matter the amount or speed of the scanning running meanwhile. Very often the scanning is either not needed (does not find any change) or quite unrelated to actual user's task for which all data is available (i.e. opening a new project or adding a library blocks navigation or CC within the existing project).
+
As usual, we can try to make things faster, more efficient and scalable, but it becomes clearer with every release that we need to put more effort to '''make the IDE work better for the user'''. That is for example making the IDE usable even during scanning, with main features always available and responsive, no matter the amount or speed of the scanning running meanwhile. Very often the scanning does not find any change, or is quite unrelated to actual user's task for which all data is available (i.e. opening a new project or adding a library blocks navigation or CC within the existing project).
====Go to Type/Go to File (navigation)====
====Go to Type/Go to File (navigation)====

Revision as of 16:44, 14 June 2010

As usual with every release, when finishing NetBeans 6.9 we start thinking about what performance issues to attack in next releases. This page collects performance topics that are actually most important for our users, and ideas how we could improve the IDE performance in various aspects (UI responsiveness, scalability, etc).

Note: This page is not a plan yet, it's an area for gathering ideas, some may be quite speculative. It's also more a list of problems than solutions or goals. If you have any idea or see an important problem, just throw it here. We'll finalize the content, prioritize and select specific topics as part of next release planning. See also previous PostNB68PerfTopics, PostNB67PerfTopics, PostNB65PerfTopics.

Indexing and parsing infrastructure

Scanning-related issues are still at the top of complaints about performance from our users. No performance improvements were planned for 6.9, there were some regressions solved and a few partial improvements (bug fixes). It is now important to do more in this area again.

As usual, we can try to make things faster, more efficient and scalable, but it becomes clearer with every release that we need to put more effort to make the IDE work better for the user. That is for example making the IDE usable even during scanning, with main features always available and responsive, no matter the amount or speed of the scanning running meanwhile. Very often the scanning does not find any change, or is quite unrelated to actual user's task for which all data is available (i.e. opening a new project or adding a library blocks navigation or CC within the existing project).

Go to Type/Go to File (navigation)

Responsive navigation is essential for perceived performance of the IDE. Despite various improvements in the past, it is still not good. General requirements:

  • fast, not influenced by other work the IDE is doing on background,
  • available as soon as possible (after startup, when opening a project), even with partial results.

Ad speed: We have bug 177274 to track reports of slow Go to Type. Several fixes have been made already, more are needed. The speed should be "as you type". E.g. there is the possibility to move the index into memory, which would make the access faster.

Ad availability: There are significant delays when trying Go to Type soon after startup or when opening a project. E.g. after startup all projects must be opened first, then indexing (up-to-date check) goes through the projects. Even if the infrastructure can provide index for Go to Type/Go to File before complete scanning is finished, the user project, from which the user most likely wants something to open quickly, is processed as last. Would be great if the index availability was independent on state of projects opening/scanning. There's an idea of a more independent "mega index" that might allow this.

Code completion, find usages, etc

Code completion, navigator view, finding usages, and other functionality not modifying sources that require java lang. infrastructure have similar general problems as navigation: speed and availability. They should not be blocked by scanning.

Ad speed: There's bug 179727 for tracking reported slow code completion. Another example is the delay in populating the navigator view when switching between files in editor, which makes the switching feel slow. (Changing file selection in projects explorer has much faster response in Navigator. Perhaps the data could be cached as there is low chance the files are modified externally.)

Ad availability: These features (that require some parsing) are often even less available than Go to Type/Go to File. They are not available at all if a scanning is in progress. (E.g. bug 185696 is a good example.) Except the initial scan, it should just work with the latest data available, even if some scanning needs to run at the same moment. There as an idea of "transactional index" that should allow this mode.

See also:

C++ projects (CND)

  • Indexing infrastructure does not scale to size of real C++ projects (tens of thousands of files).
  • CND team developed own infrastructure, specialized for C++ projects. E.g. Go to File/Go to Type is known to be faster and better available than in NB. (It's possible to turn on the NB indexing infrastructure for C++ projects, but it is too slow.)
  • CND infrastructure is able to scale with additional HW (memory, CPU cores), e.g. use memory for more caching, CPUs for parallel parsing.
  • We should investigate how to make the general indexing infrastructure usable for C++ projects, without affecting the users (in expected performance and scalability).

Other notes to scanning

  • Lots of slowness is also due to I/O contention. Ideally all I/O but the one serving immediate user request should be temporarily stopped in the IDE. Introducing "idle I/O" might help...
  • There are known inefficiencies in the way projects and Java infrastructure work together, how the classpath is composed and updated, how queries work... Besides speed, one manifestation is the inability to fix the infamous "scanning started when project is closed" (related to GlobalPathRegistry).
  • Check for up-to-date indexes is needlessly loading all providers (besides it's not working correctly for projects reopened later - bug 180262).

Editor

Really big files are required to be editable in the IDE, e.g. C++ or XML. The IDE cannot handle well files bigger than ~1MB.

The new view hierarchy being prepared (e.g. for the word wrapping feature) is also a potentially big risk for editor performance.

Performance tab in Options

There are various settings and switches that affect performance of the IDE in various ways. People often don't know about them. Would be nice to have a dedicated tab for performance in Options for that.

Some examples what could be configured here:

  • automatic checking for external changes (on/off),
  • scope of scanning after change in a file (same projects, all opened projects, etc),
  • memory settings (e.g. max heap or max perm. gen. size),
  • self profiling settings (limits when it starts and when slowness is reported),
  • Lucene indexes in memory (on/off),
  • some diagnostics (e.g. turning on/off RepositoryUpdater logging)?
  • etc.

Platform

Native filesystem listeners - should solve problems with recursive listeners and detecting external file changes every time the IDE gains focus. There's an agreement it would improve performance and reliability, it's also the only way to recognize all changes correctly.

Performance optimizations for Equinox (mainly startup).

Maven 3 - quite some performance problems with maven based projects are attributed to the maven embedder. In Maven 3 it's performance should be improved, with more caching, etc, much more suitable for IDE integration.

Self-profiling available for platform introduced some performance problems, e.g. bug 185688, bug 186690. Perhaps should consider to use the original sampler available in profiler, if profiler is part of the IDE run by user.

Debugger

  • Fast stepping in debugger (bug 186517). Currently stepping is slowed down by updating views after each step (thread call stacks, variables, watches), next step must wait on the updates to finish. Instead, the updates should be quickly canceled when user invokes another step.
  • Faster start, less I/O, e.g. bug 186808.
Not logged in. Log in, Register

By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo