PostNB69PerfTopics

(Difference between revisions)
m (Code completion, find usages, etc)
m
Line 8: Line 8:
Scanning-related issues are still at the top of complaints about performance from our users. No performance improvements were planned for 6.9, there were some regressions solved and a few partial improvements (bug fixes). It is now important to do more in this area again.
Scanning-related issues are still at the top of complaints about performance from our users. No performance improvements were planned for 6.9, there were some regressions solved and a few partial improvements (bug fixes). It is now important to do more in this area again.
-
As usual, we can try to make things faster, more efficient and scalable, but we should also pay more attention to making '''the IDE work better for the user'''. That is for example making the IDE usable even during scanning, with main features always available and responsive, no matter the amount or speed of the scanning running meanwhile.
+
As usual, we can try to make things faster, more efficient and scalable, but we should also pay more attention to '''make the IDE work better for the user'''. That is for example making the IDE usable even during scanning, with main features always available and responsive, no matter the amount or speed of the scanning running meanwhile.
====Go to Type/Go to File (navigation)====
====Go to Type/Go to File (navigation)====
Line 15: Line 15:
* '''available''' as soon as possible (after startup, when opening a project), even with partial results.
* '''available''' as soon as possible (after startup, when opening a project), even with partial results.
-
Ad speed: We have [http://netbeans.org/bugzilla/show_bug.cgi?id=177274 bug 177274] to track reports of slow response of Go to Type. Several fixes have been made already, more are needed. The speed should be "as you type". E.g. there is the possibility to move the index into memory, which would make the access faster.
+
Ad speed: We have [http://netbeans.org/bugzilla/show_bug.cgi?id=177274 bug 177274] to track reports of '''slow Go to Type'''. Several fixes have been made already, more are needed. The speed should be "as you type". E.g. there is the possibility to move the index into memory, which would make the access faster.
Ad availability: There are significant delays when trying Go to Type soon after startup or when opening a project. E.g. after startup all projects must be opened first, then indexing (up-to-date check) goes through the projects. Even if the infrastructure can provide index for Go to Type/Go to File before complete scanning is finished, the user project, from which the user most likely wants something to open quickly, is processed as last. Would be great if the index availability was independent on state of projects opening/scanning. There's an idea of a more independent "mega index" that might allow this.
Ad availability: There are significant delays when trying Go to Type soon after startup or when opening a project. E.g. after startup all projects must be opened first, then indexing (up-to-date check) goes through the projects. Even if the infrastructure can provide index for Go to Type/Go to File before complete scanning is finished, the user project, from which the user most likely wants something to open quickly, is processed as last. Would be great if the index availability was independent on state of projects opening/scanning. There's an idea of a more independent "mega index" that might allow this.
====Code completion, find usages, etc====
====Code completion, find usages, etc====
-
Code completion, navigator view, finding usages, and other functionality not modifying sources that require java lang. infrastructure have similar general problems as navigation: speed and availability.
+
Code completion, navigator view, finding usages, and other functionality not modifying sources that require java lang. infrastructure have similar general problems as navigation: speed and availability. They should not be blocked by scanning.
-
There's [http://netbeans.org/bugzilla/show_bug.cgi?id=179727 bug 179727] for tracking reported slow code completion.
+
There's [http://netbeans.org/bugzilla/show_bug.cgi?id=179727 bug 179727] for tracking reported '''slow code completion'''. Another example is the '''delay of populating navigator view''' when switching between files in editor, which makes the switching feel slow. (Changing file selection in projects explorer has much faster response in Navigator. Perhaps the data could be cached as there is low chance the files are modified externally.)
-
Availability is often even worse than with Go to Type/Go to File. It's not available at all if a scanning is in progress. (E.g. [http://netbeans.org/bugzilla/show_bug.cgi?id=185696 bug 185696] is a good example.) Except the initial scan, it should just work with the latest data available, even if some scanning needs to run at the same moment. There as an idea of "transactional index" that should allow this mode.
+
Availability of these features is often even worse than with Go to Type/Go to File. They are '''not available at all if a scanning is in progress'''. (E.g. [http://netbeans.org/bugzilla/show_bug.cgi?id=185696 bug 185696] is a good example.) Except the initial scan, it should just work with the latest data available, even if some scanning needs to run at the same moment. There as an idea of "transactional index" that should allow this mode.
See also:
See also:
Line 33: Line 33:
*  Indexing infrastructure does not scale to size of real C++ projects (tens of thousands of files).
*  Indexing infrastructure does not scale to size of real C++ projects (tens of thousands of files).
* CND team developed own infrastructure, specialized for C++ projects. E.g. Go to File/Go to Type is known to be faster and better available than in NB. (It's possible to turn on the NB indexing infrastructure for CND projects, but it is too slow.)
* CND team developed own infrastructure, specialized for C++ projects. E.g. Go to File/Go to Type is known to be faster and better available than in NB. (It's possible to turn on the NB indexing infrastructure for CND projects, but it is too slow.)
-
* CND infrastructure is able to scale with additional HW (memory, CPU cores), e.g. use memory for more caching, CPUs for parallel parsing.
+
* CND infrastructure is able to '''scale with additional HW''' (memory, CPU cores), e.g. use memory for more caching, CPUs for parallel parsing.
-
* We should investigate how to combine both technologies for better results, whether is possible to extend the general indexing infrastructure to be usable for CND projects, without affecting CND users (expected performance and scalability).
+
* We should investigate how to make the general indexing infrastructure to be usable for CND projects, without affecting CND users (expected performance and scalability).
====Other notes to scanning====
====Other notes to scanning====
-
* Lots of slowness is also due to I/O contention. Ideally all I/O but the one serving immediate user request should be temporarily stopped in the IDE. Introducing "idle I/O"...
+
* Lots of slowness is also due to I/O contention. Ideally all I/O but the one serving immediate user request should be temporarily stopped in the IDE. Introducing "idle I/O" might help...
* There are known inefficiencies in the way projects and Java infrastructure work together, how the classpath is composed and updated, how queries work... Besides speed, one manifestation is the inability to fix the infamous "scanning started when project is closed" (related to GlobalPathRegistry).
* There are known inefficiencies in the way projects and Java infrastructure work together, how the classpath is composed and updated, how queries work... Besides speed, one manifestation is the inability to fix the infamous "scanning started when project is closed" (related to GlobalPathRegistry).
* Check for up-to-date indexes is needlessly loading all providers (besides it's not working correctly for projects reopened later - [http://netbeans.org/bugzilla/show_bug.cgi?id=180262 bug 180262]).
* Check for up-to-date indexes is needlessly loading all providers (besides it's not working correctly for projects reopened later - [http://netbeans.org/bugzilla/show_bug.cgi?id=180262 bug 180262]).

Revision as of 08:53, 14 June 2010

As usual with every release, when finishing NetBeans 6.9 we start thinking about what performance issues to attack in next releases. This page collects performance topics that are actually most important for our users, and ideas how we could improve the IDE performance in various aspects (UI responsiveness, scalability, etc).

Note: This page is not a plan yet, just an area for gathering ideas. If you have any idea or see an important problem, just throw it here. We'll finalize the content, prioritize and select specific topics as part of next release planning. See also previous PostNB68PerfTopics, PostNB67PerfTopics, PostNB65PerfTopics.

Indexing and parsing infrastructure

Scanning-related issues are still at the top of complaints about performance from our users. No performance improvements were planned for 6.9, there were some regressions solved and a few partial improvements (bug fixes). It is now important to do more in this area again.

As usual, we can try to make things faster, more efficient and scalable, but we should also pay more attention to make the IDE work better for the user. That is for example making the IDE usable even during scanning, with main features always available and responsive, no matter the amount or speed of the scanning running meanwhile.

Go to Type/Go to File (navigation)

Responsive navigation is essential for perceived performance of the IDE. Despite various improvements in the past, it is still not good. General requirements:

  • fast, not influenced by other work the IDE is doing on background,
  • available as soon as possible (after startup, when opening a project), even with partial results.

Ad speed: We have bug 177274 to track reports of slow Go to Type. Several fixes have been made already, more are needed. The speed should be "as you type". E.g. there is the possibility to move the index into memory, which would make the access faster.

Ad availability: There are significant delays when trying Go to Type soon after startup or when opening a project. E.g. after startup all projects must be opened first, then indexing (up-to-date check) goes through the projects. Even if the infrastructure can provide index for Go to Type/Go to File before complete scanning is finished, the user project, from which the user most likely wants something to open quickly, is processed as last. Would be great if the index availability was independent on state of projects opening/scanning. There's an idea of a more independent "mega index" that might allow this.

Code completion, find usages, etc

Code completion, navigator view, finding usages, and other functionality not modifying sources that require java lang. infrastructure have similar general problems as navigation: speed and availability. They should not be blocked by scanning.

There's bug 179727 for tracking reported slow code completion. Another example is the delay of populating navigator view when switching between files in editor, which makes the switching feel slow. (Changing file selection in projects explorer has much faster response in Navigator. Perhaps the data could be cached as there is low chance the files are modified externally.)

Availability of these features is often even worse than with Go to Type/Go to File. They are not available at all if a scanning is in progress. (E.g. bug 185696 is a good example.) Except the initial scan, it should just work with the latest data available, even if some scanning needs to run at the same moment. There as an idea of "transactional index" that should allow this mode.

See also:

C++ projects (CND)

  • Indexing infrastructure does not scale to size of real C++ projects (tens of thousands of files).
  • CND team developed own infrastructure, specialized for C++ projects. E.g. Go to File/Go to Type is known to be faster and better available than in NB. (It's possible to turn on the NB indexing infrastructure for CND projects, but it is too slow.)
  • CND infrastructure is able to scale with additional HW (memory, CPU cores), e.g. use memory for more caching, CPUs for parallel parsing.
  • We should investigate how to make the general indexing infrastructure to be usable for CND projects, without affecting CND users (expected performance and scalability).

Other notes to scanning

  • Lots of slowness is also due to I/O contention. Ideally all I/O but the one serving immediate user request should be temporarily stopped in the IDE. Introducing "idle I/O" might help...
  • There are known inefficiencies in the way projects and Java infrastructure work together, how the classpath is composed and updated, how queries work... Besides speed, one manifestation is the inability to fix the infamous "scanning started when project is closed" (related to GlobalPathRegistry).
  • Check for up-to-date indexes is needlessly loading all providers (besides it's not working correctly for projects reopened later - bug 180262).

Editor

Really big files are required to be editable in the IDE, e.g. C++ or XML. The IDE cannot handle well files bigger than ~1MB.

The new view hierarchy being prepared (e.g. for the word wrapping feature) is also a potentially big risk for editor performance.

Performance tab in Options

There are various settings and switches that affect performance of the IDE in various ways. People often don't know about them. Would be nice to have a dedicated tab for performance in Options for that.

Some examples what could be configured here:

  • automatic checking for external changes (on/off),
  • scope of scanning after change in a file (same projects, all opened projects, etc),
  • memory settings (e.g. max heap or max perm. gen. size),
  • self profiling settings (limits when it starts and when slowness is reported),
  • Lucene indexes in memory (on/off),
  • some diagnostics (e.g. turning on/off RepositoryUpdater logging)?
  • etc.

Platform

Native filesystem listeners - should solve problems with recursive listeners and detecting external file changes every time the IDE gains focus. There's an agreement it would improve performance and reliability, it's also the only way to recognize all changes correctly.

Performance optimizations for Equinox (mainly startup).

Maven 3 - quite some performance problems with maven based projects are attributed to the maven embedder. In Maven 3 it's performance should be improved, with more caching, etc, much more suitable for IDE integration.

Self-profiling available for platform introduced some performance problems, e.g. bug 185688, bug 186690. Perhaps should consider to use the original sampler available in profiler, if profiler is part of the IDE run by user.

Debugger

  • Fast stepping in debugger (bug 186517). Currently stepping is slowed down by updating views after each step (thread call stacks, variables, watches), next step must wait on the updates to finish. Instead, the updates should be quickly canceled when user invokes another step.
  • Faster start, less I/O, e.g. bug 186808.
Not logged in. Log in, Register

By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo