Scanning, indexing, refreshing sources

Technical Council meeting notes, January 20, 2009


Pavel Flaska, Tomas Zezula, Vita Stejskal, Jarda Tulach, Marian Mirilovic, Petr Hrebejk, Petr Pisl, Petr Jiricka, Jesse Glick, Jano Rojcek


Users have been complaining about slow or frequent scanning of sources, in several contexts. Not a single issue, rather, this is a set of loosely related problems.

Discussion about Parsing Requirements

Many of these complaints will be solved by addressing the Performance requirements on the Parsing API: ParsingAPIRequirements, section Performance Requirements. Here is the status:

  • Perf.1, Perf.2 - on track for 7.0 (Parsing API), except:
   YAML, Python are a question, YAML more important
  • Perf.3 - on track
  • Perf.4 - was not feasible in 6.5, doable with Parsing API
   not staffed, estimated 1 week effort
  • Perf.5 - in progress by Standa Aubrecht, facing technical issues:
    1. Query per-project rather than per-sourcegroup, need list of source groups per project in a generic way,
    or index could provide indexes inside directory). Vita (or Tomas) need to talk to Standa.
    1. Need a way to trigger rescan by tasklist if the user changes a filter.
    also will need changes in the Parsing/Indexing API
  • Perf.6 - hard to do (slow): topological sort involving SourceForBinaryQuery is slow. Could it be optimized?
   Pavel Flaska to investigate. This item is not critical.
  • Perf.7 - done
  • Perf.8 - infrastructure on track for 7.0 (Parsing API), some more code needs to be rewritten (on track)

Discussion about Extra Module to Disable Automatic Refresh

In addition to this, users complain about the IDE rescanning too often, in situations such as:

  • after startup
  • misconfigured freeform project

Need a patch that would:

  • disable up to date check after (second) startup
  • disable scanning after removing a jar file
  • provide a manual refresh action

Item Perf.6 is required for this (speeding up topological sort). An additional speedup can be gained by not refreshing source directories and files, by only refreshing source roots and jars. This would be useful in scenarios with many files inside few source roots.

Should we strive to have 100% reliable listening on file changes (issue 33162), including getting file change events after IDE startup for files changed since last IDE session? Probably not - this would help the reliability, but would not help the problem with long scanning after startup. Users who requested the elimination of the startup scan are ready to give up the reliability.

Java team's help needed to polish the patch - not staffed. Required well before beta; not for M2. Then we would provide an autoupdate module that would enable this behavior.

Additional use case: After external changes (git update), indexes are not up to date: it is preferable to have a "refresh sources" action than forcing users to restart the IDE

Not logged in. Log in, Register

By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo