BackgroundScan HowTo

Contents

Introduction

The background scanning integration does not affect current API clients as the change is fully backward compatible. But there are some areas where feature can benefit from the possibility to run concurrently with scan and improve the user experience. This document describes such a changes and also points to problems with wrongly written code.

Tasks that will run concurrently with the scan will see unresolvable types much more often that they do in 7.1. Virtually any task can occasionally encounter unresolvable types even in <=7.1. Tasks running concurrently with scan will not be able to resolve classes that were not scanned yet, and so they will encounter unresolvable types more often. These tasks therefore need to handle this situation gracefully. Suggestions on how to deal with such situations are given below.

Clients of runWhenScanFinished

The runWhenScanFinished method is invoked when the scan is finished and provides full ordering of submitted tasks and scan tasks. The behavior of this method was not changed by the background scan. From user point of view, some tasks can be reasonably done even during scan or up to date check. This includes navigation tasks, features that require one-file scope, like folding. Tasks such as the code completion can reasonably be performed during up to date check. To allow tasks to run during the scan, invocations of the runWhenScanFinished have to be replaced by runUserActionTask. runUserActionTask are performed without waiting till the end of the scan.

Some tasks require references (javax.lang.model.element.Element) from other files, which may not be available during scan, but still should run during the scan/up to date check. Such tasks may primarily use runUserActionTask, and use runWhenScanFinished as a fallback. The pattern is like this:

JavaSource source = JavaSource.forFile(some_file);
SomeTask task = SomeTask();
source.runUserTask(task);

// isSuccess indicates that the task was unable to complete because of potentially
// incomplete data.
if (SoureUtils.isScanInProgress() && !task.isSuccess()) {
      if(source.runWhenScanFinished(task) {
          ui.showScanInProgress();
      }
}

Such a pattern is used for example in Java Code Completion.

For support methods see Support Utilities

For navigation features to work during scan it was common to open at least the file during scan, it was done by checking SourceUtils.isScanInProgress or IndexingManager.isIndexing(). When this methods returned true only the source file was opened otherwise the Element was resolved if the resolution was unsuccessful the source file is opened otherwise the navigation to resolved Element was done. In the background scan the situation is much simple and the check for indexing can be removed, as seen in ElemenOpen. Also it's strongly recommended to prefer the ElementOpen support API for navigation implementation to custom implementation.

Clients of runUserActionTasks

Such a tasks are executed synchronously and concurrently to the scan. Also this is not a semantical change as the scan process used to be a special phase completion task which can be interrupted by the runUserActionTask. When the scan task was interrupted by the runUserTask the metadata for such a task may not be ready yet so such a task should be able to handle such an errors. The interruption was pretty common in NB 6.0 - NB 6.8 where the IDE was able to interrupt scan after indexing each source root or each class file. In the NB 6.8+ the IDE did not do the scan interruption after single file. The background scan runs the scan in parallel to runUserActionTask so the behavior is similar to binaries scan in NB 6.0 (interruption after each file) but with fixed isolation and atomicity, the runUserActionTask either sees the old state or a new state it does not see partial result as it was in case of scan interruption in NB 6.0.

As the scan cancel was broken in NB 6.8+ and the runUserActionTask javadoc talks about phase completion tasks and it may not be clear that scan is also phase completion task it's important to be defensive in them. Defensive as important even now as broken project dependencies cause the same kind of problem.

The special places to check are while:

  • Working with TypeMirror and Elements they should be checked for error types. When the tree has an error type and scan is in progress the action should be rescheduled into runWhenScanFinished as described below.
  • Using Elements.getTypeElement and Elements.getPackageElement they can return null when the old index generation does not contain them.
  • Using ElementHandle and TreePathHandle they can resolve to null when the old index generation does not contain type to which they should resolve.

The user of runUserTask should be aware of this. It's up to the concrete task semantic to choose the right solution. Tasks which inherently need correct up to date state like refactoring, compile on save, etc should use runWhenScanFinished. For task which operate on a single file using the runUserTask on JavaSource constructed for this file may improve user experience during the scan as the user may perform the action while the scan is running. For tasks providing an UI with some type info (for example list of all EJBs in the project) using the current cache state with update after scan finished is the preferred behavior. But the simple runWhenScanFinished works as well, it's the current trunk behavior.

The current cache state with update pattern looks like:

JavaSource src;
UI ui;
boolean scan = SourceUtil.isScanInProgress();
if (scan) {
    ui.setWarning("Scan in progress, showing non up to date data");
}
js.runUserActionTask({
    Elements elements = findSomeInterestingElements();
    ui.setModel(elements);
});
if (scan) {
    js.runWhenScanFinished({
    Elements elements = findSomeInterestingElements();
    ui.setModel(elements);
    ui.setWarning(null);
});

For support methods see Support Utilities

Clients of runModificationTask

Similarly as runUserActionTask the runModificationTask is executed synchronously and concurrently to the scan. When the API client does a local change it can directly call runModificationTask, such a task can be executed before the scan completes and types may not be correctly resolved. Clients which are doing more complex modifications and require correct type resolution, for example refactoring, should use runWhenScanFinish to ensure that the types are up to date.

The best way how to ensure that the types are up to date is:

final JavaSource src = javaSource.forFileObject(javaFile);
src.runWenScanFinished(
  new Task<CompilationController>() {
        public void run(CompilationController cc) {
            ModificationResult result = src.runModificationTask(new Task<WorkingCopy>() {
                 public void run(WorkingCopy wc) {
                    //Do changes
                 }
            });
            result.commit();
        }
  }, true);

In this case the runModificationTask is executed when types are up to date (all scans started before the call are finished) and no scan is started during the change.

Clients of Phase Completion Tasks

The Phase completion Tasks are registered by JavaSourceTaskFactories or Schedulers and executed automatically on event. The background scan does not introduce any incompatible change for them, only tasks marked as scan aware will be executed during the scan, unmarked tasks behave as in trunk.

  • For parsing.api users to mark the ParserResultTask as capable to run during the scan you need to extend the specialization of the ParserResultTask called IndexingAwareParserResultTask which takes a TaskIndexingMode parameter in constructer. The TaskIndexingMode.ALLOWED_DURING_SCAN enables the scan during the scan.

For generic ParserResultTask

public class MyTask extends IndexingAwareParserResultTask {
     MyTask() {
        super(TaskIndexingMode.ALLOWED_DURING_SCAN);
    }
}

For Java ParserResultTask

public class SemanticHighlighter extends JavaParserResultTask {
    SemanticHighlighter() {
        super(Phase.RESOLVED, TaskIndexingMode.ALLOWED_DURING_SCAN);
    }
}
  • For java.source users the new constructors with TaskIndexingMode were added into the base class JavaSourceTaskFactory and supporting classes CaretAwareJavaSourceTaskFactory, EditorAwareJavaSourceTaskFactory, LookupBasedJavaSourceTaskFactory, SelectionAwareJavaSourceTaskFactory. To enable the tasks during the scan you need to pass TaskIndexingMode.ALLOWED_DURING_SCAN into these constructors.
@org.openide.util.lookup.ServiceProvider(service=org.netbeans.api.java.source.JavaSourceTaskFactory.class)
public final class ClassMemberNavigatorJavaSourceFactory extends LookupBasedJavaSourceTaskFactory {
    public ClassMemberNavigatorJavaSourceFactory() {        
        super(
            Phase.ELEMENTS_RESOLVED,
            Priority.NORMAL,
            TaskIndexingMode.ALLOWED_DURING_SCAN);
    }
}

Support Utilities

To support the most common use case for runUserActionTask and runWhenScanFinished the ScanUtils utility class was added into java.source module. The ScanUtils provides both a blocking and a non blocking implementation of the use current state with possible refresh when scan finished pattern. The ScanUtils also provides utility methods to signal that task should be rescheduled in the runWhenScanFinished.

Typical use case of the ScanUtils blocking version is:

 final JavaSource src = JavaSource.forFileObject(java_file);
 final TreePath tp = ... ; // obtain a tree path
 ScanUtils.waitUserActionTask(src, new Task<CompilationController>() {
     public void run(CompilationController ctrl) {
         // possibly abort and wait for all info to become available
         Element e = ScanUtils.findElement(tp);
         if (!ScanUtils.isElementUsable(e)) {
               // report the error; element is not found or is otherwise unusable
        } else {
            // do the really useful work on Element e
        }
    }
},true);

The task is firstly run using runUserActionTask. It tries to resolve an Element from a TreePath using the ScanUtils.findElement(tp) helper method. When such an element exists and it's not an error it returns the Element otherwise it checks if the scan is active if so it throws a special Error subtype which signals that the task should be restarted in the runWhenScanFinished. This is done under assumption, that the element might not be yet reached by the scanner, and can actually exist. Reporting an error at this stage may lead to user confusion. Next the task checks if the element is an error using 'ScanUtils.isElementUsable(e). If it's an error it means that the source is broken and user should be notified about it otherwise it completes the task.

For non blocking call use ScanUtils.postUserActionTask which does the initial runUserActionTask synchronously but does not block until the runWhenScanFinished completes.

Test Utilities

In tests which do not involve scanning but use Phase Completion Tasks the tasks were performed immediately, the executer was not blocked by indexing. With the background scan the tasks, which are not subclasses of IndexingAwareParserResultTask or are not allowed during scan, are blocked until scan finishes. This may cause tests to fail or deadlock. The parsing.api unit tests provide a base class IndexingAwareTestCase which allows to run tests in old compatible mode.

Usage of IndexingAwareTestCase

public class CachingTest extends IndexingAwareTestCase {
    
    public CachingTest (String testName) {
        super (testName);
    }
....
}
Not logged in. Log in, Register

By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo