PluginManagerIssues

Contents


Issues

These are the most important issues with the current implementation of plugin manager (as of M10):

  1. New format to define grouping of modules. The new format has not been reviewed. I does not scale - has to be maintained centrally. Instead of generating the new features centrally, module writers should be able to define the items visible in plugin manager for every module and every update center (even third party), preferably using the existing module system APIs and tools.
  2. Two modes in UI: Plugins and NBMs, neither of them solves all use cases. Both modes are mixed together, e.g. when disabling a Plugin the IDE shows a list of modules that will be disabled. It is hard to find the switch. We need one mode that works for all use cases.
  3. Many implementation modules do not make any sense for the end user, should not ever be visible in UI for the end user at all. Current autoload modules are an example of this but more modules should be hidden then the current autoloads.
  4. "Plugins" (AKA installable components) do not provide enough information about IDE features and are two big so that another level of detail is needed.

Solutions

Currently there are modules (NBMs), each of which can specify a localizable OpenIDE-Module-Display-Category, and there are the new "features". The "features" which require generation of a new format of update center definition will be eliminated, and the model will be returned back to using only NetBeans modules which specify their categories.

The list of modules that are displayed to the user will be trimmed down to show only modules which represent important user level features. These will be either existing modules, in cases where there is naturally a module that represents a user level feature and depends on other implementation modules which do not represent important user level features. In case where there is no such natural module that depends on all implementation modules we will create a new module to represent the visible feature in UI and it will depend on all implementation modules. Implementation modules that do not represent important user level features will be turned into autoload or eager modules. Neither autoload nor eager modules will be displayed in UI.

The "important user level features" will be determined to satisfy a few requirements:

  • It should represent a feature that the user can understand and cares about
  • It should be granular enough that the user can enable/disable and install/uninstall parts of IDE to control performance, UI complexity, disable offending buggy modules, etc. I.e. not too large.
  • It should be safe to disable, i.e. disabling it will not put the application into a state in which the Plugin Manager itself is inaccessible.
  • Important IDE features should appear on this list to make it easy to find then for installation and to make it clear that the IDE provides them.
  • This is even more important for modules on update center then for modules in NetBeans distribution where the user will likely quite often select the whole module category.
  • Providing enough information about features can be helped by providing good description for the visible modules so that they describe all functionality of the hidden modules which they represent.

Obviously these are more guidelines then rules. Final decision will be made case by case by module owners, technical leads, etc.

Module category (String) will be set to match the installable components where a module belongs, such as Base IDE, Java IDE, Web and Java EE, SOA, C/C++, UML, Ruby, etc.

The UI will have one mode in which it will show all regular modules and it will show the column with module categories, i.e. what's currently called "NetBeans Modules (NBM)" mode. The "Advanced" switch between Plugins and NetBeans Modules (NBM) will be removed.

To install additional installer components the user will select all modules in the corresponding category.

Note: This UI does not work for module developer. But even the current UI does not provide all details that were available in Module Manager in 5.5. For 6.0 we will keep the Module Manager on update center as a solution for module developers. In next release we need to reconsider if all the details (and possibly more) needed by module developers should be integrated into the plugin manager UI in a special "Details" mode (which will not be shown to the user unless she is doing plugin development).

List of regular modules in distribution

The list of modules in standard distribution should be reasonably short. The goal here is not so much to impress the user (that is more important for the update center plugins) as to install/uninstall and enable/disable parts of IDE. Specifically most module related to platform, editing of basic files, etc., do not need to be shown individually.

Default ordering of the features within category is important, needs to be defined (if the user sorts the list by categories).

Module Category
Platform Platform
Editing Files Base IDE
Database Base IDE
CVS Base IDE
Local History Base IDE
Subversion Base IDE
Task List Base IDE
Java Script Base IDE
NetBeans Plugin Development NetBeans Plugin Development
GUI Builder Java
Java Java
XML and Schema Java (?!)
C/C++ C/C++
JRuby and Rails Ruby
Ruby and Rails Ruby
Java EE Web and Java EE
Web Applications Web and Java EE
BEA WebLogic Server Web and Java EE
GlassFish Web and Java EE
JBoss Application Server Web and Java EE
Tomcat Web and Java EE
Struts Web and Java EE
JSF Web and Java EE
Visual JSF Web and Java EE
Web Services Web and Java EE
BPEL SOA
Composite Application SOA
Identity SOA
IEP SOA
SOA SOA
CAPS SOA
UML UML
Profiler Profiling
Mobility Mobility
Visual Mobility Designer Mobility
CLDC Mobility
CDC Mobility
CDC Nokia S80 Mobility
CDC NSIcom Mobility
CDC Ricoh Mobility
CDC SavaJe Mobility
CDC Sony Ericsson Mobility
CDC Sun VM and CDC Tool Mobility


List of regular modules on update center

Modules from development update center, for illustration only. Specific list TBD.

Module Category
Code Template Tools Editing
Insert Unicode Editing
Next Error Action Editing
Eclipse Project Importer Import
JBuilder Project Importer Import
Ant 1.7.0 Documentation Java
Archiver Java
Freeform Project Extras Java
INSANE module Java
Jackpot Java
Jalopy Java Source Code Formatter Java
JMX Java
Maven Java
NetBeans API Documentation NetBeans Plugin Development
NetBeans API Samples NetBeans Plugin Development
NetBeans Runtime Inspector NetBeans Plugin Development
NetBeans Scripting Language Development Tools NetBeans Plugin Development
Groovy Languages
PHP Languages
Prolog Editor Support Languages
Scala Languages
TCL Editor Support Languages
Velocity Editor Support Languages
Embedded JavaScript (EJS) Phobos
Ruby Samples Ruby
Load Generator Testing
Build Monitor Utilities
Developer Collaboration Utilities
DocBook Utilities
Explore from Here Utilities
Filters Utilities
Folder To Html Utilities
Generic Navigator Utilities
Group of Files Utilities
Mercurial Utilities
Mount Utilities
Object Type Changer Utilities
Project Minder Utilities
Project Packager Utilities
Quick File Chooser Utilities
Task List Utilities
GlassFish V3 Module Web and Java EE
GWT4NB Web and Java EE
HTML Project Support Web and Java EE


Implementation

  • Enable/disable - should work correctly (+/- defects)
  • When showing dependencies - modules that will have to be enabled/disabled as a result of enabling/disabling selected modules - only regular modules will be displayed
  • Potential issue is that when the user disables a feature she expects all of it's UI to disappear. Modules which provide UI and API will be loaded if another module has a dependency on them and there will be no way to indicate their status in plugin manager. If we encounter cases when module's API is needed but UI should not be shown we will fix that case, e.g. split the functionality into multiple modules.
  • Install
  • Selecting regular modules will automatically install autoload modules
  • How to install eager modules? The visible regular modules cannot depend on it. It needs to be installed even if some of its dependencies are not met (they can be installed later).
  • not obvious how to know which eager modules to download & install when installing some new regular modules;
   probable semantics: install those eager modules which could not have been enabled before,
   but which could be enabled now
  • How to display autoloads if only autoloads are updated or if new autoloads are added? If they are part of multiple components? Display changes, not modules that should be downloaded. See a possible solution in Updates section.
  • Uninstall - uninstall all unused autoloads after uninstalling the regular modules
  • Current UC descriptor format does not distinguish autoload and eager modules
 (NBM format has this information, just UC XML is missing it)
  • Eager module semantics problem
  • Non-bug #22536 illustrates that
  the semantics of eager modules are consistent but impractical
  • It is fine for an eager module to bridge two regular modules
  • It is not wise for an eager module to bridge two autoload modules,
  or an autoload and a regular module
  (since its very existence would cause them to be enabled!)
  • It would not suffice to change the semantics of eager modules
  to not turn on any autoloads
  (since it may innocently use some API autoloads unrelated to what it is "bridging")
  • The root problem is that an eager module has two kinds of dependencies,
  which are not distinguished properly by the module system
  • Plain dependencies are just things it needs, e.g. random APIs, like any other module
  • Generally it will have two special dependencies on what it is bridging,
   e.g. form and i18n for i18n/form;
   unless these modules would have been enabled anyway, the eager module has no purpose
  • This problem has been long known, and is independent of Auto Update
  • OpenIDE-Module-Recommends cannot solve the issue
  (e.g. if form recommends an autoload i18n/form, that will still cause
  an autoload i18n to be enabled when it is otherwise unwanted)
  • (At least) two solutions are possible:
  • Change the module system to permit an eager module
   to declare its bridge dependencies explicitly
   and use this facility in all cases
  • Ensure that all eager modules bridge regular modules only;
   in some cases this would be the case anyway;
   in the remaining cases it is probably possible to make the module autoload,
   then introduce a new dummy eager module with dependencies on the original bridge
   as well as on two regular (feature) modules
  • Certain "core UI" modules, e.g. Project UI or Window System, should never be disabled
  • OK, leave them as regular modules (for the benefit of platform users)
  but omit the display category and never show them in Plugin Manager
  • Master Filesystem is not a good example;
  this should probably be an autoload which Filesystems API "recommends"

TODO List

  • Hide the Advanced mode from autoupdate UI
  • Hide autoload and eager modules
 (and perhaps regular modules with no display category, TBD)
  • Group modules, turn the rest to autoload
  • Change install/uninstall logic to account for eagers & autoloads

Plugin Portal and other update centers

Problem: generally we want to show as many useful plugins as possible, but sometimes dubious quality of some plugins, not tested by QA. Same problem exists for alpha/beta update centers and for update centers provided by third parties (for example various java.net projects).

Question: How to help the user to distinguish the modules that are less tested. Possible solutions:

  1. Mark all modules from plugin portal in UI (currently the source is in module description which does not allow to sort modules and it hard to find, maybe show the source in modules table?)
  • Advantage: easy to do
  • Disadvantage: no way to distinguish between modules in plugin portal, not all have the same level of quality/testing
  1. Provide a way for users to rate the pugins
  • Advantage: if we get enough ratings this is probably pretty reliable indication of the quality
  • Disadvantage: we may get none/few ratings (as plugins in other IDEs do), requires infrastructure work, highly subjective - no common level of expectations, i.e. hard to compare
  1. Provide more rules and more testing for plugins. Setup a checklist of plugin quality in terms of defects (e.g. no known P1s, no harm, etc.), performance, UI integration, basic attributes like license, documentation, etc. and actually test the plugins. There can be various level of
  • Advantage: more information for the user then option 1.
  • Disadvantage: obviously requires testing of plugins (work)

Simplified Updates UI

This part is only me suggestion, not strictly related to the above issues although it would provide a solution for the updates of hidden autoload modules.

I suggest that the updates tab would not show a list of modules, at least not by default. Showing a list of modules that will be updated without showing a list of actual changes in the new update is nearly useless. The user cannot determine if she wants these updates anyway. It would be enough to simply say that there are updates for the IDE and show the size of updates and provide a one click update of everything. In a future release a "details" view could be provided with a list of changes compiled by the module developers. Possibly also a list of modules, but that is a lower priority.

For 6.0 we will not be able to implement infrastructure for showing a list of changes so the Updates tab should be removed a replaced with a single button in other tabs or better yet by an icon in status bar or toolbar shown only when there are updates.

Note that IDEA does exacly this. The update UI is not related to the new/installed plugins UI. Many other software products provide a simple view of updates that does not list updated components, it lists changes and that only if the user asks for details.

Jesse's changelog suggestion

For archival purposes.

  1. Add a section to Info.xml such as
<changes>
<change href="..." date="..." version="1.5">some text...</change>
<!-- ... -->
</changes>

The Plugin Manager would let you browse all changes for modules you were updating, paying attention to versions. (Example: I update editor.nbm from 1.3 to 1.5. Only <change>s marked "1.4" or "1.5" are displayed. For new NBMs, no changes are displayed, just a note that something is new.) Identical <change>s from different modules would be coalesced. You could easily click hyperlinks for details.

  1. Some simple changelog.xml in project sources could hold the changelog for that module, to be included by <makenbm>.
  2. A target with a special task in projectized.xml could be run on any module, behaving as follows:
    1. Examine changelog.xml looking for the most recent changes recorded and when in CVS history those were recorded.
    2. Run a CVS rlog of the module between the date of the last update to changelog.xml and now. (Exclude submodules, e.g. html/editor/** ignored for module project html.) Might need to check both branch and trunk, in case a branch was created since the last update.
    3. Add an entry to changelog.xml for each commit message (possibly spanning several files) mentioning an issue number. Href would be to IZ, or to the URL mentioned in the issue if there is one (typically a wiki page or similar). Text would be IZ summary.
    4. Increment the spec version (as in nbbuild/build.xml#increment-spec-versions) if any changes were added. Use new spec version in <change>s, of course.
  3. Developer flow: do your regular development, just being diligent about including bug numbers in your commit messages, and making sure your IZ titles are informative. Whenever you decide you want to push updates from a module you are working on:
    1. Sanity-check your module.
    2. Run the target.
    3. Diff & commit manifest.mf & changelog.xml.
  4. For the trunk, we already have a near-continuous build of NBMs; we could pretty easily create bugfix branches and set up builds of them. No further changes to the NBM build process needed (beyond #2 above).
  5. QE could at its option sanity-check outgoing updated NBMs, or we could build to a staging server used by adventurous community members which would automatically push to a live server after a week's delay unless cancelled.

Review of proposal - Jirka Rechtacek

JR: Basic problem IMHO is a concept of mapping 'features' to regular modules and turn rest of modules to autoload modules is pushing UI level into module type's declaration. Concept of declaration of 'feature' in independent place separates out UI and module declaration. The proposal can lead to several technically problems:

  • need to extend Update Center DTD w/ information if a module is autoload or eager
  • most of modules must be switch to autoupdate/eager is looks too risky to me for now
  • hard to address which autoload module will be uninstall when regular module is uninstalled
  • hard to install eager module (no one regular module depends on them) from UC
  • TODO list should contain new implementation as well as change of UC DTD, change display category of all modules, discover possible performance regression, etc.
  • proposed UI changes should be reviewed by HIE

Above all - the proposal doesn't solve all issues noted above and it can lead to unexpected problems. In place of that general change I propose to keep solving detected problem w/o huge changes in current implementation and go to case by case basis and find possible solutions.

Review and Comments - Tonda Nebuzelsky

This proposal consists of several proposals which can be separated:

1) different granularity of grouping and reintroduction of category column

  • makes sense to me from the user's point of view because it allows users to install separately individual technology supports (e.g. CVS versus Subversion)
  • makes it impossible to track active users for a big component (e.g. Web and Java EE) if the user installed Basic IDE and then added the features of the big component via Plugin Manager; see also issue #107868.

2) different implementation of the way we show only some items in Plugin Manager

  • it is not possible to change arbitrarily majority of modules to autoloads!
  • eager modules depending only on autoloads would be immediately turned on by the module system
  • having most modules in the Platform as autoloads would not work for our RCP story - the developers building on top of Platform would have no control over the enable/disable state of the Platform's modules
  • AFAIK there is no query in module system which would tell you easily which modules depend on a disabled autoload, this would be a problem for the uninstallation action
  • if an update is available for a module which has been made autoload and should be hidden in the Plugin Manager, there would be no easy way of showing this update
  • Info.xml in NBM file and its copy in update center's catalog.xml does not contain any information about the type of the module(s) included in the NBM (normal versus autoload versus eager); but this proposal assumes you get this information from the Update Center catalog
  • current way of grouping (using the "feature" elements in catalog.xml) does not necessarily centralize the administration of the group definitions
  • users may be allowed to group their modules into features easily by:
  • enhancing APIsupport to create "feature" element for a module suite
  • enhancing Plugin Portal to allow users to specify the "feature" metadata for their modules
  • creating a how-to document so that users understand this new concept and are able to use it (this would be necessary for the autoload-hiding mechanism as well!)

3) simplification of Update tab up to the point where user cannot see individual updates

  • first I absolutely agree that the individual updates should show changes from the installed version, this information should be added to the Info.xml of NBMs
  • I am against hiding the details about individual updates because there is a strong group of users who insist on knowing what they are going to update in the installed product - my experience from Sustaining team is that especially users from large companies are very hesitant to install every update automatically, they want to apply only the fixes that they know are fixing the issues they encountered, updating broadly the whole system may be a stopper for them

Memo from the Past

I'd like to remind that similar proposal has been removed from trunk three years ago. It worked, but had its own problems. For example:

Moreover times has changed and we have new challenges:

  • Turning most of the modules to autoloads will negatively affect apisupport. Suites' Library selection dialog is not ready for "artificial" autoloads.

In short, solving a UI problem with complete architecture overhaul does not seem really appropriate after feature freeze.

Not logged in. Log in, Register

By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo