EnhancedConfigurationOfNBRCPProjects

Contents


Motivation and Goals

Possibilities of current NB RCP suite and application projects layout are limited, especially features supported directly in the IDE. Most important requirements that should be adressed are:

  1. Ability to reuse modules (suites) in several applications (and/or in other suites in more complicated setups) as:
    1. binary modules (NBMs)
    2. module projects
  2. Support such enhanced configuration in UI
  3. Ability to further extend deployed NB platform app by 3-rd party modules
    • This is actually possible in any scenario as built application can always be used as platform for further separated development.
  4. Retain backward compatibility, that is NB Module and Suite projects from 6.5 and earlier must work both in 7.0 IDE and harness.
  5. There are also other nice to have features:
    1. Establish a connection between J2SE (J2EE) projects and their wrapper NB module projects. Changes in the originating project could be propagated to the wrapper module automatically.
    2. Speed up build time, e.g. by possibility to separate a set (or a separate suite) of "stable" modules that are built manually when needed or by building dependent projects selectively

Use Cases

Not all use cases must be supported by chosen solution, they are here mainly to compare the solutions against each other.

UC1

Team develops RCP application and needs to use a common set of existing 3-rd party modules (e.g. NB library wrappers, but also regular modules).

The easiest and most maintainable solution is to have 3-rd party modules unpacked into platform-like structure of clusters and modules somewhere on filesystem and have the whole dir checked into VCS for sharing across team. To use the modules:

  1. user opens
    Properties / Libraries
    dialog of his RCP app project
  2. presses
    Add Cluster
    button and {Add cluster} dialog shows up
  3. user finds one of 3-rd party module clusters and optionally adds source and/or javadoc ZIP/folder
  4. pressing
    OK
    adds cluster into {Projects and Clusters} tree
  5. repeats step 2) for all 3-rd party clusters
Modules in RCP app now can "see" 3-rd party modules, can depend on them and
Build ZIP
, {Build JNLP}, etc. targets include the modules into resulting app.

UC2

As UC1 but they develop some of the modules themselves and want to be able to work with module projects as well so that they can be developed together with the application.

Standalone module project can be added to Libraries:

  1. user presses
    Add Project
  2. finds standalone module project in filechooser
  3. OK
    adds project into {Projects and Clusters} tree

Sources and Javadoc are taken from the project, ant tasks work as in UC1. Note that only Ant-based projects can be added this way, Maven does not build into clusters, on which this solution relies. Maven module projects must be built into NBMs, unpacked into cluster and added as in UC1.

UC3

As UC2 plus developer(s) develop multiple applications upon set of common modules._

Optionally they also want to deploy that set of modules separately, e.g. as a custom platform, as a useful RCP "library".

  • (tomwheeler): UC3 describes my case exactly. It is also very important to be able to build a complete cross-linked set of Javadoc (and source archives) for the custom platform as can currently be built using the nbbuild scripts. In other words, it should be just as easy for a developer to work with a custom platform as it is to work with the NetBeans platform, including features like source/javadoc associations (for both source modules and library wrapper modules), API changes documentation and architecture documentation.

Things work just like in UC2, user only adds suite project instead of module project. Added suite remains independent of RCP app and can be deployed separately.

UC4

As UC1-UC3, only user develops standalone module instead of RCP app/suite.

Everything works just like in UC1-UC3 respectively, classes from Libraries are of course not packed into resulting NBM. It is responsibility of the deployment to ensure that module NBM has all needed modules in the app it is deployed into.

Chosen Solution

Chosen solution is based on Jesse's proposal to use cluster.path property, as it solves both project and binary modules sharing simultaneously. It is also probably cleaner than use additional properties atop of existing ones. See Enhanced Configuration Of NB RCP Projects - Implementation and Enhanced Configuration Of NB RCP Projects - UI Spec for details.

Other considered solutions

Several Applications within One Suite Project

Current NB application project contains two distinct configurations: suite which is a list of module projects and pointer to platform and application consisting of branding and included platform clusters and modules. Application setup would be factored out into one or more separate app.${app.token}.properties file, "active" app to be built by harness would be set with app.token property in private.properties, IDE would show all configured apps simultaneously. Branding would also contain branding layer allowing to rearrange menus, toolbars, default window layout, etc. (this conflicts a bit with current initiative to use annotations for layer entries rather than editing layer.xml, as there is nothing to annotate in branding and UI would need to edit XML file anyway).

3-rd party module projects can be supported by generating chained platform from the suite, which the developers would use. "Generate custom platform" would get popup menu action.

Pros:

  • Can be done with minor or no changes to the harness, changes in IDE would amount mostly to UI changes.
  • Backward compatible

Cons:

  • Modules cannot actually be reused in different suites, only different applications
  • Used 3rd-party modules must be placed into platform (someone has to maintain it) or into the common suite as projects.
  • Cannot put modules into different clusters
  • Cannot reuse binary modules - either must be used together with suite chaining or suite project must be extended to allow NBMs

Non-linear Chaining of Suites

Allow non-linear inclusion of suites/platforms either by extending existing suite-chaining technique or in a different manner, e.g. one describe in Fabrizio's blog. Also probably doable without or with little changes to the harness.

Not yet investigated the possibilities, any comments welcome.

Full-fledged Multiple Suites and Applications Support

This would require splitting of current Suite project type into different Suite and Application project types as described in Several Applications within One Suite Project. Suites could then be configurable to "see" other suites so that their modules could depend on modules from other suites. Suite project would map 1:1 to concrete cluster in compiled form. Platform configuration would probably have to be system-wide in order to avoid clashes between different suites using different platforms.

Pros:

  • Clean and scalable solution to complicated project layouts.
  • NB Platform itself could be later (at least in theory) migrated into this project layout, simplifying code maintenance
  • Automatic migration from current suite and module projects to new ones would be possible, of course without any custom properties, Ant targets, etc.

Cons:

  • A lot of work, including rewriting a lot of code in apisupport.project and harness, very likely not achievable for 7.0
  • In order to keep backward compatibility both current and proposed versions of apisupport.projects and harness would have to be supported and maintained in parallel.

Other possibilities

  • Simply changing current 1:n relationship of suite and its modules to m:n (allowing to add one module project to multiple suites) would lead to a lot of confusing context-sensitive behavior (module could compile when open from one suite and have broken references when open from another), so it is probably not an option
  • Wade Chandler Does this approach really have anything more complicated than the current JSE or JEE projects when they do this? i.e. one can have multiple projects using the same JSE, EJB, or WEB project as a dependency
  • Richard Michalsky: This is not so much about dependency but rather about ownership (they are related though). The problem is that suite declares context or environment for NBM project, namely platform and set of other visible modules outside of platform (module then depend on some visible module or not, but cannot depend on any module not listed there). An example: User has suite S1 with modules A and B, where A depends on B, and suite S2 with module A alone. When A is open in context of S1, it compiles well. In context of S2 it has broken reference. Also context would have to be specified somehow when no suite is open or when building from command line. Such a problem does not arise with J2SE, visible jars are specified in NB Library Manager.
  • Wade Chandler OK, what I write here may belong in one of the other options, but it seems more of extending the existing functionality than a complete rewrite as I expect this will need to just be an option along with the way things work now for backward compatibility sake.

    JSE libraries aside, and noting in the case of JSE and JEE projects, those JARs don't have to reside in the library manager as individual JARs and projects can be individual separate dependencies, and on the topic of why the suite and modules are linked they way they are, I guess the main thing I have been thinking is why necessarily; I get the ownership/dependency things as they are now, but still ask why so singular.

    Currently to the RCP project support, modules are really only important in the context of a given suite except for the way standalone module projects work, but they are still linked to a static platform for dependencies. Now, it is true when a module, or module project for all purposes, declares things based on the layer it is then very specific to a suite, or at least possibly specific, and dependencies within a specific suite are certainly specific even if dependencies were to be shared between multiple suites (i.e. the end result is that in its final form a suite is unique and self contained...less chaining and clusters of course).

    So, yes I see the issue in that, but it seems the lock in to the context itself is mostly related to the project support less the final outcome. Technically it seems like if a module could declare dependencies upon modules directly then that would certainly solve many issues. This is exactly what happens at runtime, less the classloading and the runtime context, and of course indirectly through platforms and suites now. A .nbm declares its dependencies upon specific modules and not a platform. Those built modules may be copied around from platform to platform at will without the worry of the context as collectively those modules or dependencies are the context which a given module is concerned.

    I see that the suite is very handy as it is now. One has a suite and then it has all these modules linked to it. The utilities then have this central place to go for classes, resources, and the modules and loaders. However, that suite, from the purposes it stands now, less common library folders which place libraries into the suites/applications common classloader instead of a specific modules and the based platform, is the collective of all its modules except for the branding which the modules almost always don't care.

    Aside from those issue above with a suite, of which I think common library dependencies (see this blog entry) and platform issues need to certainly be resolved and worked into the IDE module projects settings, the only other real issue I see is building from the command line an individual module project and having its binaries placed into the correct output of the needed platform/suite build artifacts.

    From a global application packaging scheme, not depending on the individual modules themselves getting those build artifacts into the correct location, this is easily enough solved with the suite project in that it is responsible for where to place a built modules final artifacts building its final project and handling ownership, and this is the analogy of a JEE EAR project or in NB specifics a JSE Java Application project, and as well taking ownership of an instance of a module. Taking all these things into consideration, I think the following would be very workable, and maybe everyone can poke holes in it :-D.

    As it relates specifically to modules 1) Allow modules themselves to be able to depend on any number of suites, but not require one, done through the IDE could optionally add this module as a dependency to the suite (ownership too), 2) Allow a module to be able to depend on any number of modules directly, whether this be by project, .nbm, or suite and all its dependencies (though suites are handled by 1), 3) Allow a module to directly depend on a JAR file which will be packaged inside the module to be deployed at install time into the platform/application exposed through the module, 4) Allow a JAR dependency to be marked as a global library which will not make the packages available for export as these will be used as noted here and the cp.exra bits should be added automatically. 5) Allow a module to directly depend on a platform or any number of them as with suites since this is just a binary artifact of a suite being built into its final application. 6) Allow JAR dependencies to be packaged as library wrappers. (this may be something that is only allowed for suites though).

    When multiple suites or platforms are used a couple things could happen: 1) force the only allowed dependencies to be those available in any/all suites and platforms or 2) at build time only place the module into a built suites artifacts if and only if its dependencies can be met there. Note: Module projects can, but are not required, have a list of suites in which their artifacts may be placed. This takes care of building individual module projects from the command line and their build artifacts being updated in suites depending on them. This then becomes configurable, and too it would be good if the particular suite project wasn't where it would need to be, as is the case with source control systems and checkouts at times, then that portion wouldn't fail the build, but it just wouldn't occur...probably make that a configurable project property whether to fail or not as that would be something good for continuous integration.

    As it relates to suites, 1) As now a suite depends on a platform. 2) A suite has dependencies on multiple modules which are pulled into the built artifacts unless the module is doing that on its own (project needs to search the properties of dependencies to know if it or the dependency is handling the copying of the dependencies build artifacts for that project) 3) A suite can depend upon any number of other suites and build them as clusters or whatever is needed (would transitive dependencies be needed for this case?). 4) A suite can depend upon any number of JAR files which get added to the final result as common/shared libraries as mentioned in this blog entry and this is what copies them there (would be used in concert with module project support from number 4 above). 5) See 6 above in modules section. Maybe do that here, allow JAR dependencies to automatically create library wrappers. This may not be as important as just having more customizable library wrapper projects though.

    And, I believe that would take care of all the general issues though of course there are details to be worked out, but this is essentially what Fabrizio and I have been discussing for an idea we were working on before finding out about this one though our concepts were to try to work around the limitations of the current project support without necessarily subverting it through the use of platform building.
  • Wade Chandler Too, as it relates to multiple suites, it seems then that a module needs project run profiles like other projects have. In these profiles the different suites can be setup to be run different ways. Too, it would be easy enough, along with run profiles, for any given suite a module depends upon to be selectable from the run/debug drop down as a profile without it needing to be specifically setup as a run profile because that is essentially what it is already.
  • Wade Chandler After more thought being given to the idea of a module or suites JAR dependencies and library wrapper projects, I now believe it makes more sense to have better or more flexible library wrapper projects versus an option to convert JAR library dependencies into library wrappers. That would be more straight forward.

    I still believe allowing JAR dependencies of a module to be marked as global, meaning something used with cp.extra in the module and installed into the system class loader at runtime by one of the suite projects, or marked to be included in the module which then means its API/package list will then be in the table of APIs to expose to dependent modules is a good idea.
  • Maven can handle hierarchical project layout automatically and is more suitable for large projects. However maven NBM projects are not in stable state yet, missing some important parts like building ZIP, JNLP, etc. and migration from existing projects is manual and non-trivial.

Jesse's proposal for cluster paths

There are strong performance reasons to want to work with unpacked cluster structures. Mainly, you want to be able to run the application with a --clusters list that directly includes all the dependent clusters, without first copying them to some other location, or unzipping anything. Export to NBMs is fine but this can be done easily with some tool/target not used in the core workflow, based on <makenbm> (with the complication that bits of Info.xml such as license cannot be determined from the cluster dir alone - maybe this could be fixed). Same for import of NBMs, which is almost as trivial as <unzip> except that you also need to create update_tracking/*.xml files.

I think you can get what you want from a generalization of the "platform" reference for a suite or standalone module project, i.e. the contents of platform.properties: you would switch to having a "clusterpath" for a project, where path entries would be individual cluster directories located anywhere on disk. Some of those clusters could be build/cluster/ subdirectories of suite projects; others could be build/cluster/ subdirectories of standalone module projects (containing just a single module); others would be subdirectories of platform installations; others could be pulled out of VCS in binary form when produced by another team - it would not matter.

You could keep the current ability to exclude named modules from the app (but see next section). The current ability to include just named clusters (from the platform) in the app would make less sense if you had a clusterpath. Probably you would prefer to just directly list those clusters you wanted in your clusterpath, with defined "platforms" being nothing more than convenient variables giving a parent dir, i.e.

cluster.path=\
    ${netbeans.platform.release65.dir}/platform9:\
    ${netbeans.platform.release65.dir}/ide10:\
    ../other-suite/build/cluster:\
    ../one-random-module/build/cluster:\
    ../some-other-unpacked-cluster-from-wherever

As with the --clusters CLI option, order is significant, so you could override some modules from a cluster with customized versions in an earlier cluster.

Complications might include:

  1. Compatibility code would be needed to interpret the old nbplatform.active and enabled.clusters properties, which would be treated as a clusterpath containing just clusters from the single named platform.
  2. We would need to do something about the current practice of putting a version number in NB IDE cluster names, which I think has always been a bad idea as it makes upgrading unnecessarily complicated. If that convention is not changed, we could use an analogy to the current hack used for enabled.clusters: if a clusterpath entry does not exist, but some dir with the requested name just followed by
    [09.]+
    does exist, then use that dir instead.

(Note: currently you can elect to use the harness from the active platform rather than the current IDE's default harness. We would still want that ability; the details of how the selection of the harness is defined might be slightly different.)

I would suggest deferring formal suite-to-suite build dependencies until a later stage. All that really buys you is the ability to build suite A automatically when suite B depending on A is built, which is sometimes convenient but probably not crucial. (Even without that, if B includes A/build/cluster/ in its cluster.path, and projects correctly implement SourceForBinaryQuery, then you can include A in B and source associations will work fine.) To implement build dependencies as an option, it would suffice to have an Ant task in the harness which runs <subant> on suite/module projects known to be associated with clusterpath entries. There is the minor complication that the code which scans a binary cluster for a list of module JARs would also need to know about clusters owned by suite or standalone module projects and automatically switch to a source module list scan in this case.

Jesse's (more or less unrelated) proposal for dependency-based module includes

Moved to DependencyBasedModuleIncludes.

Resources

  • Related issues: Issue 152960 (umbrella issue for the feature), 83088 (target platform to include ext. clusters), 125522 (programmatic installation of NBMs into platform), 124910 (customized platform distribution), 61227 (broken NBM refs), 70894 (let lib wrapper use built JAR from J2SE prj)
  • UI: 70883 (Suite branding UI), 67868 (UI for jar-excludes), 61681 (UI for multiple JARs in wrapper), 122821 (transitively include missing modules into plaf.), 135247 (showing runtime deps), 148986 (icon branding UI), 64618 (VM args in properties dialog), 147525 (props. dlg. for modules in suite cfg)
  • other info: HowToReuseModules, PXCustomPlatformProject
Not logged in. Log in, Register

By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo