Requirements for NetBeans Packaging and Installation

Do not read this document! It is a working draft. It has not been reviewed. It is not even finished.

There are two areas in which the current NetBeans installation experience should be improved. First there is the technique used by the installer, i.e. HOW to install. For example we have a choice between a local or network installer, multiple installers for add-ons or one centralized installer, InstallShield based installer, linux packages, stadalone or web startable Java installer, installer that is part of NetBeans, a simple .zip (tar.gz) file etc. We have to deal with technical difficulties like installation of bundled software, platform and software dependencies, running the installer in a silent mode from command line or from another installer, etc.

The second area of requirements for installer is often overlooked. It is connected to product definition, i.e. WHAT to install. The modularity of NetBeans and independent versioning of its components poses a challenge in selecting the right granularity for packaging which would make sense both for the users - to select what parts of the tool they need - and for us - to select what part we can develop, test and release as a unit.

Current Situation

NetBeans 5.5 has six different installers - one for NetBeans and five the feature packs. This is an improvement over the previous state when each of the products bundled in it its own (more or less modified and/or dated) version of NetBeans. But the necesity to download and run multiple installers is still not optimal. The grouping of features is also not ideal (e.g. XML, SOA and BPEL togeather in one pack). On the positive side, the common parts are shared: NetBeans platform and IDE and there are no custom changes made in them by the packs. User can choose to install only the packs that they need. On Windows using Install Shield provides optimal user experience for installation from CD or from downloaded file, it is a de facto standard for a native plaform installer. On linux distributions an optimal installer (i.e. native to the platform) would be a linux package, which is currently missing.

Part of the installation story are the update centers. The functionality available from update centers is currently (1) modules which are optional, i.e. not important for all user and thus not included into standard distribution (note that the fact that this is an optional "add on" functionality is similar to the packs) or (2) modules which are developed independently of main schedule and finished after the release (again similar to packs) or (3) patches with bugfixes or (4) unstable experimental modules or (5) third party modules.

Impact of "One NetBeans"

Going forward we want to discontinue the packs and point products as the primary delivery of features and move all of the functionality into NetBeans[1]. This means that there will be one product - NetBeans - with common binary platform and all tools produced by Sun will be distributed as additional modules installed into this platform. In other words all the tools need to be able to coexist in the same environment on the same NetBeans platform. Some parts of out tools stack may be simply put into NetBeans standard distribution, but in general it does not mean that we would only produce one monolithical tool with all features always present in it. The feature set will still need to be customizable for several reasons:

• To allow users to select only features they want to use, thus reducing the complexity of menu options, etc.
• To improve performance - memory consumption, start up time, etc.
• To reduce download size and disk space required for installation

[Note:] There may still occasionally be products created for special target audiences, such as Creator or to be bundled with runtimes, such as JSE for JES or Sun Studio for Solaris. But the installers required for these are also quite special, there is likely no modularity required for them.

The notion of "One NetBeans" also does not mean that all features will always be developed and release synchronously at one time. NetBeans 5.5 was unique in this regard and was close (though not 100%) to release NetBeans, VWP, Enterprise pack, Mobility, C/C++ and Profiler in sync. Going forward there will be external drivers (such as releases of targetted runtimes), organizational reasons, scheduling issues, etc., which may make it impossible or inpractical to release all components in sync.

Download Size

Download size is often quoted as one of the main requirements for building a new installer. First, let's look at what is the real download size of NetBeans today (5.5), what is the download size of competitive products.

NetBeans 5.5 Download Size

A sum of all installers as of today: 42.1 + 22.5 + 41.5 + 7.8 + 9.0 + 87 + 41.8 = 251.7. This is increased a little bit by the installer overhead and a lot by inclusion of app server into enterprise pack. Real size of one installer for NetBeans plus all packs w/o application server would be approximately 140 MB, with !GlassFish approximately 200 MB.

Download Size Comparison

Why Does Size Matter?

• user complains
• unsuccesful downloads - errors or canceled by the user?
• comparison to other IDEs

• slow connection
  • what is an acceptable size for a slow link?
  • reliability of download (reconnect after failure, etc.)
• first time users
  • get something they can try quickly
• returning users
  • when you cross some threashold there is probably not much difference after that
• real data!

Addressing The Download Size Problem

In this analyses I am looking at options to actaully decrease the size of the bits for the same functionality, although there is probably some potential in that (the experiment of reducing size of NetBeans to fit into JDK, etc.). I am looking at how the installer can help to the address the problem of too large download size in many different ways:

• Install less - provide a smaller set of features, resulting in a smaller download size. This may be all that many users need for example in universities or to try NetBeans. Allow the user to get more features later.
• Selectively download only the features that the user wants to install. Allow the user to get more features later.
• Make the download succeed, even over a slow connection. Do not block the user from working while downloading. There appear to be many failed downloads - either the download really fails or the user stops it because they do not want to wait. A download manager which can reconnect after failure may help. Downloading from NetBeans while the user is still able to work would also help (this could be done even for auto update).
• Put NetBeans into other software such as linux distro, into JDK, into GlassFish, etc. so that no installation is required to get the initial bits. When installed check for updates and add on components.

Packaging

From architectural point of view the units are modules and clusters.

From the user perspective the existing units of packaging in NetBeans:

• NetBeans Platform
• NetBeans IDE
• modules (AU)
• packs

Modules are two small units of packaging, not all modules are interesting for the user. From user perspective it makes more sense to talk about features.

Packs are a little bit artificial grouppings of features. From user perspective it makes more sense to talk about feature sets - a set of modules, e.g. SOA, XML Tools, Java SE, Web Apps, etc. which covers a technology or an area of development. Clusters probably fit pretty closely into this although the primary motivation for clusters is architectural (unit of compatibility) not as an end user unit of functionality.

How Others Do It?

TBD

• competitive analyses, approach used by other modular products (ubuntu, eclipse, etc.)
  • eclipse - versioning hell, build you own IDE (and test it before using!), calisto
  • ubuntu - a lot in the distribution, more supported/tested, stable APIs?
  • oracle jdev - 3 distributions - java, j2ee, everything

Packaging Options

• NetBeans "light" - basic Java development, small download
• Simple way to install everything
• "standard distribution"?
• finer granularity customization of features to install

• if we cancel the packs and put everything into one bundle
  • all users do not necessarilly want to use (and therefore install) all features
  • the download size would be too big for users who do not want the whole IDE
  • users should not have to download parts that they do not want to install
• there is no small download for beginner users (e.g. universities)
• the packs are artificially put togeather, the user would prefer to select features at different (generally smaller) level
• solve the interaction between autoupdate and feature sets
• we still need an offline installer (from CD, from downloaded bits to multiple workstations, etc.)
• update netbeans core or features sets individually and maybe automatically, install new feature sets later easilly

Versionning

Different parts of NetBeans will be likely developed and released at different schedules. In past and mostly at present it has been problematic to develop some of the packs without making changes and bugfixes in NetBeans, althought in some cases (Buzz release of JSE) it was possible. Another issue is how to make all functionality existing in previous version of packs available for the latest NetBeans platform and IDE. It seems unacceptable to release new version of NetBeans and not have all features available (even if an older version) for it. The answer is in backward compatibility, integration testing, etc., but it is not trivial.

The installer should actually help user to manage versions and dependencies, like the auto update does or like the ubuntu installer does. It should not just passively demand that the user downloads and installs the correct versions, like the pack installers.

The Installer

• native installers (e.g. linux packages, Mac installer, InstallShield)
• install from a downloaded file (e.g. install shield, zip file)
• installer from media (CD/DVD/USB key)
• network installer (download only selected components, e.g. update center, [Netbeans.NBIScope]NBI], etc.)
• auto launch from web (e.g. JWS)
• update center

Interaction of native installers and installers of other software w/update functionality in NetBeans:

Proposed Solution

Install NetBeans only once. Upgrade and extend the features from then on. Similar model to Firefox or linux distributions.

Produce linux packages and try to get them into all distributions. Ability to upgrade is crutial for this. If the IDE can upgrade itself it does not matter that much if we distribute an older version. These installers should have packages for individual packs (feature sets) and suggest installation of runtimes (application servers, databases, etc.). Produce Mac OS X and windows native-looking installers.

For the first installation there will be several prepackaged options:

• Native installers
• JWS started from netbeans.org, selectively download components (NBI)
• Download an installer file (light, full)
• Install from media (full)

The user will be able to select components to install (Full/Custom).

The installer will be implemented in Java. It will download data over network or take from media or downloaded file. There will be one common UI that can be started from JWS, from command line (wrap into exe?) or from NetBeans. This UI will be used as the NetBeans module manager / updater.

The NBI provides capabilities to install third party components, registeration on platform, etc. which will be used for NetBeans module manager / updater.