---

Remote artifact repository index utilization #134677

UI feature possibilities

• hint to find binaries based on classname (requires: lucene)
• create a library along with transitive dependencies inclusion..(requires: lucene, maven)
• identify binaries based on SHA/MD5 - allows to utilize additional maven metadata
  • find sources and javadoc jars (identify classes in stacktrace, suggest to download sources?) (requires: lucene, maybe maven for download)
  • checkout sources (how? using Netbeans SCM integration or Maven SCM?) (requires: lucene, maven+maven scm)
  • view project page (requires: lucene)
  • file bugs (requires: lucene)
  • suggest upgrade to later version of jar (requires: lucene, if transitivity gets into play, also maven)
• utilize artifact identification for use by ant ivy as dependency manager. (Code completion in ivy.xml file) (requires: lucene)
• download the binaries, source, javadoc found, utilize Maven Wagon for it. (requres: lucene, maven, either wagon only or all package)

Technical background

central repository index == a lucene index of 18M in zipped form. (cca 40M unzipped)

• the index is created by Nexus, but should be possible to use directly.
  • alternatively can be created by a command line tool. I've done that at deadlock
  • can only index maven repository content, not arbitrary jars..
• how to distribute to users?
  • in a netbeans module?
  • direct download from repo1.maven.org
  • how to update data? currently non incremental, this features is supposedly planned.
• we'll need some kind of management UI to allow additional indices to be plugged in, quite common for big enterprise teams to have their own repositories. Most ui should work across all known repositories?
• shall we utilize local maven repository as well if present?

Alternate solutions

• binaries created by maven have some basic metadata written in META-INF/maven, we could check that in library jars and elsewhere.
  • pros: will figure the Maven ID of such binary, the rest is to be retrieved from remote repository anyway..
  • con: what features will it enable?
• remote calls to Maven repository managers (there's a bunch - a (maybe biased) comparison
  • pro: most uptodate data
  • pro: no 18M download
  • con: every query requires internet
  • con: none is publicly running against central maven repository (not speaking about 24/7 at all)
  • con: each has probably different means of accessing the data, if even possible

Reuse of nexus CLI tool for accessing the index.

• downloadable CLI tool: http://nexus.sonatype.org/downloads/
• more information about usage: http://docs.codehaus.org/display/M2ECLIPSE/Nexus+indexer, the bits can be used not only for creating the index but also for reading and retrieving it.
• pros:
  • is capable of downloading the index from remote locations when updated on remote site.
  • allows merging multiple indices together
  • provides utility methods for accessing the data.
• cons:
  • licensing
  • big binary - 3MB
• licensing:
  • the nexus stuff is EPL, however it's dragging quite a bit of dependencies along with it. I've filed it as an issue with nexus http://issues.sonatype.org/browse/NEXUS-23 The public svn repository is at http://svn.sonatype.org/nexus/
  • lots of apache stuff.. some maven bits, lucene, jakarta commons.
  • plexus stuff from codehaus http://plexus.codehaus.org is apache + mit license http://www.ohloh.net/projects/plexus/analyses/latest
  • aspectj runtime is Eclipse public license according to ohloh
  • retrotranslator http://retrotranslator.sourceforge.net/#license BSD like license, uses ASM (BSD like http://www.ohloh.net/projects/5419/analyses/latest) and backport (public domain )

Repository content stats

Stats retrieved by a simple tool from index created on Apr 25

Number of documents in index: 38031

• jar 28631
  • 39 percent have sources attached
  • 25 percent have javadoc attached

• Maven related
  • maven-archetype 235
  • maven-plugin 1222
  • pom 3850

• J2ee related
  • rar 198
  • war 1418
  • mar 22
  • ear 50
  • car 682
  • ejb 36
  • sar 12

• soa related
  • jbi-service-assembly 28
  • jbi-component 298
  • jbi-shared-library 8
  • jbi-service-unit 76

• others
  • so 18
  • bundle 691
  • nbm 71

UI Design

Usecases

• User adds a jar to project or library.
• User wants to update the library or jar to more recent binaries.
• User wants to learn know more about a given artifact he has in the project.
• User adds a java class reference to his sources and it cannot be found in the project's classpath

UI

Most of the UI functionality shall be provided as hints in the dialogs and panels or as popup menu actions. The hints should have the word "Hint:" or "Tip:" prepended, possibly with a special icon as well.

Hint in project customizer:

and in Library definition.

Hints include:

• found javadoc/sources
• upgrade to later version of a given artifact.

popup menu of Libraries subnode (representing a single jar or library).

• Add an action that will show a dialog with additional information we retrieved about the project.
  • link to project page
  • link to bug tracking system
  • link to browsing sources
    • From here invoke additional dialog to checkout the sources.
  • license of the binaries
  • the SHA1 checksum
  • etc..

more TBD

Attachments