General setup and view
See Hudson Builders in the Services tab.
Connect to one or more Hudson servers by giving the URL and a display label. (Jenkins will also work though it is not tested or formally supported.)
Set a refresh interval in minutes. If zero, after a restart the server will initially be left unconnected - you can manually "synchronize" when you want to. You can always manually synchronized rather than waiting. If synchronizing hangs (server not responding, ...) the task can be canceled and resynchronization should be suppressed for the rest of the IDE session.
All jobs in the default view are shown; you can select an alternate view if you like. Each job shows its current workspace, and a list of builds (shown with their status color - stable, unstable, failed, aborted - plus a badge if currently running). Completed builds show a list of artifacts; builds of native Maven jobs show a list of modules, each of which may have its own artifacts.
Most nodes permit you to open the corresponding URL in an external web browser for more information and control. Job nodes let you open the config.xml (Hudson's persisted job configuration file) in the IDE's editor as a convenience, though currently this is read-only.
Jobs may be started manually from the context menu.
You may use the Log In... action to supply credentials to a controlled server. For publicly browsable servers it is unnecessary - if and when you take some action which requires a login, such as creating or starting a job, you will be prompted automatically. Less commonly a server may be configured to simply hide some jobs when not logged in, in which case you need to do so explicitly. Credentials are remembered using the secure keyring, though currently you are not logged in automatically after restarting the IDE.
Supported authentication types (as of 7.2):
- username/password in Hudson's own database (the default mode)
- servlet container authentication
- API token (currently Jenkins only)
If a build on a registered server is not stable, a popup notification appears in the IDE. If it failed, you are offered a chance to see the log (#Build log browsing). If it succeeded but was unstable, you are offered a chance to see the test failures (#Failed test display).
Notifications are displayed only for jobs that are "watched". All jobs are initially watched on a manually registered server; for a transient server registered according to a project association which specifies a job (#Project/job associations), just that job is watched. A notification will offer to stop watching a job in case it is irrelevant to you. You can always turn watching on or off directly in the Properties of the job (should probably be made into a checkbox menu item).
Once you have seen a notification for a particular failed/unstable build, it will not be displayed again (even after IDE restart), but if a new build fails a fresh notification will appear (only one at a time for a given job).
Changelogs and diffs
A context menu item on a build lets you see its changelog, currently displayed in a simple text format in the Output Window. Subversion, Git, and Mercurial are currently supported.
Per-file changes are hyperlinked where possible, displaying the change in the IDE's diff viewer (typically with syntax coloring); this works only for SCM setups where the versioned file contents can be retrieved in a straightforward way over HTTP.
Currently the changeset ID or revision number is not displayed, and there is no hyperlink to an online changeset URL, but these things would be possible where supported by the corresponding Hudson SCM plugin.
Assuming a job has a workspace, it is displayed in a tree view beneath the job node. (Only the most recently used workspace is shown, for jobs running on multiple nodes with independent workspaces.)
Subdirectories and files can be browsed just like in the Files or Favorites tabs, and files can even be opened (read-only) in the IDE's editor with syntax coloring. When the server is resynchronized, changes to the workspace should be reflected in the IDE's view. ZIP/JAR files can be expanded to see entries, you can copy and paste files into a project source tree, etc.
Builds which created artifacts also show these using the same kind of viewer, so you can easily verify that file contents are correct, copy a JAR into Favorites, etc.
All such file operations run directly against the remote server, so they can be slow if the server is heavily loaded, but no more is transferred over the network than necessary.
Failed test display
Unit test failures from a build can be displayed in the IDE's Test Results window. (Only failed tests are shown, since there might be thousands of passing tests and it would be inefficient to retrieve information about all of them.) Standard output and error are displayed where available, and this output together with the stack trace is also displayed in the Output window for ease of scrolling, searching, and copying.
Java stack trace lines are hyperlinked (in both Test Results and Output). Currently these jump to the specified line in your own source checkout (you must have the project open), which might not be correct in case you have been changing the file since the revision that Hudson was building (or if you have an older checkout). Ideally the IDE would compare file content from the built revision to your local checkout, and if there were a discrepancy potentially affecting the line number, warn you; or try to adjust the line to the current position; or offer to open a read-only copy of the historical file version.
Build log browsing
A context menu item allows you to open a build's console log in the Output window. If the build is still running, new content is automatically appended every few seconds. (This is pull not push, but if you close the tab, the IDE stops listening for new content.)
Lines that appear to contain references to files in the Hudson workspace are hyperlinked so as to open that file directly from the workspace (as in #Workspace/artifact browsing), assuming that the file is still there. Where the job's SCM configuration provides sufficient information (currently just Subversion supported), the corresponding file in local project sources is opened instead; as mentioned in #Failed test display, this can lead to line numbers being skewed.
Java stack trace lines are hyperlinked to local sources as for #Failed test display.
URLs are also hyperlinked.
The Quick Search toolbar may be used to jump to servers or jobs by name.
Automatic setup of CI
Can take an IDE project, or project tree, and set up a new CI job for it. (Team > Create Build Job...)
For an IDE-managed Ant project (e.g. j2seproject), this means a "freeform" job (i.e. job's config.xml lists every build step and file location explicitly) invoking build.xml with targets like jar/test/javadoc, collecting dist/*.jar, test result artifacts, and/or Javadoc.
For a freeform or automatic Ant project, this is likely impossible - the user would have to set everything up themselves anyway.
For a Maven project (possibly consisting of multiple modules), this is handled by using Hudson's native Maven 2+ support. (Note: Hudson after the Jenkins split supposedly deprecates this job type, but it still seems to work fine, and is much more convenient for the IDE's purposes.)
PHP projects are also supported, using a semistandard PHP setup system for Hudson.
The job can only be set up if the IDE can determine the location of the version control repository used by the job, based on metadata in your current project's checkout. The contents of the local project tree are ignored thereafter - to trigger new builds, you need to commit (and, if applicable, push) changes. The job config will include a simple default SCM configuration, polling for changes at some interval.
Currently Subversion (pre-1.7), Git, and Mercurial are supported for this purpose. For the DVCS, it is supported to start from a project repository with no remote server location, if you are running the Hudson server on localhost (i.e. solo development or just giving a demo); then the job config will use a file-protocol URL, and just committing locally will trigger builds.
When the IDE creates a job from a Maven project, it will also populate the standard ciManagement field in pom.xml, so that if the project is opened again by a NetBeans user this Hudson server will automatically be registered (with this job being "watched"). The IDE will also recognize ciManagement (of type hudson or jenkins) defined manually in any POM, which well-maintained projects may already have.
For other project types (Ant/PHP), the server link is added to project metadata too.
Servers registered as a result of a project being opened are marked as "transient" (a label is shown in the UI) and are unregistered when the project is closed. You may choose to persist the server configuration so it will be shown even after closing the project.
You can also manually associate a job with a specific project, useful if you set up the job manually in the web UI.
Static analysis result display
If a job is using the Static Analysis plugin suite, when the IDE's Action Items window is opened (or refreshed) with a scope containing projects associated with that job, any errors or warnings reported by supported plugins (such as FindBugs) are displayed. Apache Ant's nightly build is an example of such a job (currently NetBeans builders do not seem to run these plugins).
Hyperlinks are to local project sources thought to correspond to the workspace path. (As with other features, there is the possibility of version skew: a warning reported by the server could refer to a different version of the code than what is checked out locally.) Where a local file cannot be found, the workspace file is opened instead.
There is currently no special gesture for resolving such items, other than fixing the reported problem or inserting tool-specific warning suppression annotations, upon which the next build will refresh the list.
The IDE includes rudimentary support for developing Hudson plugins (<packaging>hpi</packaging>) beyond what you would get with generic Maven development (such as the archetype):
- The project gets a Hudson icon.
- Run/Debug Project are by default bound to the right goals.
- The web browser is opened on the home page when Jetty starts.
- An XML catalog is provided for Stapler Jelly tags (TBD if this still works).
Originally a third-party plugin by Michal Mocnak (mmocnak).
Feature set as of original integration into NB official distro: NewAndNoteworthyMilestone3NB67#HudsonIntegration
Original HudsonInNetBeansUI spec.
ContinuousIntegrationHOWTO is obsolete (relies on Kenai).
In hudson* modules.
How to develop
Build the IDE with at least
Now create a project group containing at least these NB modules:
hudson hudson.ant hudson.git hudson.maven hudson.mercurial hudson.subversion hudson.tasklist
and optionally also:
hudson.php java.helpset nbbuild
Commit to core-main; NB-Core-Build will run tests for those modules which have any. Set up hudson*/* notifications in .hgmail.
Running in devel mode
Now run Debug Project on Hudson. This debug target is set up to enable verbose console logging in Hudson-related modules. (It is also suitable for setting as "main project" since its debug target rebuilds all the submodules too.)
Originally "Try Hudson on Localhost" was the easiest way to get started, though this is currently disabled (#214210), so just download the latest WAR and java -jar it (will open at http://localhost:8080/ by default).
Install the Mercurial plugin from the update center if you want to test this integration (though the version available on the Hudson UC is quite old; the Jenkins UC has the latest). Subversion and Git plugins should be installed by default.
For testing projects using Ant or Maven, you will need to configure at least one installation of the relevant tool: go to Manage Hudson, and ask to automatically install the latest version from Apache (beware that this list is sometimes out of date or even broken).
Various pieces of code have unit tests, which should all be stable.
There are no functional tests currently that would confirm that interaction with a particular version of Hudson using the remote API actually works. This would be possible if a hudson.war could be kept in ExternalBinaries and launched somehow; might need other tools like HttpUnit etc.
Could alternately create a mock for ConnectionBuilder to test its usages, such as parsing of unusual server responses. Would be fast and reliable enough to include in unit tests, but would not turn up problems caused by changes in response format in new Hudson releases, which is often the cause of reported bugs.
Working with non-IDE sources
Useful to have checkouts of:
- git://git.eclipse.org/gitroot/hudson/org.eclipse.hudson.core.git (3.x)
- git://github.com/hudson/hudson.git (2.x)
- git://github.com/jenkinsci/findbugs-plugin.git (etc.)
- https://bitbucket.org/jlahoda/jackpot30 (subdir hudson.indexerinstaller; experimental)
To build and run Hudson from sources, just open the top aggregator project and Build Project (probably want to suppress tests for non-test-related executions in Maven settings). Then open the hudson-war module and Run Project, which should automatically call hudson-dev:run. Theoretically you can make incremental changes to Java and Jetty will automatically restart, though this seems to break things. Can edit Jelly XML (HTML templates) and the results should be live upon save.
To build and run plugins from sources, just open them and Run Project. This will run the plugin using Jetty against its configured baseline Hudson (or Jenkins) version. Or, just Build Project to create a *.hpi, and then install this manually into a running Hudson instance (see Manage Plugins / Advanced).
Hudson uses the Stapler library to manage RESTful URLs, and one Stapler feature is that HTML URLs can be turned into web services by appending /api/$format where supported formats are xml and json. This is called the remote API or remote access API. (Not to be confused with the remoting API, which is a Java RPC library akin to RMI used mainly to communicate between Hudson master and slaves.)
NetBeans uses /api/xml for most of its retrieval of information from the server, including modeling of jobs, views, builds, Maven modules, and test results. (Some special tasks use ad-hoc URLs: loading job XML configuration, creating jobs, starting jobs, performing authentication, viewing the console, browsing the workspace or artifacts.)
Stapler query parameters
The Stapler-based API URLs can be refined using various query parameters. Just looking at $base/api/ in a web browser will give some more details and a link to the wiki documentation.
depth=n (default 1) controls what nested objects are displayed. An @ExportedBean has various @Exported attributes, which can be other beans in addition to primitive values; the depth parameter controls this recursion, in conjunction with the visibility setting of the attribute. It is tricky to predict from source code what will actually be displayed - easiest is just to play with different depth settings and see what appears.
xpath=... (xml format only) lets you select only certain parts of the document to return to the client. In conjunction with depth, you can then create a more detailed document, but throw out the unnecessary parts - the server will still waste time computing them but at least they will not go over the network.
tree=... is a more recent parameter, added expressly to support efficient communication with clients like NetBeans, which supersedes depth and xpath (and works also with json, which is a bit more compact). Here the client explicitly requests certain attributes at various levels of the tree, and the server produces just that information, which is both faster and easier to use (since you do not need to know what should be excluded).
NetBeans uses /api/xml?tree=... wherever possible, falling back to /api/xml?depth=...&xpath=... where necessary (an older server, or selecting just failures from testReport).
The IDE attempts to defer loading large documents until needed: when expanding a server node, a quick list of jobs and their statuses is returned, whereas the potentially much larger list of builds is retrieved only if and when a job node is expanded. On the other hand, an attempt is made to retrieve as much information as is expected to be needed immediately in one HTTP request, which would probably not be possible using #Jersey.
The utility class ConnectionBuilder is used to manage all HTTP connections to the server, whether using the remote API or something else. The most important detail it handles is authentication. If a response has a 403 FORBIDDEN code, by default ConnectionBuilder will automatically try to log you in and retry, as mentioned in #Authentication. A pluggable ConnectionAuthenticator is used for this.
The normal impl is ServletConnectionAuthenticator, which in turn delegates to a PasswordAuthorizer to get a login. (There is currently only a simple form implementation, but there used to be a special impl for kenai.com, and it is easy to write one that ties into some other kind of SSO system.) This then goes through a delicate dance of pretending to log in using the login form and then accepting the resulting redirect or two, especially tricky using Tomcat container-based authentication. An added complication is that some servers are configured to block CSRF attacks using "crumbs" which must also be requested and subsequently sent back.
APITokenConnectionAuthenticator is also available, currently only for Jenkins (HUDSON-9017), which offers a simpler and more reliable mechanism using HTTP BASIC authentication.
Newer versions of Hudson also appear to offer a web service interface using Jersey, but NetBeans does not use it; unknown if there is any major performance or robustness advantage, and the existing remote API would still be wanted to support Jenkins and older Hudson releases.
The hudson module has a variety of classes in the api and spi packages which are used from satellite modules. Notably:
HudsonInstance, HudsonJob, etc. provide a simple model of registered servers and their contents.
ConnectionBuilder should be used for all network connections, especially to support #API Authentication (which also described ConnectionAuthenticator and PasswordAuthorizer).
HudsonLogger permits hyperlinking to be extended by plugins. (Currently JavaHudsonLogger is in the main module since it does not actually use any APIs from the java cluster.)
ProjectHudsonProvider permits pluggable #Project/job associations.
ProjectHudsonJobCreatorFactory permits pluggable #Automatic setup of CI.
HudsonSCM manages interactions with version control. It can detect whether a local project is versioned, and if so decide how a corresponding Hudson job should be configured; load the changelog for a build; and attempt to convert between workspace paths and local project paths by matching up version control metadata (currently implemented only for Subversion).
Other Hudson IDE plugins: