Using Mercurial to work with NetBeans Sources in the IDE

This document is a simple primer for anyone who has no experience using Mercurial, and wants to get set up working with NB sources as quickly as possible. The steps below provide some useful background information, then show how to install and configure Hg, and provide a simple workflow use-case in the IDE.

Understanding the Hg workflow

The following diagram shows how you can expect to work with Hg in NetBeans IDE:

Understanding the structural representation

The following structural diagram indicates how the shared repo, local repo, and working directory are unique elements, and how the commands you use in the IDE can affect each of them. Note that when you use Fetch, you are traversing three paths in one action: pull, update (or merge then update), and commit.

This diagram is not to be confused with the workflow diagram above: specifically, after you commit changes to the local repo, your next step is to pull from the shared repo, not push!

Comparing Hg commands with CVS

Before stepping through the workflow example, understand how the following Hg terms correlate to CVS commands and match the options provided in the IDE's context-sensitive menu:

• hg clone: (Mercurial > Clone Other) There is no direct correlation to a CVS command. This is due to the differences in workflows between the two version control systems. A clone in Mercurial is your local copy of the entire repository. Before working with Hg, you must have a full snapshot of the shared repository on your system. As demonstrated below, your first step in the IDE is to clone an external repository.
• hg status: The status command (Mercurial > Status) allows you to compare your working directory against your local repository. This is similar to Show Changes in the IDE's CVS support, and opens the Mercurial versioning window in the bottom region of the IDE.
• hg commit: (Mercurial > Commit) Mercurial commit does not serve the same function as CVS commit. In CVS, you commit changes to the shared repository. In Hg, you are committing to your local repo only.
• hg out: (Mercurial > Show Out) There is no direct correlation to a CVS command. This command shows all outgoing changesets in your local repo, which would be sent on the next push.
• hg incoming: (Mercurial > Show Incoming) There is no direct correlation to a CVS command. This command allows you to view a list of incoming changesets from shared repository, which would be brought over on the next pull.
• hg fetch: (Mercurial > Fetch) Fetch is actually an extension built into the Hg client that allows you to perform a useful combination of commands, thus making the workflow more convenient. In CVS, you would perform an update prior to committing local changes. In Mercurial, you must pull changes from the shared repository into your local repo, then update your working directory with your local repo. Then, if any new changesets are pulled from the shared repository, Hg requires that you immediately merge them with your local repo. Fetch automatically performs the merge, then commits any differences between your working dir and the newly-updated local repo. Fetch, in effect, provides a lot of transparency, allowing you to minimize the need to execute low-level commands when stepping through the workflow.
• hg pull: The Hg pull command (Mercurial > Pull from - default) retrieves all changes (i.e., changesets) from the shared repository to your local repository. Note that in the IDE, when you perform a pull, the hg update command is automatically initiated immediately after. When these two commands are performed together, as is the case in the IDE, it is similar to Update in CVS.
• hg update: Mercurial update (Mercurial > Update) simply synchronizes all changesets in your local repository (i.e., the .hg file in the top-level dir of your local repo) with your working directory.
• hg merge: (Mercurial > Merge) Conceptually, a Merge in both CVS and Hg refers to the integration of changes from two or more file revisions into a new revision. However, in Hg a Merge takes on a slightly different role due to differences in the workflows: In Hg, each time you pull new changesets from the shared repo, the changes listed in these changesets must be integrated into your local repo. This requires a merge, and must occur regardless of whether you have specifically modified files that are affected by the changesets. Specifically, hg merge combines two separate changesets in a repository into a new changeset that describes how they combine.
• hg push: (Mercurial > Push to - default) Similar to a CVS commit. An Hg Push sends local changes to the shared repository. Specifically, it sends your local changesets (created after you committed them) to the shared repository. Therefore, it is necessary to commit any changes in your working directory to your local repo before pushing them to the shared repository.

Installation and preparations

Getting the Hg client software

Prebuilt, binary packages exist for most platforms: http://www.selenic.com/mercurial/wiki/index.cgi/BinaryPackages. If a package is not available for your platform, or if you want to download and build from sources, you will also need a Python interpreter and C compiler. See here for more details.

Hg client version 1.0.1 is recommended for working in the IDE (as of May 2008).

Integrating Hg into the IDE

It is recommended that you install NetBeans IDE 6.1, where Mercurial support is already included. There is also a downloadable module for NetBeans IDE 6.0, but it is not as mature or full-featured.

Note: Currently (20 Feb, 2008) the IDE presents you with a warning dialog stating that you are not using the correct client version:

If you are using 0.9.5 or later, you are fine. Click OK to continue.

Making sure the path to the Mercurial executable is set properly

If the IDE can't find the executable, a pop-up warning displays immediately after you've installed the plugin. The IDE searches for the Hg executable on your computer's $PATH. If installation did not add the executable to the $PATH variable, or if you installed to a unique location, you'll need to specify this manually in the IDE.

You can set the path in the IDE by choosing Tools > Options from the main menu, then Miscellaneous > Versioning > Mercurial. Enter the path in the text field. If you are using a Mac, try /usr/local/bin, otherwise look at this entry in Igor Minor's blog.

Configuring the Hg client

You can configure the Hg client directly through the IDE. Basic configuration requires that you specify your NetBeans username, and activate certain Hg extensions that allow you more convenience when working with Mercurial. Several Hg extensions are packaged with the client, namely Fetch.

To configure the Hg client, open the Mercurial Options window (Tools > Options from main menu. Then: Miscellaneous > Versioning > Mercurial).

• Make sure your NetBeans username is set in the Mercurial User Name text field.
• To enable Fetch, click the Manage Extensions button, then add 'fetch' to the Property Name text field and click Add. The Fetch extension is activated and listed in the bottom region of the editor. You do not need to add anything to the Property Value field because presumably extensions from the Hg client are already added to your $PATH.

Note: The Mercurial Options window allows you to make changes to the Hg configuration file located in your user directory. On Linux, this is the .hgrc}} file located your home directory. On Windows, this is the hgrc file located in: C:\Documents and Settings\my name\Application Data\Mercurial. Declarations to the Hg config file located in your user directory take precedence over declarations in the global configuration file (Mercurial.ini on Windows), which is located in your Hg installation directory.

• If you are using Windows, you also need to convert new lines to CRLF in your checkout. To do so, you should modify your global configuration file Mercurial.ini. See the note about configuring this here.

Creating a clone of NB sources repository

• In the IDE, choose Versioning > Mercurial > Clone Other from the main menu. The Clone External Repository wizard opens. In Repository URL, enter your netbeans.org username and password, followed by the URL to the repository, e.g.:

http://<myUsername>@hg.netbeans.org/main/

where <myUsername> is your NetBeans username.

• Click Next and in the second step enter the location on your computer where you want the local repository to be.

Note: If you are running Windows, be careful of the path length that you specify; i.e., C:\Documents and Settings\myName\My Documents\NetBeans\etc\etc can cause an unsuccessful clone due to very long file paths! Try using C:\ instead.

Also note that the default clone name is main, which matches the name of the NB shared repository. Click Finish to initiate the clone. Output is generated in the Output window, however be aware that the output does not list any files or directories that are being pulled from the remote repo. A clone can take anywhere between 1 and 5 hours to complete depending on connectivity, and requires approx. 800 MB of space. One way to see whether content is actually being pulled is to navigate to the local repo's location on your system's explorer and periodically refresh the size of folder to make sure it is increasing. The output for a successful clone looks similar to the following:

Mercurial Clone
---------------
adding changesets
adding manifests
adding file changes
...
75298 files updated, 0 files merged, 0 files removed, 0 files unresolved
The number of output lines is greater than 500; see message log for complete output
INFO Clone From: http://myName@hg.netbeans.org/main
INFO To:         C:\hg-repo\main

INFO: End of Clone

Note: Upon completing the clone, the IDE confronts you with a dialog that asks: "When working with this Clone, if you are doing any Merges or Reverts Mercurial may generate additional files with extensions, .orig, .chg and .rej. Would you like the IDE to automatically ignore them by adding these extensions to the clone's .hgignore file?" Click No. The expressions are already in the .hgignore file. Clicking Yes will duplicate those entries and ensure that you will have to commit and push a change to .hgignore, which will cause scorn and embarrassment.

• If you need to build a project from the repository, do so now. Otherwise, if you are just going to be editing static files (i.e., .html pages for online help), open the repository (or sub-directory thereof) in the Favorites window. The IDE scans the folder for the .hg file and Mercurial support is automatically enabled.

Configuring default protocols

Before you begin stepping through the workflow, you need to specify the default protocols for pulling changes from, and pushing changes to the shared repository. You can do this by opening the Hg Properties Editor (Mercurial > Properties). Note that the Properties Editor is associated with the clone you are working in. Therefore, changes you make in the Properties Editor are recorded in the hgrc file located in the .hg directory found in the top-level folder of your clone.

The Properties Editor should open with the default push and pull locations set to the URL you specified when cloning. Modify your default-push entry by add your NetBeans password and changing the protocol to https.

Important: Use http to pull, and https to push (as shown below). Also note that you only need your password for default-push:

Stepping through the workflow in the IDE

Editing files, viewing changes to your working directory

• To view changed files: Right-click a versioned file or folder and choose Mercurial > Status. This invokes the Hg Versioning window, which displays all changes in your working directory:

Committing changes to your local repository

• To commit changes to your local repository: The UI is exactly the same as for CVS: Right-click a file or folder that contains changes and choose Mercurial > Commit. You can also click the Commit icon from the Versioning window ().

You should see a similar message to the following in the Output window, indicating that your local repository has been updated to include changes from your working directory:

Mercurial Commit
----------------
Committing 3 files to repository for General Online Help:
        C:\hg-repo\main\usersguide\javahelp\org\netbeans\modules\usersguide\files\files_icons.html
        C:\hg-repo\main\usersguide\javahelp\org\netbeans\modules\usersguide\files\files_properties.html
        C:\hg-repo\main\usersguide\javahelp\org\netbeans\modules\usersguide\files\files_ignore-files.html
INFO: End of Commit

• Note: Because you maintain a copy of the entire repository on your system, the general practice when using Hg is to make multiple commits to your local repository, then, only when a given task is complete, you push your changes to the shared repo.

Viewing changesets created in your local repository

In order to verify what changesets were created from committing changes to your local repo, you can choose Mercurial > Show Out. This is a handy command because after you commit changes, all modified files return to their 'unchanged' state in the IDE (i.e., versioning badges disappear and color coding for file names returns to black).

The below capture indicates that 3 files have been committed within the usersguide/files folders of the user's local repository:

Performing a Fetch

Before you can push changes that you have committed locally to the shared repository, you need to bring your local repository in sync with the shared repository. You can most easily do this with the Fetch command. This is similar to performing an update prior to committing in CVS. However, it is more involved since an HG setup has three levels (shared repository, local repository, working directory) whereas CVS only has two (repository, working directory).

When you perform a Fetch, the Hg client actually performs three (or four) actions: pull, update and commit (or: pull, merge (if necessary), then update and commit the merge). If new changesets are received, Fetch will merge them into your working directory, then perform a commit in order to create the most recent changeset (i.e., tip) in your local repository.

Note: It is also possible to run pull, merge (if necessary), then update and commit sequence of commands separately. However, this is not recommended. Any advantage gained doing this is small and the risks of getting something wrong are significant. If you do opt to use these commands instead of fetch, remember that the IDE runs Pull with the -u option, which means that the update command follows the pull.

• To run fetch, right-click a versioned item and choose Mercurial > Fetch. If your Fetch is successful, you will receive output similar to the following in the Output window:

Mercurial Fetch
---------------
INFO: Running Fetch for C:\hg-repo\main
pulling from http://tgiunipero@hg.netbeans.org/main
searching for changes
adding changesets
adding manifests
adding file changes
added 11 changesets with 58 changes to 57 files (+1 heads)
merging with new head 68065:b04a33653113
58 files updated, 0 files merged, 0 files removed, 0 files unresolved
new changeset 68066:377b8c7d19fd merges remote changes with local
INFO: End of Mercurial Fetch

The output indicates that 11 changesets were pulled from the shared repository and merged into the local repo. (Note that the working directory is also updated to reflect changes, but this is not indicated in the output. A new changeset is created that merges the current state of the shared repository with that of the local repository. Note that the local ID of this changeset is 68066. This changeset is listed below in the push, as it is now the most current version of the repository (i.e., tip).

Note: If merge conflicts arise when you perform a Push or Fetch, you will need to sort these out manually in the Source Editor.

Important: If merge conflicts arise in files that you never intended to touch, take this as a sign of trouble. You should STOP and ask someone for HELP.

Pushing your changes to the shared repository

After you perform a successful Fetch, your local repository becomes in sync with the tip (i.e., most recent changeset in the repository). You are now ready to push any new changesets that you've created in your local repo to the shared repository. To do so, choose Versioning > Push to - default.

Note that all changes must be committed to your local repository prior to being pushed. If not, Mercurial prevents you from performing a Push.

• To push changes, choose Versioning > Push to - default. The output from a successful Push will list any changesets created (the first, i.e., oldest, is shown below):

Mercurial Push
--------------
INFO Changesets to push:
comparing with https://tgiunipero:***@hg.netbeans.org/main
searching for changes
changeset:   68054:6ab9355c2f2c
user:        tgiunipero@netbeans.org
date:        Tue Feb 19 02:40:53 2008 +0100
files:       usersguide/javahelp/org/netbeans/modules/usersguide/files/files_icons.html
usersguide/javahelp/org/netbeans/modules/usersguide/files/files_ignore-files.html
usersguide/javahelp/org/netbeans/modules/usersguide/files/files_properties.html
description:
minor fix.

The Push concludes, listing one more changeset, which is the one created by the merge from the previous Fetch. Finally, there is closing information indicating a successful push:

changeset:   68066:377b8c7d19fd
tag:         tip
parent:      68054:6ab9355c2f2c
parent:      68065:b04a33653113
user:        tgiunipero@netbeans.org
date:        Tue Feb 19 03:16:42 2008 +0100
description:
Automated merge with http://hg.netbeans.org/main

pushing to https://tgiunipero:***@hg.netbeans.org/main
searching for changes
adding changesets
adding manifests
adding file changes
added 2 changesets with 3 changes to 3 files
notify: sending 1 subscribers 1 changes
INFO Push To: https://tgiunipero:HzQm9CIu@hg.netbeans.org/main
INFO From:    C:\hg-repo\main

Useful notes and comments

• If your Fetch fails and includes an error about hgmerge not being found and/or info like "There are unresolved merges, you can redo the full merge using....", hgmerge needs to be set up. That might be done by getting a more recent binary (which does this for you) or by setting hgmerge manually, as indicated http://www.johnodonovan.net/blog/?p=6 or at http://www.selenic.com/mercurial/wiki/index.cgi/MergeProgram (scroll down to the Windows heading).

• USE FETCH. If you use Fetch, you will never be asked to manually commit merges (except for cases of merge conflicts). This will save you a lot of doubt and confusion about whether you are committing something you shouldn't be. If you are using Fetch and you are asked to manually commit changes to files you did not intend to change, you know for sure that something is wrong (STOP and ask for help). If you use pull/update/merge, you will be asked to manually commit merges, so it will be harder to tell if anything is really wrong.

• For the time being, DO NOT USE ROLLBACK OR REVERT MODIFICATIONS. If you revert to a previous changeset, make 100% sure it is a changeset you created locally. You are effectively reverting all changes in the NB Sources repo to that snapshot, so it is important to avoid the risk of inadvertently reverting changes made by others.

• When in doubt about what you are pushing to the shared repository, choose Mercurial > Show Out. This lists all outgoing changesets from your local repository to the shared repo. If there are many changes to files you are not familiar with, or in general the info looks suspicious, STOP and ask for help.

Troubleshooting

• You perform a Fetch and receive the following dialog:

...with the following output:

Mercurial Fetch
---------------
INFO: Running Fetch for C:\hg-repo\main
ERROR Command failed:
Command: [hg, --config, extensions.fetch=, fetch, --repository, C:\hg-repo\main]
Output: [abort: outstanding uncommitted changes]
INFO: End of Mercurial Fetch

This can happen if you have not committed all changes in your working directory. Try committing changes then retry Fetch.

• You perform a Fetch and receive the following dialog:

This can happen if someone has pushed to the shared repo since the time that you last fetched (or pulled). Try performing a Fetch again, then Push.

• You're running Windows and get the following output error when trying to push:

Mercurial Push
--------------
ERROR Command failed:
Command: [hg, outgoing, -v, --repository, F:\NBsource\main, https://username:****@hg.netbeans.org/main/]
Output: [comparing with https://username:***@hg.netbeans.org/main/, abort: error: error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol]
INFO: End of Mercurial Push

This can happen if MSIE is adding global proxy settings to your computer. Try opening MSIE, then: Tools > Internet Options, chose Connections tab, click LAN Settings button, and uncheck the "Use Proxy server for your LAN" box.

Other Resources

The instructions presented here are not exhaustive. If you are searching for more in-depth info, the following list shows some good places to start. The first two links are particularly useful:

selenic.com

• http://www.selenic.com/mercurial/wiki/index.cgi/UnderstandingMercurial: Mercurial is a "decentralized development model," which is why the structure and workflow is slightly different than that of CVS and Subversion. This page explains some basic concepts, as well as Hg versioning terminology like cloning, pushing and pulling, changesets.
• http://www.selenic.com/mercurial/wiki/index.cgi/CvsConcepts: The heading entitled "Workflow in Mercurial compared to CVS" is worth a quick read.
• http://www.selenic.com/mercurial/wiki/index.cgi/Tutorial: The official, hands-on, introductory tutorial.

wiki.netbeans.org

• http://wiki.netbeans.org/HgHowTos: Developers' NetBeans Mercurial How-To
• http://wiki.netbeans.org/HgTrainingMaterials: NetBeans + Mercurial Training
• http://wiki.netbeans.org/HgMigrationDocs: Documentation for Users of hg.netbeans.org

Attachments

HgWorkflow.png		48136 bytes
HgWorkflow2.png		90788 bytes
clientVersionConflict.png		8515 bytes
commandFailed.png		7076 bytes
commitIcon.png		593 bytes
confirmMergeAfterPull.png		14256 bytes
hg-prop-editor.png		9790 bytes
hgExtEditor.png		9451 bytes
hgMenu.png		13129 bytes
hgShowOut.png		7560 bytes
hgVersioningWin.png		8368 bytes
structuralDiagram.png		15202 bytes
unableToPush.png		6726 bytes