NetBeans + Mercurial Training

What is Mercurial?

• One of three popular distributed version control systems (DVCS)
  • Git and Bzr are the other widely used choices
• Also known as Hg (hydrargyrum or mercury)
• Hosted on selenic.com/mercurial
• Initially written by Matt Mackall at Selenic Consulting
• Vigorous open-source community (under GPL)
  • Several patches per day posted, reviewed, sometimes accepted

What does it do?

• Manages changes to a file tree called a repository
• Allows you to record and examine changes to files
• Manages interaction with other developers when you are ready
• Superficial feel similar to CVS
• Underlying model closer to TeamWare

Basic UI

• Basic command structure similar to CVS; one executable: hg
  • a few global options, lots of options on subcommands, abbreviations
  • hg -R nb_all loc -f \*.java
  • hg --repository=nb_all locate --fullpath \*.java
• Command defaults and lots of other options can be configured
• Interactive help: hg help, hg help locate

Distributed development

• Each repository is self-contained: history in .hg subdir
  • fully offline operation possible
• You can clone a remote repository to work on it locally
  • or clone a local repository to make a temporary work area
• You can pull changes from a source you trust
• You can push changes to a destination that trusts you
• Changes made in parallel can be merged and thus resolved
• Branches can be isolated physically
  • each repository is effectively a branch in Hg!

Terminology (1 / 2)

• Working directory (or checkout) has your editable files
  • a repository may also have no checkout, just .hg
• Revision is a snapshot of the source tree
  • no special labels for revisions of individual files
• SHA-1 hashes identify revisions uniquely and globally
  • 40 hex digits, but usually abbreviated to 12
  • Sequence numbers can be used as shortcuts inside a single clone only
• a tag is just a label for a revision

Terminology (2 / 2)

• Changeset is the difference from a parent revision to a child
  • a patch or diff could be applied to many similar parents
• a head is a revision with no children
  • tip is the changeset last saved in local history (always a head)
• a merge revision has two parents (more later...)
  • its merge ancestor is the closest common ancestor of both parents

Typical history with merges (hg glog excerpt)

o    changeset:   63829:97db7e2e1dc8
|\   parent:      63827:5df9cf7e1ae8
| |  parent:      63828:9def4e8126ee
| |  user:        Jesse Glick <jglick@netbeans.org>
| |  date:        Tue Jan 15 14:47:33 2008 -0500
| |  summary:     Automated merge with http://hg.netbeans.org/main/
| |
| o  changeset:   63828:9def4e8126ee
| |  parent:      63826:67b7cf411f30
| |  user:        Michal Zlamal <mzlamal@netbeans.org>
| |  date:        Tue Jan 15 18:27:34 2008 +0100
| |  summary:     Production build changes
| |
o |  changeset:   63827:5df9cf7e1ae8
|/   user:        Jesse Glick <jglick@netbeans.org>
|    date:        Tue Jan 15 10:19:41 2008 -0500
|    summary:     Top-level build.xml for convenience.
|
o  changeset:   63826:67b7cf411f30
[etc...]

Architecture and extensions

• Hg is almost all Python
  • a little C for bottlenecks and OS services
• Core is small, many features added in extensions
  • extensions usually add commands but can do other things too
  • registered in some hgrc
  • several useful extensions bundled with Hg
• Custom hooks can be run before or after commands
  • can be external scripts, or in-process Python for more power
  • example: check that your code compiles before commit

Why is NB using Hg?

• CVS is awful
  • making a branch of NB sources can take all day
• Subversion is better, but
  • rather bloated and complex
  • still lacks merge tracking, a basic VCS function
• OpenJDK, OpenSolaris, soon Glassfish will all use Hg
  • we can share expertise, maybe infrastructure

Key Hg advantages

• small: ~37 kLOC (incl. web view!) plus docs and tests
• extensible (and extensions are very easy to write)
• fast: disk access carefully optimized, cheap local clones
• decentralized: offline operations, simplified permissions
• straightforward UI, clear documentation
• actively maintained and developed, portable

Current Hg limitations

• Young system (Apr 2005)
  • future relative to Git and Bzr still uncertain
• Cannot clone just a subdirectory of a repo
• Cannot clone just the last year of a repo's history
• Some annoyances on Windows (long paths, case sensitivity)
• Massive renames will make the repo larger

Getting your feet wet

Everyday tasks you will perform with Mercurial.

Installing Hg

• See selenic.com/mercurial for downloads
• Windows users: use all-in-one installer
  • you probably want to configure CRLF newline translation
• Mac users: use an installer (for 10.4 or 10.5)
• Linux users: Ubuntu package, Fedora package, ...
  • inotify extension speeds up many operations, but still buggy
• Building from sources
  • generally easy, just need Python and a C compiler
• Configure your username!

Typical per-user configuration

Basic configuration (~/.hgrc):

[ui]
username = John Q. Hacker <jhacker@netbeans.org>
[extensions]
fetch =

Extra Windows configuration (...\Mercurial.ini):

[extensions]
win32text =
[encode]
{}** = cleverencode:
[decode]
{}** = cleverdecode:

Typical repository configuration

..../main/.hg/hgrc:

[paths]
default = http://hg.netbeans.org/main/
default-push = https://jhacker:secretpass@hg.netbeans.org/main/
golden = http://hg.netbeans.org/main-golden/
[defaults]
log = --no-merges

• default is the default pull location (can also be dir path)
• default-push is default push location (if different)
• golden is just a convenient alias for another location
• hg log run here will implicitly use --no-merges (-M)

NB IDE Integration

• Now in NB development builds
  • or available from Tools > Plugin Manager for 6.0 release
• No configuration needed, will autodetect files under Hg repo
• Diff, commit, push, annotate, etc. from inside IDE
• Report bugs: use mercurial component in Issuezilla

Cloning main

• All sources needed for regular NB build are in one repo
• hg clone http://hg.netbeans.org/main
• Easy to create a mirror for a local team and clone that
  • Update mirror when you like, or just pull remainder from main
• Clone also checks out tip unless you use -U

Doing a build

• Type: ant
• We can have top-level build.xml, unlike with CVS
• Should be able to open modules as NB projects as before

Updating sources

• Update your repository whenever you like
  • hg pull -u
  • (this may say you need to merge... more later)
• Or look at what would be pulled in
  • hg incoming for summary or hg in -p to see diffs
• Can use the Fetch extension to make this easier (more later)

Committing some changes

• See what you have to commit
  • hg stat for a summary
  • hg di for the patch
• Record the commit
  • hg ci (may specify files, else commits everything)
  • uses your default $EDITOR for the log message
  • first line of a log message should be self-contained summary
• Push one or more commits when you are ready
  • wise to do a build first; no "rush to commit" like with CVS
  • hg push will stop if you need to pull and merge first

Collaboration: merging

• If you and someone else have both made changes, you need to merge
  • even if you were not working on the same file(s)
• hg merge does a 3-way merge (two heads + common ancestor)
• If there are file-level conflicts, a merge tool will be launched
  • or conflict markers will be left to be resolved in a text editor
• Commit the result of a merge when you are satisfied with it
• For safety, Hg records every merge explicitly
  • CVS/SVN do an implicit merge after updating a modified tree
• If fetch extension enabled, hg fetch does pull, merge, commit
  • do not use with inotify, do not put auth info in push path

Adds, renames, deletes

• Add a file or whole dir with hg add
• Delete a file or whole dir with hg rem
  • if all files in a dir are gone, so is the dir
• Add & delete new/missing files: hg addremove, or hg ci -A
  • one top-level .hgignore replaces per-dir .cvsignore
• Rename a file or dir: hg ren
  • preserves history, not like CVS! (use --follow later)
  • and renames merge sensibly with modifications
• Adds/renames/deletes get committed alongside other changes

Browsing history

• hg log to browse history (of specified files, or all)
  • -v for full log message, -p for patches
  • -l 10 for last ten
• hg glog (graphlog extension) visualizes branches & merges
• hg tip shows tip revision, hg heads shows all heads
• hg di -r start:end to see a patch
• hg ann file shows which changesets created which lines

Fixing mistakes

• Normal programming errors ("hey, this is wrong")
  • hg backout reverts last change (tip) as new rev
  • with -m, reverts some earlier change (requires merge)
  • or, just fix somehow and commit
• Catastrophic mistakes ("did I just commit my ATM PIN?!")
  • hg rollback undoes last operation, such as commit
  • hg strip (mq extension) destroys rev and descendants
  • hg clone -r makes copy without last few changesets
  • none of these will save you if you have already pushed

Working with branches

• A "branch" is just a repo clone you treat specially
• hg tag sometag marks a revision with a symbolic name
• hg clone -r sometag original branch clones up to that point
• you can pull from or push to any repo whenever you like
  • paths section in .hg/hgrc can define aliases
  • default & default-push used unless you specify location
• hg transplant -s main rev copies a change from another repo
  • you might need to resolve merge conflicts
• Hg also has named branches but better not use them for now

hg.netbeans.org

• NB code lives on http://hg.netbeans.org/
• You can browse the repositories and their contents online
  • Hg bundles the equivalent of ViewCVS / FishEye
• If you can commit to CVS, you should have a login here too
  • for administrative reasons you may need a new password
• Until "flag day", these repositories are for testing only
  • all changes will be discarded
  • temporary password for testing: hguser
• commit notifications still sent to cvs@module.netbeans.org

What are the different repositories?

• main: modules in standard distro, infra, Stable AU
• main/contrib: alpha AU modules
• main/misc: other as-yet-uncategorized content
  • can be moved to main or main/contrib later
• main-golden: mirror of main that always compiles
  • should always be safe to pull from
• core-main etc.: team integration areas
• release61: fake demo of release branch

New file layout

• Modules were rearranged as part of Hg import
  • currently impractical to do this as Hg changeset
• Every module now named according to code name base
  • thus at top level, with contrib/* at second level
  • abbreviations: org.netbeans.modules.xml.core -> xml.core
• Some sources (tests, j2se projects) moved inside modules
• nbbuild, installer, etc. unchanged
• complete mapping recorded in nbbuild/translations

Cloning the forest

• Install Forest extension (see Hg wiki for details)
• Register extension: forest=..../forest.py
• hg fclone http://hg.netbeans.org/main
• contrib and misc get cloned as subdirectories
  • e.g. ant init all-contrib/docbook builds DocBook module
• hg fpull, etc. work on all three repos at once

Things you cannot push

To fix: save patches, strip bad revs, recommit patches, retry push

• Text files with CRLF newlines
  • please set up CRLF conversion on commit
  • or you can configure local repo to reject CRLF on commit
• Changesets using a non-@netbeans.org author
• Merge changesets created by hg fetch with passwords
• New unmerged heads (just need to merge & retry)
• Paths too long for Windows to deal with

External binaries

• To reduce repo size, we may store libraries externally
• Applies to: */external/*.zip,jar,gz,bz2,gem,dll
• Custom extension hooks into working copy storage
  • registered in .hg/hgrc by bootstrap Ant target
  • mostly automated already, may be some rough spots still
• Upon checkout, "decode" hook downloads binary from server
• Upon add & commit, "encode" hook uploads binary to server
• Local cache prevents same binary from being downloaded 2x
• Mostly transparent: operate on binaries using normal Hg commands

Cool Stuff

Hg has some unique capabilities you can explore.

Using a team repository

• Should be familiar to old TeamWare hands!
• Set up a clone of main, make sure it is OK
• Team members clone team repo, pull from it, push to it
• If the team repo is broken, you know who to blame!
• When things are stable (clean build & tests), push to main
• Pull from main-golden whenever you feel comfortable

Working with changesets

• hg bundle will save 1+ changesets as a compressed binary
  • with --base null, create archive of whole repo
  • hg unbundle to apply to "related" clone
  • hg incoming --bundle to preview contents
• hg export saves standard patch
  • use --git to track renames, binaries, execute bit
  • hg import loads patch as commit
• hg transplant applies a changeset to another repo
• hg mail sends changesets by mail, e.g. for review

Investigating bugs

• hg grep says when in history some text was added to any file
• hg bisect lets you narrow down a regression
  • set up a reliable unit test
  • identify a "last known good" revision
  • start it running and go to lunch!

Advanced repository browsing

• Commands like log take templates
  • can include various fields, e.g. full SHA-1 hash
  • powerful formatting options (fill para, ...)
  • save valuable templates as styles
• hg locate enumerates files by pattern (quickly!)
  • to do more: hg locate -0 ... | xargs -0 ...
• hg log -k to look for author or log message keyword
• hg log -k GPL --template '{rev}\t{desc|fill68|tabindent}\n'

Using MQ

• Mercurial Queues extension: manage and develop patches
• useful if you are not ready to commit a variety of changes
  • or for non-committers who can only submit patches in IZ
• qnew to begin work on a new patch, added to a stack
• qpush / qpop to select "active" patch in stack
• qdiff to review or save the current patch
• qrefresh to update current patch with working changes
• hg mail qtip to send a proposed patch to someone
• Not trivial to use, but very powerful!

Resources

• Remember to run hg help or hg help diff!
• http://www.selenic.com/mercurial/wiki/
• Free online book: ''Distributed Revision Control with Mercurial''
• Will be IZ component for NB-specific problems: nbbuild/mercurial
• Tips, some NB-specific: HgHowTos

Attachments

preso.xhtml

22320 bytes