bos@558: <!-- vim: set filetype=docbkxml shiftwidth=2 autoindent expandtab tw=77 : -->
bos@558: 
dongsheng@625: <chapter id="chap.mq">
bos@572:   <?dbhtml filename="managing-change-with-mercurial-queues.html"?>
bos@558:   <title>Managing change with Mercurial Queues</title>
bos@558: 
dongsheng@625:   <sect1 id="sec.mq.patch-mgmt">
bos@558:     <title>The patch management problem</title>
bos@558: 
bos@558:     <para>Here is a common scenario: you need to install a software
bos@558:       package from source, but you find a bug that you must fix in the
bos@558:       source before you can start using the package.  You make your
bos@558:       changes, forget about the package for a while, and a few months
bos@558:       later you need to upgrade to a newer version of the package.  If
bos@558:       the newer version of the package still has the bug, you must
bos@558:       extract your fix from the older source tree and apply it against
bos@558:       the newer version.  This is a tedious task, and it's easy to
bos@558:       make mistakes.</para>
bos@558: 
bos@558:     <para>This is a simple case of the <quote>patch management</quote>
bos@558:       problem.  You have an <quote>upstream</quote> source tree that
bos@558:       you can't change; you need to make some local changes on top of
bos@558:       the upstream tree; and you'd like to be able to keep those
bos@558:       changes separate, so that you can apply them to newer versions
bos@558:       of the upstream source.</para>
bos@558: 
bos@558:     <para>The patch management problem arises in many situations.
bos@558:       Probably the most visible is that a user of an open source
bos@558:       software project will contribute a bug fix or new feature to the
bos@558:       project's maintainers in the form of a patch.</para>
bos@558: 
bos@558:     <para>Distributors of operating systems that include open source
bos@558:       software often need to make changes to the packages they
bos@558:       distribute so that they will build properly in their
bos@558:       environments.</para>
bos@558: 
bos@558:     <para>When you have few changes to maintain, it is easy to manage
bos@558:       a single patch using the standard <command>diff</command> and
bos@558:       <command>patch</command> programs (see section <xref
dongsheng@625: 	linkend="sec.mq.patch"/> for a discussion of these
bos@559:       tools). Once the number of changes grows, it starts to make
bos@559:       sense to maintain patches as discrete <quote>chunks of
bos@559: 	work,</quote> so that for example a single patch will contain
bos@559:       only one bug fix (the patch might modify several files, but it's
bos@559:       doing <quote>only one thing</quote>), and you may have a number
bos@559:       of such patches for different bugs you need fixed and local
bos@559:       changes you require.  In this situation, if you submit a bug fix
bos@559:       patch to the upstream maintainers of a package and they include
bos@559:       your fix in a subsequent release, you can simply drop that
bos@559:       single patch when you're updating to the newer release.</para>
bos@558: 
bos@558:     <para>Maintaining a single patch against an upstream tree is a
bos@558:       little tedious and error-prone, but not difficult.  However, the
bos@558:       complexity of the problem grows rapidly as the number of patches
bos@558:       you have to maintain increases.  With more than a tiny number of
bos@558:       patches in hand, understanding which ones you have applied and
bos@558:       maintaining them moves from messy to overwhelming.</para>
bos@558: 
bos@558:     <para>Fortunately, Mercurial includes a powerful extension,
bos@558:       Mercurial Queues (or simply <quote>MQ</quote>), that massively
bos@558:       simplifies the patch management problem.</para>
bos@558: 
bos@558:   </sect1>
dongsheng@625:   <sect1 id="sec.mq.history">
bos@558:     <title>The prehistory of Mercurial Queues</title>
bos@558: 
bos@558:     <para>During the late 1990s, several Linux kernel developers
bos@558:       started to maintain <quote>patch series</quote> that modified
bos@558:       the behaviour of the Linux kernel.  Some of these series were
bos@558:       focused on stability, some on feature coverage, and others were
bos@558:       more speculative.</para>
bos@558: 
bos@558:     <para>The sizes of these patch series grew rapidly.  In 2002,
bos@558:       Andrew Morton published some shell scripts he had been using to
bos@558:       automate the task of managing his patch queues.  Andrew was
bos@558:       successfully using these scripts to manage hundreds (sometimes
bos@558:       thousands) of patches on top of the Linux kernel.</para>
bos@558: 
dongsheng@625:     <sect2 id="sec.mq.quilt">
bos@558:       <title>A patchwork quilt</title>
bos@558: 
bos@558:       <para>In early 2003, Andreas Gruenbacher and Martin Quinson
bos@558: 	borrowed the approach of Andrew's scripts and published a tool
bos@558: 	called <quote>patchwork quilt</quote>
bos@558: 	<citation>web:quilt</citation>, or simply <quote>quilt</quote>
bos@558: 	(see <citation>gruenbacher:2005</citation> for a paper
bos@558: 	describing it).  Because quilt substantially automated patch
bos@558: 	management, it rapidly gained a large following among open
bos@558: 	source software developers.</para>
bos@558: 
bos@558:       <para>Quilt manages a <emphasis>stack of patches</emphasis> on
bos@558: 	top of a directory tree. To begin, you tell quilt to manage a
bos@558: 	directory tree, and tell it which files you want to manage; it
bos@558: 	stores away the names and contents of those files.  To fix a
bos@558: 	bug, you create a new patch (using a single command), edit the
bos@558: 	files you need to fix, then <quote>refresh</quote> the
bos@558: 	patch.</para>
bos@558: 
bos@558:       <para>The refresh step causes quilt to scan the directory tree;
bos@558: 	it updates the patch with all of the changes you have made.
bos@558: 	You can create another patch on top of the first, which will
bos@558: 	track the changes required to modify the tree from <quote>tree
bos@558: 	  with one patch applied</quote> to <quote>tree with two
bos@558: 	  patches applied</quote>.</para>
bos@558: 
bos@558:       <para>You can <emphasis>change</emphasis> which patches are
bos@558: 	applied to the tree.  If you <quote>pop</quote> a patch, the
bos@558: 	changes made by that patch will vanish from the directory
bos@558: 	tree.  Quilt remembers which patches you have popped, though,
bos@558: 	so you can <quote>push</quote> a popped patch again, and the
bos@558: 	directory tree will be restored to contain the modifications
bos@558: 	in the patch.  Most importantly, you can run the
bos@558: 	<quote>refresh</quote> command at any time, and the topmost
bos@558: 	applied patch will be updated.  This means that you can, at
bos@558: 	any time, change both which patches are applied and what
bos@558: 	modifications those patches make.</para>
bos@558: 
bos@558:       <para>Quilt knows nothing about revision control tools, so it
bos@558: 	works equally well on top of an unpacked tarball or a
bos@558: 	Subversion working copy.</para>
bos@558: 
bos@558:     </sect2>
dongsheng@625:     <sect2 id="sec.mq.quilt-mq">
bos@558:       <title>From patchwork quilt to Mercurial Queues</title>
bos@558: 
bos@558:       <para>In mid-2005, Chris Mason took the features of quilt and
bos@558: 	wrote an extension that he called Mercurial Queues, which
bos@558: 	added quilt-like behaviour to Mercurial.</para>
bos@558: 
bos@558:       <para>The key difference between quilt and MQ is that quilt
bos@558: 	knows nothing about revision control systems, while MQ is
bos@558: 	<emphasis>integrated</emphasis> into Mercurial.  Each patch
bos@558: 	that you push is represented as a Mercurial changeset.  Pop a
bos@558: 	patch, and the changeset goes away.</para>
bos@558: 
bos@558:       <para>Because quilt does not care about revision control tools,
bos@558: 	it is still a tremendously useful piece of software to know
bos@558: 	about for situations where you cannot use Mercurial and
bos@558: 	MQ.</para>
bos@558: 
bos@558:     </sect2>
bos@558:   </sect1>
bos@558:   <sect1>
bos@558:     <title>The huge advantage of MQ</title>
bos@558: 
bos@558:     <para>I cannot overstate the value that MQ offers through the
bos@558:       unification of patches and revision control.</para>
bos@558: 
bos@558:     <para>A major reason that patches have persisted in the free
bos@559:       software and open source world&emdash;in spite of the
bos@559:       availability of increasingly capable revision control tools over
bos@559:       the years&emdash;is the <emphasis>agility</emphasis> they
bos@559:       offer.</para>
bos@558: 
bos@558:     <para>Traditional revision control tools make a permanent,
bos@558:       irreversible record of everything that you do.  While this has
bos@558:       great value, it's also somewhat stifling.  If you want to
bos@558:       perform a wild-eyed experiment, you have to be careful in how
bos@558:       you go about it, or you risk leaving unneeded&emdash;or worse,
bos@559:       misleading or destabilising&emdash;traces of your missteps and
bos@559:       errors in the permanent revision record.</para>
bos@558: 
bos@558:     <para>By contrast, MQ's marriage of distributed revision control
bos@558:       with patches makes it much easier to isolate your work.  Your
bos@558:       patches live on top of normal revision history, and you can make
bos@558:       them disappear or reappear at will.  If you don't like a patch,
bos@558:       you can drop it.  If a patch isn't quite as you want it to be,
bos@559:       simply fix it&emdash;as many times as you need to, until you
bos@559:       have refined it into the form you desire.</para>
bos@558: 
bos@558:     <para>As an example, the integration of patches with revision
bos@558:       control makes understanding patches and debugging their
bos@558:       effects&emdash;and their interplay with the code they're based
bos@559:       on&emdash;<emphasis>enormously</emphasis> easier. Since every
bos@559:       applied patch has an associated changeset, you can give <command
bos@559: 	role="hg-cmd">hg log</command> a file name to see which
bos@559:       changesets and patches affected the file.  You can use the
bos@559:       <command role="hg-cmd">hg bisect</command> command to
bos@559:       binary-search through all changesets and applied patches to see
bos@559:       where a bug got introduced or fixed.  You can use the <command
bos@558: 	role="hg-cmd">hg annotate</command> command to see which
bos@558:       changeset or patch modified a particular line of a source file.
bos@558:       And so on.</para>
bos@558: 
bos@558:   </sect1>
dongsheng@625:   <sect1 id="sec.mq.patch">
bos@558:     <title>Understanding patches</title>
bos@558: 
bos@558:     <para>Because MQ doesn't hide its patch-oriented nature, it is
bos@558:       helpful to understand what patches are, and a little about the
bos@558:       tools that work with them.</para>
bos@558: 
bos@558:     <para>The traditional Unix <command>diff</command> command
bos@558:       compares two files, and prints a list of differences between
bos@558:       them. The <command>patch</command> command understands these
bos@558:       differences as <emphasis>modifications</emphasis> to make to a
bos@558:       file.  Take a look below for a simple example of these commands
bos@558:       in action.</para>
bos@558: 
bos@567: &interaction.mq.dodiff.diff;
bos@558: 
bos@558:     <para>The type of file that <command>diff</command> generates (and
bos@558:       <command>patch</command> takes as input) is called a
bos@558:       <quote>patch</quote> or a <quote>diff</quote>; there is no
bos@558:       difference between a patch and a diff.  (We'll use the term
bos@558:       <quote>patch</quote>, since it's more commonly used.)</para>
bos@558: 
bos@558:     <para>A patch file can start with arbitrary text; the
bos@558:       <command>patch</command> command ignores this text, but MQ uses
bos@558:       it as the commit message when creating changesets.  To find the
bos@558:       beginning of the patch content, <command>patch</command>
bos@558:       searches for the first line that starts with the string
bos@558:       <quote><literal>diff -</literal></quote>.</para>
bos@558: 
bos@558:     <para>MQ works with <emphasis>unified</emphasis> diffs
bos@558:       (<command>patch</command> can accept several other diff formats,
bos@558:       but MQ doesn't).  A unified diff contains two kinds of header.
bos@558:       The <emphasis>file header</emphasis> describes the file being
bos@558:       modified; it contains the name of the file to modify.  When
bos@558:       <command>patch</command> sees a new file header, it looks for a
bos@558:       file with that name to start modifying.</para>
bos@558: 
bos@558:     <para>After the file header comes a series of
bos@558:       <emphasis>hunks</emphasis>.  Each hunk starts with a header;
bos@558:       this identifies the range of line numbers within the file that
bos@558:       the hunk should modify.  Following the header, a hunk starts and
bos@558:       ends with a few (usually three) lines of text from the
bos@558:       unmodified file; these are called the
bos@558:       <emphasis>context</emphasis> for the hunk.  If there's only a
bos@558:       small amount of context between successive hunks,
bos@558:       <command>diff</command> doesn't print a new hunk header; it just
bos@558:       runs the hunks together, with a few lines of context between
bos@558:       modifications.</para>
bos@558: 
bos@558:     <para>Each line of context begins with a space character.  Within
bos@558:       the hunk, a line that begins with
bos@558:       <quote><literal>-</literal></quote> means <quote>remove this
bos@558: 	line,</quote> while a line that begins with
bos@558:       <quote><literal>+</literal></quote> means <quote>insert this
bos@558: 	line.</quote>  For example, a line that is modified is
bos@558:       represented by one deletion and one insertion.</para>
bos@558: 
bos@558:     <para>We will return to some of the more subtle aspects of patches
dongsheng@625:       later (in section <xref linkend="sec.mq.adv-patch"/>), but you
bos@559:       should have
bos@559:       enough information now to use MQ.</para>
bos@558: 
bos@558:   </sect1>
dongsheng@625:   <sect1 id="sec.mq.start">
bos@558:     <title>Getting started with Mercurial Queues</title>
bos@558: 
bos@558:     <para>Because MQ is implemented as an extension, you must
bos@558:       explicitly enable before you can use it.  (You don't need to
bos@558:       download anything; MQ ships with the standard Mercurial
bos@559:       distribution.)  To enable MQ, edit your <filename
bos@559: 	role="home">~/.hgrc</filename> file, and add the lines
bos@559:       below.</para>
bos@558: 
bos@558:     <programlisting>[extensions] hgext.mq =</programlisting>
bos@558: 
bos@558:     <para>Once the extension is enabled, it will make a number of new
bos@558:       commands available.  To verify that the extension is working,
bos@558:       you can use <command role="hg-cmd">hg help</command> to see if
bos@559:       the <command role="hg-ext-mq">qinit</command> command is now
bos@558:       available.</para>
bos@558: 
bos@567: &interaction.mq.qinit-help.help;
bos@558: 
bos@558:     <para>You can use MQ with <emphasis>any</emphasis> Mercurial
bos@558:       repository, and its commands only operate within that
bos@558:       repository.  To get started, simply prepare the repository using
bos@559:       the <command role="hg-ext-mq">qinit</command> command.</para>
bos@559: 
bos@567: &interaction.mq.tutorial.qinit;
bos@559: 
bos@559:     <para>This command creates an empty directory called <filename
bos@558: 	role="special" class="directory">.hg/patches</filename>, where
bos@558:       MQ will keep its metadata.  As with many Mercurial commands, the
bos@559:       <command role="hg-ext-mq">qinit</command> command prints nothing
bos@558:       if it succeeds.</para>
bos@558: 
bos@558:     <sect2>
bos@558:       <title>Creating a new patch</title>
bos@558: 
bos@558:       <para>To begin work on a new patch, use the <command
bos@559: 	  role="hg-ext-mq">qnew</command> command.  This command takes
bos@559: 	one argument, the name of the patch to create.</para>
bos@559: 
bos@559:       <para>MQ will use this as the name of an actual file in the
bos@559: 	<filename role="special"
bos@559: 	  class="directory">.hg/patches</filename> directory, as you
bos@559: 	can see below.</para>
bos@559: 
bos@567: &interaction.mq.tutorial.qnew;
bos@558: 
bos@558:       <para>Also newly present in the <filename role="special"
bos@558: 	  class="directory">.hg/patches</filename> directory are two
bos@558: 	other files, <filename role="special">series</filename> and
bos@558: 	<filename role="special">status</filename>.  The <filename
bos@558: 	  role="special">series</filename> file lists all of the
bos@558: 	patches that MQ knows about for this repository, with one
bos@558: 	patch per line.  Mercurial uses the <filename
bos@558: 	  role="special">status</filename> file for internal
bos@558: 	book-keeping; it tracks all of the patches that MQ has
bos@558: 	<emphasis>applied</emphasis> in this repository.</para>
bos@558: 
bos@558:       <note>
bos@558: 	<para>  You may sometimes want to edit the <filename
bos@558: 	    role="special">series</filename> file by hand; for
bos@558: 	  example, to change the sequence in which some patches are
bos@558: 	  applied.  However, manually editing the <filename
bos@558: 	    role="special">status</filename> file is almost always a
bos@558: 	  bad idea, as it's easy to corrupt MQ's idea of what is
bos@558: 	  happening.</para>
bos@558:       </note>
bos@558: 
bos@558:       <para>Once you have created your new patch, you can edit files
bos@558: 	in the working directory as you usually would.  All of the
bos@558: 	normal Mercurial commands, such as <command role="hg-cmd">hg
bos@558: 	  diff</command> and <command role="hg-cmd">hg
bos@558: 	  annotate</command>, work exactly as they did before.</para>
bos@558: 
bos@558:     </sect2>
bos@558:     <sect2>
bos@558:       <title>Refreshing a patch</title>
bos@558: 
bos@558:       <para>When you reach a point where you want to save your work,
bos@559: 	use the <command role="hg-ext-mq">qrefresh</command> command
bos@559: 	to update the patch you are working on.</para>
bos@558: 
bos@567: &interaction.mq.tutorial.qrefresh;
bos@558: 
bos@559:       <para>This command folds the changes you have made in the
bos@559: 	working directory into your patch, and updates its
bos@559: 	corresponding changeset to contain those changes.</para>
bos@559: 
bos@559:       <para>You can run <command role="hg-ext-mq">qrefresh</command>
bos@559: 	as often as you like, so it's a good way to
bos@559: 	<quote>checkpoint</quote> your work.  Refresh your patch at an
bos@559: 	opportune time; try an experiment; and if the experiment
bos@558: 	doesn't work out, <command role="hg-cmd">hg revert</command>
bos@558: 	your modifications back to the last time you refreshed.</para>
bos@558: 
bos@567: &interaction.mq.tutorial.qrefresh2;
bos@558: 
bos@558:     </sect2>
bos@558:     <sect2>
bos@558:       <title>Stacking and tracking patches</title>
bos@558: 
bos@558:       <para>Once you have finished working on a patch, or need to work
bos@559: 	on another, you can use the <command
bos@559: 	  role="hg-ext-mq">qnew</command> command again to create a
bos@559: 	new patch. Mercurial will apply this patch on top of your
bos@559: 	existing patch.</para>
bos@559: 
bos@567: &interaction.mq.tutorial.qnew2;
bos@559:       <para>Notice that the patch contains the changes in our prior
bos@559: 	patch as part of its context (you can see this more clearly in
bos@559: 	the output of <command role="hg-cmd">hg
bos@558: 	  annotate</command>).</para>
bos@558: 
bos@558:       <para>So far, with the exception of <command
bos@559: 	  role="hg-ext-mq">qnew</command> and <command
bos@559: 	  role="hg-ext-mq">qrefresh</command>, we've been careful to
bos@558: 	only use regular Mercurial commands.  However, MQ provides
bos@558: 	many commands that are easier to use when you are thinking
bos@559: 	about patches, as illustrated below.</para>
bos@559: 
bos@567: &interaction.mq.tutorial.qseries;
bos@558: 
bos@558:       <itemizedlist>
bos@558: 	<listitem><para>The <command
bos@559: 	      role="hg-ext-mq">qseries</command> command lists every
bos@558: 	    patch that MQ knows about in this repository, from oldest
bos@558: 	    to newest (most recently
bos@559: 	    <emphasis>created</emphasis>).</para>
bos@559: 	</listitem>
bos@558: 	<listitem><para>The <command
bos@559: 	      role="hg-ext-mq">qapplied</command> command lists every
bos@558: 	    patch that MQ has <emphasis>applied</emphasis> in this
bos@558: 	    repository, again from oldest to newest (most recently
bos@559: 	    applied).</para>
bos@559: 	</listitem></itemizedlist>
bos@558: 
bos@558:     </sect2>
bos@558:     <sect2>
bos@558:       <title>Manipulating the patch stack</title>
bos@558: 
bos@558:       <para>The previous discussion implied that there must be a
bos@558: 	difference between <quote>known</quote> and
bos@558: 	<quote>applied</quote> patches, and there is.  MQ can manage a
bos@558: 	patch without it being applied in the repository.</para>
bos@558: 
bos@558:       <para>An <emphasis>applied</emphasis> patch has a corresponding
bos@558: 	changeset in the repository, and the effects of the patch and
bos@558: 	changeset are visible in the working directory.  You can undo
bos@558: 	the application of a patch using the <command
bos@559: 	  role="hg-ext-mq">qpop</command> command.  MQ still
bos@558: 	<emphasis>knows about</emphasis>, or manages, a popped patch,
bos@558: 	but the patch no longer has a corresponding changeset in the
bos@558: 	repository, and the working directory does not contain the
bos@558: 	changes made by the patch.  Figure <xref
dongsheng@625: 	  linkend="fig.mq.stack"/> illustrates
bos@559: 	the difference between applied and tracked patches.</para>
bos@558: 
dongsheng@625:       <informalfigure id="fig.mq.stack">
bos@559: 	<mediaobject><imageobject><imagedata
dongsheng@625: 				    fileref="images/mq-stack.png"/></imageobject><textobject><phrase>XXX 
bos@559: 	      add text</phrase></textobject><caption><para>Applied and
bos@559: 	      unapplied patches in the MQ patch
bos@559: 	      stack</para></caption></mediaobject>
bos@558:       </informalfigure>
bos@558: 
bos@558:       <para>You can reapply an unapplied, or popped, patch using the
bos@559: 	<command role="hg-ext-mq">qpush</command> command.  This
bos@558: 	creates a new changeset to correspond to the patch, and the
bos@558: 	patch's changes once again become present in the working
bos@559: 	directory.  See below for examples of <command
bos@559: 	  role="hg-ext-mq">qpop</command> and <command
bos@559: 	  role="hg-ext-mq">qpush</command> in action.</para>
bos@567: &interaction.mq.tutorial.qpop;
bos@558: 
bos@559:       <para>Notice that once we have popped a patch or two patches,
bos@559: 	the output of <command role="hg-ext-mq">qseries</command>
bos@559: 	remains the same, while that of <command
bos@559: 	  role="hg-ext-mq">qapplied</command> has changed.</para>
bos@559: 
bos@559: 
bos@558:     </sect2>
bos@558:     <sect2>
bos@558:       <title>Pushing and popping many patches</title>
bos@558: 
bos@559:       <para>While <command role="hg-ext-mq">qpush</command> and
bos@559: 	<command role="hg-ext-mq">qpop</command> each operate on a
bos@558: 	single patch at a time by default, you can push and pop many
bos@558: 	patches in one go.  The <option
bos@559: 	  role="hg-ext-mq-cmd-qpush-opt">hg -a</option> option to
bos@559: 	<command role="hg-ext-mq">qpush</command> causes it to push
bos@558: 	all unapplied patches, while the <option
bos@558: 	  role="hg-ext-mq-cmd-qpop-opt">-a</option> option to <command
bos@559: 	  role="hg-ext-mq">qpop</command> causes it to pop all applied
bos@558: 	patches.  (For some more ways to push and pop many patches,
dongsheng@625: 	see section <xref linkend="sec.mq.perf"/>
bos@559: 	below.)</para>
bos@558: 
bos@567: &interaction.mq.tutorial.qpush-a;
bos@558: 
bos@558:     </sect2>
bos@558:     <sect2>
bos@558:       <title>Safety checks, and overriding them</title>
bos@558: 
bos@558:       <para>Several MQ commands check the working directory before
bos@558: 	they do anything, and fail if they find any modifications.
bos@558: 	They do this to ensure that you won't lose any changes that
bos@559: 	you have made, but not yet incorporated into a patch.  The
bos@559: 	example below illustrates this; the <command
bos@559: 	  role="hg-ext-mq">qnew</command> command will not create a
bos@558: 	new patch if there are outstanding changes, caused in this
bos@558: 	case by the <command role="hg-cmd">hg add</command> of
bos@558: 	<filename>file3</filename>.</para>
bos@558: 
bos@567: &interaction.mq.tutorial.add;
bos@558: 
bos@558:       <para>Commands that check the working directory all take an
bos@558: 	<quote>I know what I'm doing</quote> option, which is always
bos@558: 	named <option>-f</option>.  The exact meaning of
bos@558: 	<option>-f</option> depends on the command.  For example,
bos@558: 	<command role="hg-cmd">hg qnew <option
bos@559: 	    role="hg-ext-mq-cmd-qnew-opt">hg -f</option></command>
bos@559: 	will incorporate any outstanding changes into the new patch it
bos@558: 	creates, but <command role="hg-cmd">hg qpop <option
bos@559: 	    role="hg-ext-mq-cmd-qpop-opt">hg -f</option></command>
bos@559: 	will revert modifications to any files affected by the patch
bos@559: 	that it is popping.  Be sure to read the documentation for a
bos@558: 	command's <option>-f</option> option before you use it!</para>
bos@558: 
bos@558:     </sect2>
bos@558:     <sect2>
bos@558:       <title>Working on several patches at once</title>
bos@558: 
bos@559:       <para>The <command role="hg-ext-mq">qrefresh</command> command
bos@558: 	always refreshes the <emphasis>topmost</emphasis> applied
bos@558: 	patch.  This means that you can suspend work on one patch (by
bos@558: 	refreshing it), pop or push to make a different patch the top,
bos@558: 	and work on <emphasis>that</emphasis> patch for a
bos@558: 	while.</para>
bos@558: 
bos@558:       <para>Here's an example that illustrates how you can use this
bos@558: 	ability. Let's say you're developing a new feature as two
bos@558: 	patches.  The first is a change to the core of your software,
bos@559: 	and the second&emdash;layered on top of the
bos@559: 	first&emdash;changes the user interface to use the code you
bos@559: 	just added to the core.  If you notice a bug in the core while
bos@559: 	you're working on the UI patch, it's easy to fix the core.
bos@559: 	Simply <command role="hg-ext-mq">qrefresh</command> the UI
bos@559: 	patch to save your in-progress changes, and <command
bos@559: 	  role="hg-ext-mq">qpop</command> down to the core patch.  Fix
bos@559: 	the core bug, <command role="hg-ext-mq">qrefresh</command> the
bos@559: 	core patch, and <command role="hg-ext-mq">qpush</command> back
bos@558: 	to the UI patch to continue where you left off.</para>
bos@558: 
bos@558:     </sect2>
bos@558:   </sect1>
dongsheng@625:   <sect1 id="sec.mq.adv-patch">
bos@558:     <title>More about patches</title>
bos@558: 
bos@558:     <para>MQ uses the GNU <command>patch</command> command to apply
bos@558:       patches, so it's helpful to know a few more detailed aspects of
bos@558:       how <command>patch</command> works, and about patches
bos@558:       themselves.</para>
bos@558: 
bos@558:     <sect2>
bos@558:       <title>The strip count</title>
bos@558: 
bos@558:       <para>If you look at the file headers in a patch, you will
bos@558: 	notice that the pathnames usually have an extra component on
bos@558: 	the front that isn't present in the actual path name.  This is
bos@558: 	a holdover from the way that people used to generate patches
bos@558: 	(people still do this, but it's somewhat rare with modern
bos@558: 	revision control tools).</para>
bos@558: 
bos@558:       <para>Alice would unpack a tarball, edit her files, then decide
bos@558: 	that she wanted to create a patch.  So she'd rename her
bos@558: 	working directory, unpack the tarball again (hence the need
bos@558: 	for the rename), and use the <option
bos@558: 	  role="cmd-opt-diff">-r</option> and <option
bos@558: 	  role="cmd-opt-diff">-N</option> options to
bos@558: 	<command>diff</command> to recursively generate a patch
bos@558: 	between the unmodified directory and the modified one.  The
bos@558: 	result would be that the name of the unmodified directory
bos@558: 	would be at the front of the left-hand path in every file
bos@558: 	header, and the name of the modified directory would be at the
bos@558: 	front of the right-hand path.</para>
bos@558: 
bos@558:       <para>Since someone receiving a patch from the Alices of the net
bos@558: 	would be unlikely to have unmodified and modified directories
bos@558: 	with exactly the same names, the <command>patch</command>
bos@558: 	command has a <option role="cmd-opt-patch">-p</option> option
bos@558: 	that indicates the number of leading path name components to
bos@558: 	strip when trying to apply a patch.  This number is called the
bos@558: 	<emphasis>strip count</emphasis>.</para>
bos@558: 
bos@558:       <para>An option of <quote><literal>-p1</literal></quote> means
bos@558: 	<quote>use a strip count of one</quote>.  If
bos@558: 	<command>patch</command> sees a file name
bos@558: 	<filename>foo/bar/baz</filename> in a file header, it will
bos@558: 	strip <filename>foo</filename> and try to patch a file named
bos@558: 	<filename>bar/baz</filename>.  (Strictly speaking, the strip
bos@558: 	count refers to the number of <emphasis>path
bos@558: 	  separators</emphasis> (and the components that go with them
bos@558: 	) to strip.  A strip count of one will turn
bos@558: 	<filename>foo/bar</filename> into <filename>bar</filename>,
bos@558: 	but <filename>/foo/bar</filename> (notice the extra leading
bos@558: 	slash) into <filename>foo/bar</filename>.)</para>
bos@558: 
bos@558:       <para>The <quote>standard</quote> strip count for patches is
bos@558: 	one; almost all patches contain one leading path name
bos@558: 	component that needs to be stripped. Mercurial's <command
bos@558: 	  role="hg-cmd">hg diff</command> command generates path names
bos@558: 	in this form, and the <command role="hg-cmd">hg
bos@558: 	  import</command> command and MQ expect patches to have a
bos@558: 	strip count of one.</para>
bos@558: 
bos@558:       <para>If you receive a patch from someone that you want to add
bos@558: 	to your patch queue, and the patch needs a strip count other
bos@558: 	than one, you cannot just <command
bos@559: 	  role="hg-ext-mq">qimport</command> the patch, because
bos@559: 	<command role="hg-ext-mq">qimport</command> does not yet have
bos@558: 	a <literal>-p</literal> option (see <ulink role="hg-bug"
bos@558: 	  url="http://www.selenic.com/mercurial/bts/issue311">issue
bos@558: 	  311</ulink>).  Your best bet is to <command
bos@559: 	  role="hg-ext-mq">qnew</command> a patch of your own, then
bos@558: 	use <command>patch -pN</command> to apply their patch,
bos@558: 	followed by <command role="hg-cmd">hg addremove</command> to
bos@558: 	pick up any files added or removed by the patch, followed by
bos@559: 	<command role="hg-ext-mq">hg qrefresh</command>. This
bos@559: 	complexity may become unnecessary; see <ulink role="hg-bug"
bos@558: 	  url="http://www.selenic.com/mercurial/bts/issue311">issue
bos@559: 	  311</ulink> for details.
bos@559:       </para>
bos@558:     </sect2>
bos@558:     <sect2>
bos@558:       <title>Strategies for applying a patch</title>
bos@558: 
bos@558:       <para>When <command>patch</command> applies a hunk, it tries a
bos@558: 	handful of successively less accurate strategies to try to
bos@558: 	make the hunk apply. This falling-back technique often makes
bos@558: 	it possible to take a patch that was generated against an old
bos@558: 	version of a file, and apply it against a newer version of
bos@558: 	that file.</para>
bos@558: 
bos@558:       <para>First, <command>patch</command> tries an exact match,
bos@558: 	where the line numbers, the context, and the text to be
bos@558: 	modified must apply exactly.  If it cannot make an exact
bos@558: 	match, it tries to find an exact match for the context,
bos@558: 	without honouring the line numbering information.  If this
bos@558: 	succeeds, it prints a line of output saying that the hunk was
bos@558: 	applied, but at some <emphasis>offset</emphasis> from the
bos@558: 	original line number.</para>
bos@558: 
bos@558:       <para>If a context-only match fails, <command>patch</command>
bos@558: 	removes the first and last lines of the context, and tries a
bos@558: 	<emphasis>reduced</emphasis> context-only match.  If the hunk
bos@558: 	with reduced context succeeds, it prints a message saying that
bos@558: 	it applied the hunk with a <emphasis>fuzz factor</emphasis>
bos@558: 	(the number after the fuzz factor indicates how many lines of
bos@558: 	context <command>patch</command> had to trim before the patch
bos@558: 	applied).</para>
bos@558: 
bos@558:       <para>When neither of these techniques works,
bos@558: 	<command>patch</command> prints a message saying that the hunk
bos@558: 	in question was rejected.  It saves rejected hunks (also
bos@558: 	simply called <quote>rejects</quote>) to a file with the same
bos@558: 	name, and an added <filename role="special">.rej</filename>
bos@558: 	extension.  It also saves an unmodified copy of the file with
bos@558: 	a <filename role="special">.orig</filename> extension; the
bos@558: 	copy of the file without any extensions will contain any
bos@558: 	changes made by hunks that <emphasis>did</emphasis> apply
bos@558: 	cleanly.  If you have a patch that modifies
bos@558: 	<filename>foo</filename> with six hunks, and one of them fails
bos@558: 	to apply, you will have: an unmodified
bos@558: 	<filename>foo.orig</filename>, a <filename>foo.rej</filename>
bos@558: 	containing one hunk, and <filename>foo</filename>, containing
bos@558: 	the changes made by the five successful hunks.</para>
bos@558: 
bos@558:     </sect2>
bos@558:     <sect2>
bos@558:       <title>Some quirks of patch representation</title>
bos@558: 
bos@558:       <para>There are a few useful things to know about how
bos@558: 	<command>patch</command> works with files.</para>
bos@558:       <itemizedlist>
bos@558: 	<listitem><para>This should already be obvious, but
bos@558: 	    <command>patch</command> cannot handle binary
bos@559: 	    files.</para>
bos@559: 	</listitem>
bos@558: 	<listitem><para>Neither does it care about the executable bit;
bos@558: 	    it creates new files as readable, but not
bos@559: 	    executable.</para>
bos@559: 	</listitem>
bos@558: 	<listitem><para><command>patch</command> treats the removal of
bos@558: 	    a file as a diff between the file to be removed and the
bos@558: 	    empty file.  So your idea of <quote>I deleted this
bos@558: 	      file</quote> looks like <quote>every line of this file
bos@559: 	      was deleted</quote> in a patch.</para>
bos@559: 	</listitem>
bos@558: 	<listitem><para>It treats the addition of a file as a diff
bos@558: 	    between the empty file and the file to be added.  So in a
bos@558: 	    patch, your idea of <quote>I added this file</quote> looks
bos@558: 	    like <quote>every line of this file was
bos@559: 	      added</quote>.</para>
bos@559: 	</listitem>
bos@558: 	<listitem><para>It treats a renamed file as the removal of the
bos@558: 	    old name, and the addition of the new name.  This means
bos@558: 	    that renamed files have a big footprint in patches.  (Note
bos@558: 	    also that Mercurial does not currently try to infer when
bos@559: 	    files have been renamed or copied in a patch.)</para>
bos@559: 	</listitem>
bos@558: 	<listitem><para><command>patch</command> cannot represent
bos@558: 	    empty files, so you cannot use a patch to represent the
bos@558: 	    notion <quote>I added this empty file to the
bos@559: 	      tree</quote>.</para>
bos@559: 	</listitem></itemizedlist>
bos@559:     </sect2>
bos@558:     <sect2>
bos@558:       <title>Beware the fuzz</title>
bos@558: 
bos@558:       <para>While applying a hunk at an offset, or with a fuzz factor,
bos@558: 	will often be completely successful, these inexact techniques
bos@558: 	naturally leave open the possibility of corrupting the patched
bos@558: 	file.  The most common cases typically involve applying a
bos@558: 	patch twice, or at an incorrect location in the file.  If
bos@558: 	<command>patch</command> or <command
bos@559: 	  role="hg-ext-mq">qpush</command> ever mentions an offset or
bos@558: 	fuzz factor, you should make sure that the modified files are
bos@558: 	correct afterwards.</para>
bos@558: 
bos@558:       <para>It's often a good idea to refresh a patch that has applied
bos@558: 	with an offset or fuzz factor; refreshing the patch generates
bos@558: 	new context information that will make it apply cleanly.  I
bos@558: 	say <quote>often,</quote> not <quote>always,</quote> because
bos@558: 	sometimes refreshing a patch will make it fail to apply
bos@558: 	against a different revision of the underlying files.  In some
bos@558: 	cases, such as when you're maintaining a patch that must sit
bos@558: 	on top of multiple versions of a source tree, it's acceptable
bos@558: 	to have a patch apply with some fuzz, provided you've verified
bos@558: 	the results of the patching process in such cases.</para>
bos@558: 
bos@558:     </sect2>
bos@558:     <sect2>
bos@558:       <title>Handling rejection</title>
bos@558: 
bos@559:       <para>If <command role="hg-ext-mq">qpush</command> fails to
bos@558: 	apply a patch, it will print an error message and exit.  If it
bos@558: 	has left <filename role="special">.rej</filename> files
bos@558: 	behind, it is usually best to fix up the rejected hunks before
bos@558: 	you push more patches or do any further work.</para>
bos@558: 
bos@558:       <para>If your patch <emphasis>used to</emphasis> apply cleanly,
bos@558: 	and no longer does because you've changed the underlying code
bos@558: 	that your patches are based on, Mercurial Queues can help; see
bos@559: 	section <xref
dongsheng@625: 	  linkend="sec.mq.merge"/> for details.</para>
bos@558: 
bos@558:       <para>Unfortunately, there aren't any great techniques for
bos@558: 	dealing with rejected hunks.  Most often, you'll need to view
bos@558: 	the <filename role="special">.rej</filename> file and edit the
bos@558: 	target file, applying the rejected hunks by hand.</para>
bos@558: 
bos@558:       <para>If you're feeling adventurous, Neil Brown, a Linux kernel
bos@558: 	hacker, wrote a tool called <command>wiggle</command>
bos@558: 	<citation>web:wiggle</citation>, which is more vigorous than
bos@558: 	<command>patch</command> in its attempts to make a patch
bos@558: 	apply.</para>
bos@558: 
bos@558:       <para>Another Linux kernel hacker, Chris Mason (the author of
bos@558: 	Mercurial Queues), wrote a similar tool called
bos@558: 	<command>mpatch</command> <citation>web:mpatch</citation>,
bos@558: 	which takes a simple approach to automating the application of
bos@558: 	hunks rejected by <command>patch</command>.  The
bos@558: 	<command>mpatch</command> command can help with four common
bos@558: 	reasons that a hunk may be rejected:</para>
bos@558: 
bos@558:       <itemizedlist>
bos@558: 	<listitem><para>The context in the middle of a hunk has
bos@559: 	    changed.</para>
bos@559: 	</listitem>
bos@558: 	<listitem><para>A hunk is missing some context at the
bos@559: 	    beginning or end.</para>
bos@559: 	</listitem>
bos@558: 	<listitem><para>A large hunk might apply better&emdash;either
bos@559: 	    entirely or in part&emdash;if it was broken up into
bos@559: 	    smaller hunks.</para>
bos@559: 	</listitem>
bos@558: 	<listitem><para>A hunk removes lines with slightly different
bos@559: 	    content than those currently present in the file.</para>
bos@559: 	</listitem></itemizedlist>
bos@558: 
bos@558:       <para>If you use <command>wiggle</command> or
bos@558: 	<command>mpatch</command>, you should be doubly careful to
bos@558: 	check your results when you're done.  In fact,
bos@558: 	<command>mpatch</command> enforces this method of
bos@558: 	double-checking the tool's output, by automatically dropping
bos@558: 	you into a merge program when it has done its job, so that you
bos@558: 	can verify its work and finish off any remaining
bos@558: 	merges.</para>
bos@558: 
bos@558:     </sect2>
bos@558:   </sect1>
dongsheng@625:   <sect1 id="sec.mq.perf">
bos@558:     <title>Getting the best performance out of MQ</title>
bos@558: 
bos@558:     <para>MQ is very efficient at handling a large number of patches.
bos@558:       I ran some performance experiments in mid-2006 for a talk that I
bos@558:       gave at the 2006 EuroPython conference
bos@558:       <citation>web:europython</citation>.  I used as my data set the
bos@558:       Linux 2.6.17-mm1 patch series, which consists of 1,738 patches.
bos@558:       I applied these on top of a Linux kernel repository containing
bos@558:       all 27,472 revisions between Linux 2.6.12-rc2 and Linux
bos@558:       2.6.17.</para>
bos@558: 
bos@558:     <para>On my old, slow laptop, I was able to <command
bos@558: 	role="hg-cmd">hg qpush <option
bos@559: 	  role="hg-ext-mq-cmd-qpush-opt">hg -a</option></command> all
bos@558:       1,738 patches in 3.5 minutes, and <command role="hg-cmd">hg qpop
bos@559: 	<option role="hg-ext-mq-cmd-qpop-opt">hg -a</option></command>
bos@558:       them all in 30 seconds.  (On a newer laptop, the time to push
bos@558:       all patches dropped to two minutes.)  I could <command
bos@559: 	role="hg-ext-mq">qrefresh</command> one of the biggest patches
bos@558:       (which made 22,779 lines of changes to 287 files) in 6.6
bos@558:       seconds.</para>
bos@558: 
bos@558:     <para>Clearly, MQ is well suited to working in large trees, but
bos@558:       there are a few tricks you can use to get the best performance
bos@558:       of it.</para>
bos@558: 
bos@558:     <para>First of all, try to <quote>batch</quote> operations
bos@558:       together.  Every time you run <command
bos@559: 	role="hg-ext-mq">qpush</command> or <command
bos@559: 	role="hg-ext-mq">qpop</command>, these commands scan the
bos@558:       working directory once to make sure you haven't made some
bos@558:       changes and then forgotten to run <command
bos@559: 	role="hg-ext-mq">qrefresh</command>.  On a small tree, the
bos@558:       time that this scan takes is unnoticeable.  However, on a
bos@558:       medium-sized tree (containing tens of thousands of files), it
bos@558:       can take a second or more.</para>
bos@558: 
bos@559:     <para>The <command role="hg-ext-mq">qpush</command> and <command
bos@559: 	role="hg-ext-mq">qpop</command> commands allow you to push and
bos@558:       pop multiple patches at a time.  You can identify the
bos@558:       <quote>destination patch</quote> that you want to end up at.
bos@559:       When you <command role="hg-ext-mq">qpush</command> with a
bos@558:       destination specified, it will push patches until that patch is
bos@558:       at the top of the applied stack.  When you <command
bos@559: 	role="hg-ext-mq">qpop</command> to a destination, MQ will pop
bos@558:       patches until the destination patch is at the top.</para>
bos@558: 
bos@558:     <para>You can identify a destination patch using either the name
bos@558:       of the patch, or by number.  If you use numeric addressing,
bos@558:       patches are counted from zero; this means that the first patch
bos@558:       is zero, the second is one, and so on.</para>
bos@558: 
bos@558:   </sect1>
dongsheng@625:   <sect1 id="sec.mq.merge">
bos@558:     <title>Updating your patches when the underlying code
bos@558:       changes</title>
bos@558: 
bos@558:     <para>It's common to have a stack of patches on top of an
bos@558:       underlying repository that you don't modify directly.  If you're
bos@558:       working on changes to third-party code, or on a feature that is
bos@558:       taking longer to develop than the rate of change of the code
bos@558:       beneath, you will often need to sync up with the underlying
bos@558:       code, and fix up any hunks in your patches that no longer apply.
bos@558:       This is called <emphasis>rebasing</emphasis> your patch
bos@558:       series.</para>
bos@558: 
bos@558:     <para>The simplest way to do this is to <command role="hg-cmd">hg
bos@559: 	qpop <option role="hg-ext-mq-cmd-qpop-opt">hg
bos@559: 	  -a</option></command> your patches, then <command
bos@559: 	role="hg-cmd">hg pull</command> changes into the underlying
bos@559:       repository, and finally <command role="hg-cmd">hg qpush <option
bos@559: 	  role="hg-ext-mq-cmd-qpop-opt">hg -a</option></command> your
bos@558:       patches again.  MQ will stop pushing any time it runs across a
bos@558:       patch that fails to apply during conflicts, allowing you to fix
bos@559:       your conflicts, <command role="hg-ext-mq">qrefresh</command> the
bos@558:       affected patch, and continue pushing until you have fixed your
bos@558:       entire stack.</para>
bos@558: 
bos@558:     <para>This approach is easy to use and works well if you don't
bos@558:       expect changes to the underlying code to affect how well your
bos@558:       patches apply. If your patch stack touches code that is modified
bos@558:       frequently or invasively in the underlying repository, however,
bos@558:       fixing up rejected hunks by hand quickly becomes
bos@558:       tiresome.</para>
bos@558: 
bos@559:     <para>It's possible to partially automate the rebasing process.
bos@559:       If your patches apply cleanly against some revision of the
bos@558:       underlying repo, MQ can use this information to help you to
bos@558:       resolve conflicts between your patches and a different
bos@558:       revision.</para>
bos@558: 
bos@558:     <para>The process is a little involved.</para>
bos@558:     <orderedlist>
bos@558:       <listitem><para>To begin, <command role="hg-cmd">hg qpush
bos@558: 	    -a</command> all of your patches on top of the revision
bos@559: 	  where you know that they apply cleanly.</para>
bos@559:       </listitem>
bos@558:       <listitem><para>Save a backup copy of your patch directory using
bos@558: 	  <command role="hg-cmd">hg qsave <option
bos@559: 	      role="hg-ext-mq-cmd-qsave-opt">hg -e</option> <option
bos@559: 	      role="hg-ext-mq-cmd-qsave-opt">hg -c</option></command>.
bos@558: 	  This prints the name of the directory that it has saved the
bos@558: 	  patches in.  It will save the patches to a directory called
bos@558: 	  <filename role="special"
bos@558: 	    class="directory">.hg/patches.N</filename>, where
bos@558: 	  <literal>N</literal> is a small integer.  It also commits a
bos@558: 	  <quote>save changeset</quote> on top of your applied
bos@558: 	  patches; this is for internal book-keeping, and records the
bos@558: 	  states of the <filename role="special">series</filename> and
bos@559: 	  <filename role="special">status</filename> files.</para>
bos@559:       </listitem>
bos@559:       <listitem><para>Use <command role="hg-cmd">hg pull</command> to
bos@559: 	  bring new changes into the underlying repository.  (Don't
bos@559: 	  run <command role="hg-cmd">hg pull -u</command>; see below
bos@559: 	  for why.)</para>
bos@559:       </listitem>
bos@558:       <listitem><para>Update to the new tip revision, using <command
bos@558: 	    role="hg-cmd">hg update <option
bos@558: 	      role="hg-opt-update">-C</option></command> to override
bos@559: 	  the patches you have pushed.</para>
bos@559:       </listitem>
bos@558:       <listitem><para>Merge all patches using
bos@559: 	  \hgcmdargs{qpush}{<option role="hg-ext-mq-cmd-qpush-opt">hg
bos@559: 	    -m</option> <option role="hg-ext-mq-cmd-qpush-opt">hg
bos@559: 	    -a</option>}.  The <option
bos@559: 	    role="hg-ext-mq-cmd-qpush-opt">hg -m</option> option to
bos@559: 	  <command role="hg-ext-mq">qpush</command> tells MQ to
bos@558: 	  perform a three-way merge if the patch fails to
bos@559: 	  apply.</para>
bos@559:       </listitem></orderedlist>
bos@558: 
bos@558:     <para>During the <command role="hg-cmd">hg qpush <option
bos@559: 	  role="hg-ext-mq-cmd-qpush-opt">hg -m</option></command>,
bos@559:       each patch in the <filename role="special">series</filename>
bos@559:       file is applied normally.  If a patch applies with fuzz or
bos@559:       rejects, MQ looks at the queue you <command
bos@559: 	role="hg-ext-mq">qsave</command>d, and performs a three-way
bos@558:       merge with the corresponding changeset.  This merge uses
bos@558:       Mercurial's normal merge machinery, so it may pop up a GUI merge
bos@558:       tool to help you to resolve problems.</para>
bos@558: 
bos@558:     <para>When you finish resolving the effects of a patch, MQ
bos@558:       refreshes your patch based on the result of the merge.</para>
bos@558: 
bos@558:     <para>At the end of this process, your repository will have one
bos@558:       extra head from the old patch queue, and a copy of the old patch
bos@558:       queue will be in <filename role="special"
bos@558: 	class="directory">.hg/patches.N</filename>. You can remove the
bos@559:       extra head using <command role="hg-cmd">hg qpop -a -n
bos@558: 	patches.N</command> or <command role="hg-cmd">hg
bos@558: 	strip</command>.  You can delete <filename role="special"
bos@558: 	class="directory">.hg/patches.N</filename> once you are sure
bos@558:       that you no longer need it as a backup.</para>
bos@558: 
bos@558:   </sect1>
bos@558:   <sect1>
bos@558:     <title>Identifying patches</title>
bos@558: 
bos@558:     <para>MQ commands that work with patches let you refer to a patch
bos@558:       either by using its name or by a number.  By name is obvious
bos@558:       enough; pass the name <filename>foo.patch</filename> to <command
bos@559: 	role="hg-ext-mq">qpush</command>, for example, and it will
bos@558:       push patches until <filename>foo.patch</filename> is
bos@558:       applied.</para>
bos@558: 
bos@558:     <para>As a shortcut, you can refer to a patch using both a name
bos@558:       and a numeric offset; <literal>foo.patch-2</literal> means
bos@558:       <quote>two patches before <literal>foo.patch</literal></quote>,
bos@558:       while <literal>bar.patch+4</literal> means <quote>four patches
bos@558: 	after <literal>bar.patch</literal></quote>.</para>
bos@558: 
bos@558:     <para>Referring to a patch by index isn't much different.  The
bos@558:       first patch printed in the output of <command
bos@559: 	role="hg-ext-mq">qseries</command> is patch zero (yes, it's
bos@558:       one of those start-at-zero counting systems); the second is
bos@558:       patch one; and so on.</para>
bos@558: 
bos@558:     <para>MQ also makes it easy to work with patches when you are
bos@558:       using normal Mercurial commands.  Every command that accepts a
bos@558:       changeset ID will also accept the name of an applied patch.  MQ
bos@558:       augments the tags normally in the repository with an eponymous
bos@558:       one for each applied patch.  In addition, the special tags
bos@558:       \index{tags!special tag
bos@558:       names!<literal>qbase</literal>}<literal>qbase</literal> and
bos@558:       \index{tags!special tag
bos@558:       names!<literal>qtip</literal>}<literal>qtip</literal> identify
bos@558:       the <quote>bottom-most</quote> and topmost applied patches,
bos@558:       respectively.</para>
bos@558: 
bos@558:     <para>These additions to Mercurial's normal tagging capabilities
bos@558:       make dealing with patches even more of a breeze.</para>
bos@558:     <itemizedlist>
bos@558:       <listitem><para>Want to patchbomb a mailing list with your
bos@558: 	  latest series of changes?</para>
bos@559: 	<programlisting>hg email qbase:qtip
bos@559: 	</programlisting>
bos@559: 	<para>  (Don't know what <quote>patchbombing</quote> is?  See
dongsheng@625: 	  section <xref linkend="sec.hgext.patchbomb"/>.)</para>
bos@559:       </listitem>
bos@558:       <listitem><para>Need to see all of the patches since
bos@558: 	  <literal>foo.patch</literal> that have touched files in a
bos@558: 	  subdirectory of your tree?</para>
bos@558: 	<programlisting>
bos@558: 	  hg log -r foo.patch:qtip <emphasis>subdir</emphasis>
bos@559: 	</programlisting>
bos@559:       </listitem>
bos@558:     </itemizedlist>
bos@558: 
bos@558:     <para>Because MQ makes the names of patches available to the rest
bos@558:       of Mercurial through its normal internal tag machinery, you
bos@558:       don't need to type in the entire name of a patch when you want
bos@558:       to identify it by name.</para>
bos@558: 
bos@558:     <para>Another nice consequence of representing patch names as tags
bos@558:       is that when you run the <command role="hg-cmd">hg log</command>
bos@558:       command, it will display a patch's name as a tag, simply as part
bos@558:       of its normal output.  This makes it easy to visually
bos@558:       distinguish applied patches from underlying
bos@559:       <quote>normal</quote> revisions.  The following example shows a
bos@559:       few normal Mercurial commands in use with applied
bos@559:       patches.</para>
bos@559: 
bos@567: &interaction.mq.id.output;
bos@558: 
bos@558:   </sect1>
bos@558:   <sect1>
bos@558:     <title>Useful things to know about</title>
bos@558: 
bos@558:     <para>There are a number of aspects of MQ usage that don't fit
bos@558:       tidily into sections of their own, but that are good to know.
bos@558:       Here they are, in one place.</para>
bos@558: 
bos@558:     <itemizedlist>
bos@558:       <listitem><para>Normally, when you <command
bos@559: 	    role="hg-ext-mq">qpop</command> a patch and <command
bos@559: 	    role="hg-ext-mq">qpush</command> it again, the changeset
bos@558: 	  that represents the patch after the pop/push will have a
bos@558: 	  <emphasis>different identity</emphasis> than the changeset
bos@558: 	  that represented the hash beforehand.  See section <xref
dongsheng@625: 	    linkend="sec.mqref.cmd.qpush"/> for
bos@559: 	  information as to why this is.</para>
bos@559:       </listitem>
bos@558:       <listitem><para>It's not a good idea to <command
bos@558: 	    role="hg-cmd">hg merge</command> changes from another
bos@558: 	  branch with a patch changeset, at least if you want to
bos@558: 	  maintain the <quote>patchiness</quote> of that changeset and
bos@558: 	  changesets below it on the patch stack.  If you try to do
bos@558: 	  this, it will appear to succeed, but MQ will become
bos@559: 	  confused.</para>
bos@559:       </listitem></itemizedlist>
bos@559: 
bos@559:   </sect1>
dongsheng@625:   <sect1 id="sec.mq.repo">
bos@558:     <title>Managing patches in a repository</title>
bos@558: 
bos@558:     <para>Because MQ's <filename role="special"
bos@558: 	class="directory">.hg/patches</filename> directory resides
bos@558:       outside a Mercurial repository's working directory, the
bos@558:       <quote>underlying</quote> Mercurial repository knows nothing
bos@558:       about the management or presence of patches.</para>
bos@558: 
bos@558:     <para>This presents the interesting possibility of managing the
bos@558:       contents of the patch directory as a Mercurial repository in its
bos@558:       own right.  This can be a useful way to work.  For example, you
bos@558:       can work on a patch for a while, <command
bos@559: 	role="hg-ext-mq">qrefresh</command> it, then <command
bos@558: 	role="hg-cmd">hg commit</command> the current state of the
bos@558:       patch.  This lets you <quote>roll back</quote> to that version
bos@558:       of the patch later on.</para>
bos@558: 
bos@558:     <para>You can then share different versions of the same patch
bos@558:       stack among multiple underlying repositories.  I use this when I
bos@558:       am developing a Linux kernel feature.  I have a pristine copy of
bos@558:       my kernel sources for each of several CPU architectures, and a
bos@558:       cloned repository under each that contains the patches I am
bos@558:       working on.  When I want to test a change on a different
bos@558:       architecture, I push my current patches to the patch repository
bos@558:       associated with that kernel tree, pop and push all of my
bos@558:       patches, and build and test that kernel.</para>
bos@558: 
bos@558:     <para>Managing patches in a repository makes it possible for
bos@558:       multiple developers to work on the same patch series without
bos@558:       colliding with each other, all on top of an underlying source
bos@558:       base that they may or may not control.</para>
bos@558: 
bos@558:     <sect2>
bos@558:       <title>MQ support for patch repositories</title>
bos@558: 
bos@558:       <para>MQ helps you to work with the <filename role="special"
bos@558: 	  class="directory">.hg/patches</filename> directory as a
bos@558: 	repository; when you prepare a repository for working with
bos@559: 	patches using <command role="hg-ext-mq">qinit</command>, you
bos@559: 	can pass the <option role="hg-ext-mq-cmd-qinit-opt">hg
bos@559: 	  -c</option> option to create the <filename role="special"
bos@558: 	  class="directory">.hg/patches</filename> directory as a
bos@558: 	Mercurial repository.</para>
bos@558: 
bos@558:       <note>
bos@558: 	<para>  If you forget to use the <option
bos@559: 	    role="hg-ext-mq-cmd-qinit-opt">hg -c</option> option, you
bos@559: 	  can simply go into the <filename role="special"
bos@558: 	    class="directory">.hg/patches</filename> directory at any
bos@559: 	  time and run <command role="hg-cmd">hg init</command>.
bos@559: 	  Don't forget to add an entry for the <filename
bos@558: 	    role="special">status</filename> file to the <filename
bos@558: 	    role="special">.hgignore</filename> file, though</para>
bos@558: 
bos@558: 	<para>  (<command role="hg-cmd">hg qinit <option
bos@559: 	      role="hg-ext-mq-cmd-qinit-opt">hg -c</option></command>
bos@558: 	  does this for you automatically); you
bos@558: 	  <emphasis>really</emphasis> don't want to manage the
bos@558: 	  <filename role="special">status</filename> file.</para>
bos@558:       </note>
bos@558: 
bos@558:       <para>As a convenience, if MQ notices that the <filename
bos@558: 	  class="directory">.hg/patches</filename> directory is a
bos@558: 	repository, it will automatically <command role="hg-cmd">hg
bos@558: 	  add</command> every patch that you create and import.</para>
bos@558: 
bos@558:       <para>MQ provides a shortcut command, <command
bos@559: 	  role="hg-ext-mq">qcommit</command>, that runs <command
bos@558: 	  role="hg-cmd">hg commit</command> in the <filename
bos@558: 	  role="special" class="directory">.hg/patches</filename>
bos@558: 	directory.  This saves some bothersome typing.</para>
bos@558: 
bos@558:       <para>Finally, as a convenience to manage the patch directory,
bos@558: 	you can define the alias <command>mq</command> on Unix
bos@558: 	systems. For example, on Linux systems using the
bos@558: 	<command>bash</command> shell, you can include the following
bos@559: 	snippet in your <filename
bos@559: 	  role="home">~/.bashrc</filename>.</para>
bos@559: 
bos@559:       <programlisting>alias mq=`hg -R $(hg
bos@559: 	root)/.hg/patches'</programlisting>
bos@558: 
bos@558:       <para>You can then issue commands of the form <command>mq
bos@558: 	  pull</command> from the main repository.</para>
bos@558: 
bos@558:     </sect2>
bos@558:     <sect2>
bos@558:       <title>A few things to watch out for</title>
bos@558: 
bos@558:       <para>MQ's support for working with a repository full of patches
bos@558: 	is limited in a few small respects.</para>
bos@558: 
bos@558:       <para>MQ cannot automatically detect changes that you make to
bos@558: 	the patch directory.  If you <command role="hg-cmd">hg
bos@558: 	  pull</command>, manually edit, or <command role="hg-cmd">hg
bos@558: 	  update</command> changes to patches or the <filename
bos@558: 	  role="special">series</filename> file, you will have to
bos@558: 	<command role="hg-cmd">hg qpop <option
bos@559: 	    role="hg-ext-mq-cmd-qpop-opt">hg -a</option></command> and
bos@558: 	then <command role="hg-cmd">hg qpush <option
bos@559: 	    role="hg-ext-mq-cmd-qpush-opt">hg -a</option></command> in
bos@558: 	the underlying repository to see those changes show up there.
bos@558: 	If you forget to do this, you can confuse MQ's idea of which
bos@558: 	patches are applied.</para>
bos@558: 
bos@558:     </sect2>
bos@558:   </sect1>
dongsheng@625:   <sect1 id="sec.mq.tools">
bos@558:     <title>Third party tools for working with patches</title>
bos@558: 
bos@558:     <para>Once you've been working with patches for a while, you'll
bos@558:       find yourself hungry for tools that will help you to understand
bos@558:       and manipulate the patches you're dealing with.</para>
bos@558: 
bos@558:     <para>The <command>diffstat</command> command
bos@558:       <citation>web:diffstat</citation> generates a histogram of the
bos@558:       modifications made to each file in a patch.  It provides a good
bos@559:       way to <quote>get a sense of</quote> a patch&emdash;which files
bos@559:       it affects, and how much change it introduces to each file and
bos@559:       as a whole.  (I find that it's a good idea to use
bos@558:       <command>diffstat</command>'s <option
bos@558: 	role="cmd-opt-diffstat">-p</option> option as a matter of
bos@558:       course, as otherwise it will try to do clever things with
bos@558:       prefixes of file names that inevitably confuse at least
bos@558:       me.)</para>
bos@558: 
bos@567: &interaction.mq.tools.tools;
bos@558: 
bos@558:     <para>The <literal role="package">patchutils</literal> package
bos@558:       <citation>web:patchutils</citation> is invaluable. It provides a
bos@558:       set of small utilities that follow the <quote>Unix
bos@558: 	philosophy;</quote> each does one useful thing with a patch.
bos@558:       The <literal role="package">patchutils</literal> command I use
bos@558:       most is <command>filterdiff</command>, which extracts subsets
bos@558:       from a patch file.  For example, given a patch that modifies
bos@558:       hundreds of files across dozens of directories, a single
bos@558:       invocation of <command>filterdiff</command> can generate a
bos@558:       smaller patch that only touches files whose names match a
bos@558:       particular glob pattern.  See section <xref
dongsheng@625: 	linkend="mq-collab.tips.interdiff"/> for another
bos@558:       example.</para>
bos@558: 
bos@558:   </sect1>
bos@558:   <sect1>
bos@558:     <title>Good ways to work with patches</title>
bos@558: 
bos@558:     <para>Whether you are working on a patch series to submit to a
bos@558:       free software or open source project, or a series that you
bos@558:       intend to treat as a sequence of regular changesets when you're
bos@558:       done, you can use some simple techniques to keep your work well
bos@558:       organised.</para>
bos@558: 
bos@558:     <para>Give your patches descriptive names.  A good name for a
bos@558:       patch might be <filename>rework-device-alloc.patch</filename>,
bos@558:       because it will immediately give you a hint what the purpose of
bos@558:       the patch is.  Long names shouldn't be a problem; you won't be
bos@558:       typing the names often, but you <emphasis>will</emphasis> be
bos@558:       running commands like <command
bos@559: 	role="hg-ext-mq">qapplied</command> and <command
bos@559: 	role="hg-ext-mq">qtop</command> over and over. Good naming
bos@558:       becomes especially important when you have a number of patches
bos@558:       to work with, or if you are juggling a number of different tasks
bos@558:       and your patches only get a fraction of your attention.</para>
bos@558: 
bos@558:     <para>Be aware of what patch you're working on.  Use the <command
bos@559: 	role="hg-ext-mq">qtop</command> command and skim over the text
bos@558:       of your patches frequently&emdash;for example, using <command
bos@558: 	role="hg-cmd">hg tip <option
bos@559: 	  role="hg-opt-tip">-p</option></command>)&emdash;to be sure
bos@559:       of where you stand.  I have several times worked on and <command
bos@559: 	role="hg-ext-mq">qrefresh</command>ed a patch other than the
bos@558:       one I intended, and it's often tricky to migrate changes into
bos@558:       the right patch after making them in the wrong one.</para>
bos@558: 
bos@558:     <para>For this reason, it is very much worth investing a little
bos@558:       time to learn how to use some of the third-party tools I
dongsheng@625:       described in section <xref linkend="sec.mq.tools"/>,
bos@559:       particularly
bos@559:       <command>diffstat</command> and <command>filterdiff</command>.
bos@559:       The former will give you a quick idea of what changes your patch
bos@559:       is making, while the latter makes it easy to splice hunks
bos@559:       selectively out of one patch and into another.</para>
bos@558: 
bos@558:   </sect1>
bos@558:   <sect1>
bos@558:     <title>MQ cookbook</title>
bos@558: 
bos@558:     <sect2>
bos@558:       <title>Manage <quote>trivial</quote> patches</title>
bos@558: 
bos@558:       <para>Because the overhead of dropping files into a new
bos@558: 	Mercurial repository is so low, it makes a lot of sense to
bos@558: 	manage patches this way even if you simply want to make a few
bos@558: 	changes to a source tarball that you downloaded.</para>
bos@558: 
bos@558:       <para>Begin by downloading and unpacking the source tarball, and
bos@567: 	turning it into a Mercurial repository.</para>
bos@567: 
bos@567:       &interaction.mq.tarball.download;
bos@558: 
bos@558:       <para>Continue by creating a patch stack and making your
bos@567: 	changes.</para>
bos@567: 
bos@567:       &interaction.mq.tarball.qinit;
bos@558: 
bos@558:       <para>Let's say a few weeks or months pass, and your package
bos@558: 	author releases a new version.  First, bring their changes
bos@567: 	into the repository.</para>
bos@567: 
bos@567:       &interaction.mq.tarball.newsource;
bos@567: 
bos@567:       <para>The pipeline starting with <command role="hg-cmd">hg
bos@558: 	  locate</command> above deletes all files in the working
bos@558: 	directory, so that <command role="hg-cmd">hg
bos@558: 	  commit</command>'s <option
bos@558: 	  role="hg-opt-commit">--addremove</option> option can
bos@558: 	actually tell which files have really been removed in the
bos@558: 	newer version of the source.</para>
bos@558: 
bos@558:       <para>Finally, you can apply your patches on top of the new
bos@567: 	tree.</para>
bos@567: 
bos@567:       &interaction.mq.tarball.repush;
bos@558: 
bos@558:     </sect2>
dongsheng@625:     <sect2 id="sec.mq.combine">
bos@558:       <title>Combining entire patches</title>
bos@558: 
bos@558:       <para>MQ provides a command, <command
bos@559: 	  role="hg-ext-mq">qfold</command> that lets you combine
bos@558: 	entire patches.  This <quote>folds</quote> the patches you
bos@558: 	name, in the order you name them, into the topmost applied
bos@558: 	patch, and concatenates their descriptions onto the end of its
bos@558: 	description.  The patches that you fold must be unapplied
bos@558: 	before you fold them.</para>
bos@558: 
bos@558:       <para>The order in which you fold patches matters.  If your
bos@558: 	topmost applied patch is <literal>foo</literal>, and you
bos@559: 	<command role="hg-ext-mq">qfold</command>
bos@558: 	<literal>bar</literal> and <literal>quux</literal> into it,
bos@558: 	you will end up with a patch that has the same effect as if
bos@558: 	you applied first <literal>foo</literal>, then
bos@558: 	<literal>bar</literal>, followed by
bos@558: 	<literal>quux</literal>.</para>
bos@558: 
bos@558:     </sect2>
bos@558:     <sect2>
bos@558:       <title>Merging part of one patch into another</title>
bos@558: 
bos@558:       <para>Merging <emphasis>part</emphasis> of one patch into
bos@558: 	another is more difficult than combining entire
bos@558: 	patches.</para>
bos@558: 
bos@558:       <para>If you want to move changes to entire files, you can use
bos@558: 	<command>filterdiff</command>'s <option
bos@558: 	  role="cmd-opt-filterdiff">-i</option> and <option
bos@558: 	  role="cmd-opt-filterdiff">-x</option> options to choose the
bos@558: 	modifications to snip out of one patch, concatenating its
bos@558: 	output onto the end of the patch you want to merge into.  You
bos@558: 	usually won't need to modify the patch you've merged the
bos@558: 	changes from.  Instead, MQ will report some rejected hunks
bos@559: 	when you <command role="hg-ext-mq">qpush</command> it (from
bos@558: 	the hunks you moved into the other patch), and you can simply
bos@559: 	<command role="hg-ext-mq">qrefresh</command> the patch to drop
bos@558: 	the duplicate hunks.</para>
bos@558: 
bos@558:       <para>If you have a patch that has multiple hunks modifying a
bos@558: 	file, and you only want to move a few of those hunks, the job
bos@558: 	becomes more messy, but you can still partly automate it.  Use
bos@558: 	<command>lsdiff -nvv</command> to print some metadata about
bos@567: 	the patch.</para>
bos@567: 
bos@567:       &interaction.mq.tools.lsdiff;
bos@558: 
bos@558:       <para>This command prints three different kinds of
bos@558: 	number:</para>
bos@558:       <itemizedlist>
bos@558: 	<listitem><para>(in the first column) a <emphasis>file
bos@558: 	      number</emphasis> to identify each file modified in the
bos@559: 	    patch;</para>
bos@559: 	</listitem>
bos@558: 	<listitem><para>(on the next line, indented) the line number
bos@559: 	    within a modified file where a hunk starts; and</para>
bos@559: 	</listitem>
bos@558: 	<listitem><para>(on the same line) a <emphasis>hunk
bos@559: 	      number</emphasis> to identify that hunk.</para>
bos@559: 	</listitem></itemizedlist>
bos@558: 
bos@558:       <para>You'll have to use some visual inspection, and reading of
bos@558: 	the patch, to identify the file and hunk numbers you'll want,
bos@558: 	but you can then pass them to to
bos@558: 	<command>filterdiff</command>'s <option
bos@558: 	  role="cmd-opt-filterdiff">--files</option> and <option
bos@558: 	  role="cmd-opt-filterdiff">--hunks</option> options, to
bos@558: 	select exactly the file and hunk you want to extract.</para>
bos@558: 
bos@558:       <para>Once you have this hunk, you can concatenate it onto the
bos@558: 	end of your destination patch and continue with the remainder
dongsheng@625: 	of section <xref linkend="sec.mq.combine"/>.</para>
bos@558: 
bos@558:     </sect2>
bos@558:   </sect1>
bos@558:   <sect1>
bos@558:     <title>Differences between quilt and MQ</title>
bos@558: 
bos@558:     <para>If you are already familiar with quilt, MQ provides a
bos@558:       similar command set.  There are a few differences in the way
bos@558:       that it works.</para>
bos@558: 
bos@558:     <para>You will already have noticed that most quilt commands have
bos@558:       MQ counterparts that simply begin with a
bos@558:       <quote><literal>q</literal></quote>.  The exceptions are quilt's
bos@558:       <literal>add</literal> and <literal>remove</literal> commands,
bos@558:       the counterparts for which are the normal Mercurial <command
bos@558: 	role="hg-cmd">hg add</command> and <command role="hg-cmd">hg
bos@558: 	remove</command> commands.  There is no MQ equivalent of the
bos@558:       quilt <literal>edit</literal> command.</para>
bos@558: 
bos@558:   </sect1>
bos@558: </chapter>
bos@558: 
bos@558: <!--
bos@558: local variables: 
bos@558: sgml-parent-document: ("00book.xml" "book" "chapter")
bos@558: end:
bos@558: -->