hgbook

annotate en/ch12-mq.xml @ 559:b90b024729f1

WIP DocBook snapshot that all compiles. Mirabile dictu!
author Bryan O'Sullivan <bos@serpentine.com>
date Wed Feb 18 00:22:09 2009 -0800 (2009-02-18)
parents 8631da51309b
children 8fcd44708f41
rev   line source
bos@558 1 <!-- vim: set filetype=docbkxml shiftwidth=2 autoindent expandtab tw=77 : -->
bos@558 2
bos@558 3 <chapter id="chap:mq">
bos@558 4 <title>Managing change with Mercurial Queues</title>
bos@558 5
bos@558 6 <sect1 id="sec:mq:patch-mgmt">
bos@558 7 <title>The patch management problem</title>
bos@558 8
bos@558 9 <para>Here is a common scenario: you need to install a software
bos@558 10 package from source, but you find a bug that you must fix in the
bos@558 11 source before you can start using the package. You make your
bos@558 12 changes, forget about the package for a while, and a few months
bos@558 13 later you need to upgrade to a newer version of the package. If
bos@558 14 the newer version of the package still has the bug, you must
bos@558 15 extract your fix from the older source tree and apply it against
bos@558 16 the newer version. This is a tedious task, and it's easy to
bos@558 17 make mistakes.</para>
bos@558 18
bos@558 19 <para>This is a simple case of the <quote>patch management</quote>
bos@558 20 problem. You have an <quote>upstream</quote> source tree that
bos@558 21 you can't change; you need to make some local changes on top of
bos@558 22 the upstream tree; and you'd like to be able to keep those
bos@558 23 changes separate, so that you can apply them to newer versions
bos@558 24 of the upstream source.</para>
bos@558 25
bos@558 26 <para>The patch management problem arises in many situations.
bos@558 27 Probably the most visible is that a user of an open source
bos@558 28 software project will contribute a bug fix or new feature to the
bos@558 29 project's maintainers in the form of a patch.</para>
bos@558 30
bos@558 31 <para>Distributors of operating systems that include open source
bos@558 32 software often need to make changes to the packages they
bos@558 33 distribute so that they will build properly in their
bos@558 34 environments.</para>
bos@558 35
bos@558 36 <para>When you have few changes to maintain, it is easy to manage
bos@558 37 a single patch using the standard <command>diff</command> and
bos@558 38 <command>patch</command> programs (see section <xref
bos@559 39 linkend="sec:mq:patch"/> for a discussion of these
bos@559 40 tools). Once the number of changes grows, it starts to make
bos@559 41 sense to maintain patches as discrete <quote>chunks of
bos@559 42 work,</quote> so that for example a single patch will contain
bos@559 43 only one bug fix (the patch might modify several files, but it's
bos@559 44 doing <quote>only one thing</quote>), and you may have a number
bos@559 45 of such patches for different bugs you need fixed and local
bos@559 46 changes you require. In this situation, if you submit a bug fix
bos@559 47 patch to the upstream maintainers of a package and they include
bos@559 48 your fix in a subsequent release, you can simply drop that
bos@559 49 single patch when you're updating to the newer release.</para>
bos@558 50
bos@558 51 <para>Maintaining a single patch against an upstream tree is a
bos@558 52 little tedious and error-prone, but not difficult. However, the
bos@558 53 complexity of the problem grows rapidly as the number of patches
bos@558 54 you have to maintain increases. With more than a tiny number of
bos@558 55 patches in hand, understanding which ones you have applied and
bos@558 56 maintaining them moves from messy to overwhelming.</para>
bos@558 57
bos@558 58 <para>Fortunately, Mercurial includes a powerful extension,
bos@558 59 Mercurial Queues (or simply <quote>MQ</quote>), that massively
bos@558 60 simplifies the patch management problem.</para>
bos@558 61
bos@558 62 </sect1>
bos@558 63 <sect1 id="sec:mq:history">
bos@558 64 <title>The prehistory of Mercurial Queues</title>
bos@558 65
bos@558 66 <para>During the late 1990s, several Linux kernel developers
bos@558 67 started to maintain <quote>patch series</quote> that modified
bos@558 68 the behaviour of the Linux kernel. Some of these series were
bos@558 69 focused on stability, some on feature coverage, and others were
bos@558 70 more speculative.</para>
bos@558 71
bos@558 72 <para>The sizes of these patch series grew rapidly. In 2002,
bos@558 73 Andrew Morton published some shell scripts he had been using to
bos@558 74 automate the task of managing his patch queues. Andrew was
bos@558 75 successfully using these scripts to manage hundreds (sometimes
bos@558 76 thousands) of patches on top of the Linux kernel.</para>
bos@558 77
bos@558 78 <sect2 id="sec:mq:quilt">
bos@558 79 <title>A patchwork quilt</title>
bos@558 80
bos@558 81 <para>In early 2003, Andreas Gruenbacher and Martin Quinson
bos@558 82 borrowed the approach of Andrew's scripts and published a tool
bos@558 83 called <quote>patchwork quilt</quote>
bos@558 84 <citation>web:quilt</citation>, or simply <quote>quilt</quote>
bos@558 85 (see <citation>gruenbacher:2005</citation> for a paper
bos@558 86 describing it). Because quilt substantially automated patch
bos@558 87 management, it rapidly gained a large following among open
bos@558 88 source software developers.</para>
bos@558 89
bos@558 90 <para>Quilt manages a <emphasis>stack of patches</emphasis> on
bos@558 91 top of a directory tree. To begin, you tell quilt to manage a
bos@558 92 directory tree, and tell it which files you want to manage; it
bos@558 93 stores away the names and contents of those files. To fix a
bos@558 94 bug, you create a new patch (using a single command), edit the
bos@558 95 files you need to fix, then <quote>refresh</quote> the
bos@558 96 patch.</para>
bos@558 97
bos@558 98 <para>The refresh step causes quilt to scan the directory tree;
bos@558 99 it updates the patch with all of the changes you have made.
bos@558 100 You can create another patch on top of the first, which will
bos@558 101 track the changes required to modify the tree from <quote>tree
bos@558 102 with one patch applied</quote> to <quote>tree with two
bos@558 103 patches applied</quote>.</para>
bos@558 104
bos@558 105 <para>You can <emphasis>change</emphasis> which patches are
bos@558 106 applied to the tree. If you <quote>pop</quote> a patch, the
bos@558 107 changes made by that patch will vanish from the directory
bos@558 108 tree. Quilt remembers which patches you have popped, though,
bos@558 109 so you can <quote>push</quote> a popped patch again, and the
bos@558 110 directory tree will be restored to contain the modifications
bos@558 111 in the patch. Most importantly, you can run the
bos@558 112 <quote>refresh</quote> command at any time, and the topmost
bos@558 113 applied patch will be updated. This means that you can, at
bos@558 114 any time, change both which patches are applied and what
bos@558 115 modifications those patches make.</para>
bos@558 116
bos@558 117 <para>Quilt knows nothing about revision control tools, so it
bos@558 118 works equally well on top of an unpacked tarball or a
bos@558 119 Subversion working copy.</para>
bos@558 120
bos@558 121 </sect2>
bos@558 122 <sect2 id="sec:mq:quilt-mq">
bos@558 123 <title>From patchwork quilt to Mercurial Queues</title>
bos@558 124
bos@558 125 <para>In mid-2005, Chris Mason took the features of quilt and
bos@558 126 wrote an extension that he called Mercurial Queues, which
bos@558 127 added quilt-like behaviour to Mercurial.</para>
bos@558 128
bos@558 129 <para>The key difference between quilt and MQ is that quilt
bos@558 130 knows nothing about revision control systems, while MQ is
bos@558 131 <emphasis>integrated</emphasis> into Mercurial. Each patch
bos@558 132 that you push is represented as a Mercurial changeset. Pop a
bos@558 133 patch, and the changeset goes away.</para>
bos@558 134
bos@558 135 <para>Because quilt does not care about revision control tools,
bos@558 136 it is still a tremendously useful piece of software to know
bos@558 137 about for situations where you cannot use Mercurial and
bos@558 138 MQ.</para>
bos@558 139
bos@558 140 </sect2>
bos@558 141 </sect1>
bos@558 142 <sect1>
bos@558 143 <title>The huge advantage of MQ</title>
bos@558 144
bos@558 145 <para>I cannot overstate the value that MQ offers through the
bos@558 146 unification of patches and revision control.</para>
bos@558 147
bos@558 148 <para>A major reason that patches have persisted in the free
bos@559 149 software and open source world&emdash;in spite of the
bos@559 150 availability of increasingly capable revision control tools over
bos@559 151 the years&emdash;is the <emphasis>agility</emphasis> they
bos@559 152 offer.</para>
bos@558 153
bos@558 154 <para>Traditional revision control tools make a permanent,
bos@558 155 irreversible record of everything that you do. While this has
bos@558 156 great value, it's also somewhat stifling. If you want to
bos@558 157 perform a wild-eyed experiment, you have to be careful in how
bos@558 158 you go about it, or you risk leaving unneeded&emdash;or worse,
bos@559 159 misleading or destabilising&emdash;traces of your missteps and
bos@559 160 errors in the permanent revision record.</para>
bos@558 161
bos@558 162 <para>By contrast, MQ's marriage of distributed revision control
bos@558 163 with patches makes it much easier to isolate your work. Your
bos@558 164 patches live on top of normal revision history, and you can make
bos@558 165 them disappear or reappear at will. If you don't like a patch,
bos@558 166 you can drop it. If a patch isn't quite as you want it to be,
bos@559 167 simply fix it&emdash;as many times as you need to, until you
bos@559 168 have refined it into the form you desire.</para>
bos@558 169
bos@558 170 <para>As an example, the integration of patches with revision
bos@558 171 control makes understanding patches and debugging their
bos@558 172 effects&emdash;and their interplay with the code they're based
bos@559 173 on&emdash;<emphasis>enormously</emphasis> easier. Since every
bos@559 174 applied patch has an associated changeset, you can give <command
bos@559 175 role="hg-cmd">hg log</command> a file name to see which
bos@559 176 changesets and patches affected the file. You can use the
bos@559 177 <command role="hg-cmd">hg bisect</command> command to
bos@559 178 binary-search through all changesets and applied patches to see
bos@559 179 where a bug got introduced or fixed. You can use the <command
bos@558 180 role="hg-cmd">hg annotate</command> command to see which
bos@558 181 changeset or patch modified a particular line of a source file.
bos@558 182 And so on.</para>
bos@558 183
bos@558 184 </sect1>
bos@558 185 <sect1 id="sec:mq:patch">
bos@558 186 <title>Understanding patches</title>
bos@558 187
bos@558 188 <para>Because MQ doesn't hide its patch-oriented nature, it is
bos@558 189 helpful to understand what patches are, and a little about the
bos@558 190 tools that work with them.</para>
bos@558 191
bos@558 192 <para>The traditional Unix <command>diff</command> command
bos@558 193 compares two files, and prints a list of differences between
bos@558 194 them. The <command>patch</command> command understands these
bos@558 195 differences as <emphasis>modifications</emphasis> to make to a
bos@558 196 file. Take a look below for a simple example of these commands
bos@558 197 in action.</para>
bos@558 198
bos@558 199 <!-- &interaction.mq.dodiff.diff; -->
bos@558 200
bos@558 201 <para>The type of file that <command>diff</command> generates (and
bos@558 202 <command>patch</command> takes as input) is called a
bos@558 203 <quote>patch</quote> or a <quote>diff</quote>; there is no
bos@558 204 difference between a patch and a diff. (We'll use the term
bos@558 205 <quote>patch</quote>, since it's more commonly used.)</para>
bos@558 206
bos@558 207 <para>A patch file can start with arbitrary text; the
bos@558 208 <command>patch</command> command ignores this text, but MQ uses
bos@558 209 it as the commit message when creating changesets. To find the
bos@558 210 beginning of the patch content, <command>patch</command>
bos@558 211 searches for the first line that starts with the string
bos@558 212 <quote><literal>diff -</literal></quote>.</para>
bos@558 213
bos@558 214 <para>MQ works with <emphasis>unified</emphasis> diffs
bos@558 215 (<command>patch</command> can accept several other diff formats,
bos@558 216 but MQ doesn't). A unified diff contains two kinds of header.
bos@558 217 The <emphasis>file header</emphasis> describes the file being
bos@558 218 modified; it contains the name of the file to modify. When
bos@558 219 <command>patch</command> sees a new file header, it looks for a
bos@558 220 file with that name to start modifying.</para>
bos@558 221
bos@558 222 <para>After the file header comes a series of
bos@558 223 <emphasis>hunks</emphasis>. Each hunk starts with a header;
bos@558 224 this identifies the range of line numbers within the file that
bos@558 225 the hunk should modify. Following the header, a hunk starts and
bos@558 226 ends with a few (usually three) lines of text from the
bos@558 227 unmodified file; these are called the
bos@558 228 <emphasis>context</emphasis> for the hunk. If there's only a
bos@558 229 small amount of context between successive hunks,
bos@558 230 <command>diff</command> doesn't print a new hunk header; it just
bos@558 231 runs the hunks together, with a few lines of context between
bos@558 232 modifications.</para>
bos@558 233
bos@558 234 <para>Each line of context begins with a space character. Within
bos@558 235 the hunk, a line that begins with
bos@558 236 <quote><literal>-</literal></quote> means <quote>remove this
bos@558 237 line,</quote> while a line that begins with
bos@558 238 <quote><literal>+</literal></quote> means <quote>insert this
bos@558 239 line.</quote> For example, a line that is modified is
bos@558 240 represented by one deletion and one insertion.</para>
bos@558 241
bos@558 242 <para>We will return to some of the more subtle aspects of patches
bos@558 243 later (in section <xref linkend="sec:mq:adv-patch"/>), but you
bos@559 244 should have
bos@559 245 enough information now to use MQ.</para>
bos@558 246
bos@558 247 </sect1>
bos@558 248 <sect1 id="sec:mq:start">
bos@558 249 <title>Getting started with Mercurial Queues</title>
bos@558 250
bos@558 251 <para>Because MQ is implemented as an extension, you must
bos@558 252 explicitly enable before you can use it. (You don't need to
bos@558 253 download anything; MQ ships with the standard Mercurial
bos@559 254 distribution.) To enable MQ, edit your <filename
bos@559 255 role="home">~/.hgrc</filename> file, and add the lines
bos@559 256 below.</para>
bos@558 257
bos@558 258 <programlisting>[extensions] hgext.mq =</programlisting>
bos@558 259
bos@558 260 <para>Once the extension is enabled, it will make a number of new
bos@558 261 commands available. To verify that the extension is working,
bos@558 262 you can use <command role="hg-cmd">hg help</command> to see if
bos@559 263 the <command role="hg-ext-mq">qinit</command> command is now
bos@558 264 available.</para>
bos@558 265
bos@558 266 <!-- &interaction.mq.qinit-help.help; -->
bos@558 267
bos@558 268 <para>You can use MQ with <emphasis>any</emphasis> Mercurial
bos@558 269 repository, and its commands only operate within that
bos@558 270 repository. To get started, simply prepare the repository using
bos@559 271 the <command role="hg-ext-mq">qinit</command> command.</para>
bos@559 272
bos@559 273 <!-- &interaction.mq.tutorial.qinit; -->
bos@559 274
bos@559 275 <para>This command creates an empty directory called <filename
bos@558 276 role="special" class="directory">.hg/patches</filename>, where
bos@558 277 MQ will keep its metadata. As with many Mercurial commands, the
bos@559 278 <command role="hg-ext-mq">qinit</command> command prints nothing
bos@558 279 if it succeeds.</para>
bos@558 280
bos@558 281 <sect2>
bos@558 282 <title>Creating a new patch</title>
bos@558 283
bos@558 284 <para>To begin work on a new patch, use the <command
bos@559 285 role="hg-ext-mq">qnew</command> command. This command takes
bos@559 286 one argument, the name of the patch to create.</para>
bos@559 287
bos@559 288 <para>MQ will use this as the name of an actual file in the
bos@559 289 <filename role="special"
bos@559 290 class="directory">.hg/patches</filename> directory, as you
bos@559 291 can see below.</para>
bos@559 292
bos@559 293 <!-- &interaction.mq.tutorial.qnew; -->
bos@558 294
bos@558 295 <para>Also newly present in the <filename role="special"
bos@558 296 class="directory">.hg/patches</filename> directory are two
bos@558 297 other files, <filename role="special">series</filename> and
bos@558 298 <filename role="special">status</filename>. The <filename
bos@558 299 role="special">series</filename> file lists all of the
bos@558 300 patches that MQ knows about for this repository, with one
bos@558 301 patch per line. Mercurial uses the <filename
bos@558 302 role="special">status</filename> file for internal
bos@558 303 book-keeping; it tracks all of the patches that MQ has
bos@558 304 <emphasis>applied</emphasis> in this repository.</para>
bos@558 305
bos@558 306 <note>
bos@558 307 <para> You may sometimes want to edit the <filename
bos@558 308 role="special">series</filename> file by hand; for
bos@558 309 example, to change the sequence in which some patches are
bos@558 310 applied. However, manually editing the <filename
bos@558 311 role="special">status</filename> file is almost always a
bos@558 312 bad idea, as it's easy to corrupt MQ's idea of what is
bos@558 313 happening.</para>
bos@558 314 </note>
bos@558 315
bos@558 316 <para>Once you have created your new patch, you can edit files
bos@558 317 in the working directory as you usually would. All of the
bos@558 318 normal Mercurial commands, such as <command role="hg-cmd">hg
bos@558 319 diff</command> and <command role="hg-cmd">hg
bos@558 320 annotate</command>, work exactly as they did before.</para>
bos@558 321
bos@558 322 </sect2>
bos@558 323 <sect2>
bos@558 324 <title>Refreshing a patch</title>
bos@558 325
bos@558 326 <para>When you reach a point where you want to save your work,
bos@559 327 use the <command role="hg-ext-mq">qrefresh</command> command
bos@559 328 to update the patch you are working on.</para>
bos@558 329
bos@558 330 <!-- &interaction.mq.tutorial.qrefresh; -->
bos@558 331
bos@559 332 <para>This command folds the changes you have made in the
bos@559 333 working directory into your patch, and updates its
bos@559 334 corresponding changeset to contain those changes.</para>
bos@559 335
bos@559 336 <para>You can run <command role="hg-ext-mq">qrefresh</command>
bos@559 337 as often as you like, so it's a good way to
bos@559 338 <quote>checkpoint</quote> your work. Refresh your patch at an
bos@559 339 opportune time; try an experiment; and if the experiment
bos@558 340 doesn't work out, <command role="hg-cmd">hg revert</command>
bos@558 341 your modifications back to the last time you refreshed.</para>
bos@558 342
bos@558 343 <!-- &interaction.mq.tutorial.qrefresh2; -->
bos@558 344
bos@558 345 </sect2>
bos@558 346 <sect2>
bos@558 347 <title>Stacking and tracking patches</title>
bos@558 348
bos@558 349 <para>Once you have finished working on a patch, or need to work
bos@559 350 on another, you can use the <command
bos@559 351 role="hg-ext-mq">qnew</command> command again to create a
bos@559 352 new patch. Mercurial will apply this patch on top of your
bos@559 353 existing patch.</para>
bos@559 354
bos@559 355 <!-- &interaction.mq.tutorial.qnew2; -->
bos@559 356 <para>Notice that the patch contains the changes in our prior
bos@559 357 patch as part of its context (you can see this more clearly in
bos@559 358 the output of <command role="hg-cmd">hg
bos@558 359 annotate</command>).</para>
bos@558 360
bos@558 361 <para>So far, with the exception of <command
bos@559 362 role="hg-ext-mq">qnew</command> and <command
bos@559 363 role="hg-ext-mq">qrefresh</command>, we've been careful to
bos@558 364 only use regular Mercurial commands. However, MQ provides
bos@558 365 many commands that are easier to use when you are thinking
bos@559 366 about patches, as illustrated below.</para>
bos@559 367
bos@559 368 <!-- &interaction.mq.tutorial.qseries; -->
bos@558 369
bos@558 370 <itemizedlist>
bos@558 371 <listitem><para>The <command
bos@559 372 role="hg-ext-mq">qseries</command> command lists every
bos@558 373 patch that MQ knows about in this repository, from oldest
bos@558 374 to newest (most recently
bos@559 375 <emphasis>created</emphasis>).</para>
bos@559 376 </listitem>
bos@558 377 <listitem><para>The <command
bos@559 378 role="hg-ext-mq">qapplied</command> command lists every
bos@558 379 patch that MQ has <emphasis>applied</emphasis> in this
bos@558 380 repository, again from oldest to newest (most recently
bos@559 381 applied).</para>
bos@559 382 </listitem></itemizedlist>
bos@558 383
bos@558 384 </sect2>
bos@558 385 <sect2>
bos@558 386 <title>Manipulating the patch stack</title>
bos@558 387
bos@558 388 <para>The previous discussion implied that there must be a
bos@558 389 difference between <quote>known</quote> and
bos@558 390 <quote>applied</quote> patches, and there is. MQ can manage a
bos@558 391 patch without it being applied in the repository.</para>
bos@558 392
bos@558 393 <para>An <emphasis>applied</emphasis> patch has a corresponding
bos@558 394 changeset in the repository, and the effects of the patch and
bos@558 395 changeset are visible in the working directory. You can undo
bos@558 396 the application of a patch using the <command
bos@559 397 role="hg-ext-mq">qpop</command> command. MQ still
bos@558 398 <emphasis>knows about</emphasis>, or manages, a popped patch,
bos@558 399 but the patch no longer has a corresponding changeset in the
bos@558 400 repository, and the working directory does not contain the
bos@558 401 changes made by the patch. Figure <xref
bos@559 402 linkend="fig:mq:stack"/> illustrates
bos@559 403 the difference between applied and tracked patches.</para>
bos@558 404
bos@558 405 <informalfigure id="fig:mq:stack">
bos@559 406 <mediaobject><imageobject><imagedata
bos@559 407 fileref="mq-stack"/></imageobject><textobject><phrase>XXX
bos@559 408 add text</phrase></textobject><caption><para>Applied and
bos@559 409 unapplied patches in the MQ patch
bos@559 410 stack</para></caption></mediaobject>
bos@558 411 </informalfigure>
bos@558 412
bos@558 413 <para>You can reapply an unapplied, or popped, patch using the
bos@559 414 <command role="hg-ext-mq">qpush</command> command. This
bos@558 415 creates a new changeset to correspond to the patch, and the
bos@558 416 patch's changes once again become present in the working
bos@559 417 directory. See below for examples of <command
bos@559 418 role="hg-ext-mq">qpop</command> and <command
bos@559 419 role="hg-ext-mq">qpush</command> in action.</para>
bos@558 420 <!-- &interaction.mq.tutorial.qpop; -->
bos@558 421
bos@559 422 <para>Notice that once we have popped a patch or two patches,
bos@559 423 the output of <command role="hg-ext-mq">qseries</command>
bos@559 424 remains the same, while that of <command
bos@559 425 role="hg-ext-mq">qapplied</command> has changed.</para>
bos@559 426
bos@559 427
bos@558 428 </sect2>
bos@558 429 <sect2>
bos@558 430 <title>Pushing and popping many patches</title>
bos@558 431
bos@559 432 <para>While <command role="hg-ext-mq">qpush</command> and
bos@559 433 <command role="hg-ext-mq">qpop</command> each operate on a
bos@558 434 single patch at a time by default, you can push and pop many
bos@558 435 patches in one go. The <option
bos@559 436 role="hg-ext-mq-cmd-qpush-opt">hg -a</option> option to
bos@559 437 <command role="hg-ext-mq">qpush</command> causes it to push
bos@558 438 all unapplied patches, while the <option
bos@558 439 role="hg-ext-mq-cmd-qpop-opt">-a</option> option to <command
bos@559 440 role="hg-ext-mq">qpop</command> causes it to pop all applied
bos@558 441 patches. (For some more ways to push and pop many patches,
bos@559 442 see section <xref linkend="sec:mq:perf"/>
bos@559 443 below.)</para>
bos@558 444
bos@558 445 <!-- &interaction.mq.tutorial.qpush-a; -->
bos@558 446
bos@558 447 </sect2>
bos@558 448 <sect2>
bos@558 449 <title>Safety checks, and overriding them</title>
bos@558 450
bos@558 451 <para>Several MQ commands check the working directory before
bos@558 452 they do anything, and fail if they find any modifications.
bos@558 453 They do this to ensure that you won't lose any changes that
bos@559 454 you have made, but not yet incorporated into a patch. The
bos@559 455 example below illustrates this; the <command
bos@559 456 role="hg-ext-mq">qnew</command> command will not create a
bos@558 457 new patch if there are outstanding changes, caused in this
bos@558 458 case by the <command role="hg-cmd">hg add</command> of
bos@558 459 <filename>file3</filename>.</para>
bos@558 460
bos@558 461 <!-- &interaction.mq.tutorial.add; -->
bos@558 462
bos@558 463 <para>Commands that check the working directory all take an
bos@558 464 <quote>I know what I'm doing</quote> option, which is always
bos@558 465 named <option>-f</option>. The exact meaning of
bos@558 466 <option>-f</option> depends on the command. For example,
bos@558 467 <command role="hg-cmd">hg qnew <option
bos@559 468 role="hg-ext-mq-cmd-qnew-opt">hg -f</option></command>
bos@559 469 will incorporate any outstanding changes into the new patch it
bos@558 470 creates, but <command role="hg-cmd">hg qpop <option
bos@559 471 role="hg-ext-mq-cmd-qpop-opt">hg -f</option></command>
bos@559 472 will revert modifications to any files affected by the patch
bos@559 473 that it is popping. Be sure to read the documentation for a
bos@558 474 command's <option>-f</option> option before you use it!</para>
bos@558 475
bos@558 476 </sect2>
bos@558 477 <sect2>
bos@558 478 <title>Working on several patches at once</title>
bos@558 479
bos@559 480 <para>The <command role="hg-ext-mq">qrefresh</command> command
bos@558 481 always refreshes the <emphasis>topmost</emphasis> applied
bos@558 482 patch. This means that you can suspend work on one patch (by
bos@558 483 refreshing it), pop or push to make a different patch the top,
bos@558 484 and work on <emphasis>that</emphasis> patch for a
bos@558 485 while.</para>
bos@558 486
bos@558 487 <para>Here's an example that illustrates how you can use this
bos@558 488 ability. Let's say you're developing a new feature as two
bos@558 489 patches. The first is a change to the core of your software,
bos@559 490 and the second&emdash;layered on top of the
bos@559 491 first&emdash;changes the user interface to use the code you
bos@559 492 just added to the core. If you notice a bug in the core while
bos@559 493 you're working on the UI patch, it's easy to fix the core.
bos@559 494 Simply <command role="hg-ext-mq">qrefresh</command> the UI
bos@559 495 patch to save your in-progress changes, and <command
bos@559 496 role="hg-ext-mq">qpop</command> down to the core patch. Fix
bos@559 497 the core bug, <command role="hg-ext-mq">qrefresh</command> the
bos@559 498 core patch, and <command role="hg-ext-mq">qpush</command> back
bos@558 499 to the UI patch to continue where you left off.</para>
bos@558 500
bos@558 501 </sect2>
bos@558 502 </sect1>
bos@558 503 <sect1 id="sec:mq:adv-patch">
bos@558 504 <title>More about patches</title>
bos@558 505
bos@558 506 <para>MQ uses the GNU <command>patch</command> command to apply
bos@558 507 patches, so it's helpful to know a few more detailed aspects of
bos@558 508 how <command>patch</command> works, and about patches
bos@558 509 themselves.</para>
bos@558 510
bos@558 511 <sect2>
bos@558 512 <title>The strip count</title>
bos@558 513
bos@558 514 <para>If you look at the file headers in a patch, you will
bos@558 515 notice that the pathnames usually have an extra component on
bos@558 516 the front that isn't present in the actual path name. This is
bos@558 517 a holdover from the way that people used to generate patches
bos@558 518 (people still do this, but it's somewhat rare with modern
bos@558 519 revision control tools).</para>
bos@558 520
bos@558 521 <para>Alice would unpack a tarball, edit her files, then decide
bos@558 522 that she wanted to create a patch. So she'd rename her
bos@558 523 working directory, unpack the tarball again (hence the need
bos@558 524 for the rename), and use the <option
bos@558 525 role="cmd-opt-diff">-r</option> and <option
bos@558 526 role="cmd-opt-diff">-N</option> options to
bos@558 527 <command>diff</command> to recursively generate a patch
bos@558 528 between the unmodified directory and the modified one. The
bos@558 529 result would be that the name of the unmodified directory
bos@558 530 would be at the front of the left-hand path in every file
bos@558 531 header, and the name of the modified directory would be at the
bos@558 532 front of the right-hand path.</para>
bos@558 533
bos@558 534 <para>Since someone receiving a patch from the Alices of the net
bos@558 535 would be unlikely to have unmodified and modified directories
bos@558 536 with exactly the same names, the <command>patch</command>
bos@558 537 command has a <option role="cmd-opt-patch">-p</option> option
bos@558 538 that indicates the number of leading path name components to
bos@558 539 strip when trying to apply a patch. This number is called the
bos@558 540 <emphasis>strip count</emphasis>.</para>
bos@558 541
bos@558 542 <para>An option of <quote><literal>-p1</literal></quote> means
bos@558 543 <quote>use a strip count of one</quote>. If
bos@558 544 <command>patch</command> sees a file name
bos@558 545 <filename>foo/bar/baz</filename> in a file header, it will
bos@558 546 strip <filename>foo</filename> and try to patch a file named
bos@558 547 <filename>bar/baz</filename>. (Strictly speaking, the strip
bos@558 548 count refers to the number of <emphasis>path
bos@558 549 separators</emphasis> (and the components that go with them
bos@558 550 ) to strip. A strip count of one will turn
bos@558 551 <filename>foo/bar</filename> into <filename>bar</filename>,
bos@558 552 but <filename>/foo/bar</filename> (notice the extra leading
bos@558 553 slash) into <filename>foo/bar</filename>.)</para>
bos@558 554
bos@558 555 <para>The <quote>standard</quote> strip count for patches is
bos@558 556 one; almost all patches contain one leading path name
bos@558 557 component that needs to be stripped. Mercurial's <command
bos@558 558 role="hg-cmd">hg diff</command> command generates path names
bos@558 559 in this form, and the <command role="hg-cmd">hg
bos@558 560 import</command> command and MQ expect patches to have a
bos@558 561 strip count of one.</para>
bos@558 562
bos@558 563 <para>If you receive a patch from someone that you want to add
bos@558 564 to your patch queue, and the patch needs a strip count other
bos@558 565 than one, you cannot just <command
bos@559 566 role="hg-ext-mq">qimport</command> the patch, because
bos@559 567 <command role="hg-ext-mq">qimport</command> does not yet have
bos@558 568 a <literal>-p</literal> option (see <ulink role="hg-bug"
bos@558 569 url="http://www.selenic.com/mercurial/bts/issue311">issue
bos@558 570 311</ulink>). Your best bet is to <command
bos@559 571 role="hg-ext-mq">qnew</command> a patch of your own, then
bos@558 572 use <command>patch -pN</command> to apply their patch,
bos@558 573 followed by <command role="hg-cmd">hg addremove</command> to
bos@558 574 pick up any files added or removed by the patch, followed by
bos@559 575 <command role="hg-ext-mq">hg qrefresh</command>. This
bos@559 576 complexity may become unnecessary; see <ulink role="hg-bug"
bos@558 577 url="http://www.selenic.com/mercurial/bts/issue311">issue
bos@559 578 311</ulink> for details.
bos@559 579 </para>
bos@558 580 </sect2>
bos@558 581 <sect2>
bos@558 582 <title>Strategies for applying a patch</title>
bos@558 583
bos@558 584 <para>When <command>patch</command> applies a hunk, it tries a
bos@558 585 handful of successively less accurate strategies to try to
bos@558 586 make the hunk apply. This falling-back technique often makes
bos@558 587 it possible to take a patch that was generated against an old
bos@558 588 version of a file, and apply it against a newer version of
bos@558 589 that file.</para>
bos@558 590
bos@558 591 <para>First, <command>patch</command> tries an exact match,
bos@558 592 where the line numbers, the context, and the text to be
bos@558 593 modified must apply exactly. If it cannot make an exact
bos@558 594 match, it tries to find an exact match for the context,
bos@558 595 without honouring the line numbering information. If this
bos@558 596 succeeds, it prints a line of output saying that the hunk was
bos@558 597 applied, but at some <emphasis>offset</emphasis> from the
bos@558 598 original line number.</para>
bos@558 599
bos@558 600 <para>If a context-only match fails, <command>patch</command>
bos@558 601 removes the first and last lines of the context, and tries a
bos@558 602 <emphasis>reduced</emphasis> context-only match. If the hunk
bos@558 603 with reduced context succeeds, it prints a message saying that
bos@558 604 it applied the hunk with a <emphasis>fuzz factor</emphasis>
bos@558 605 (the number after the fuzz factor indicates how many lines of
bos@558 606 context <command>patch</command> had to trim before the patch
bos@558 607 applied).</para>
bos@558 608
bos@558 609 <para>When neither of these techniques works,
bos@558 610 <command>patch</command> prints a message saying that the hunk
bos@558 611 in question was rejected. It saves rejected hunks (also
bos@558 612 simply called <quote>rejects</quote>) to a file with the same
bos@558 613 name, and an added <filename role="special">.rej</filename>
bos@558 614 extension. It also saves an unmodified copy of the file with
bos@558 615 a <filename role="special">.orig</filename> extension; the
bos@558 616 copy of the file without any extensions will contain any
bos@558 617 changes made by hunks that <emphasis>did</emphasis> apply
bos@558 618 cleanly. If you have a patch that modifies
bos@558 619 <filename>foo</filename> with six hunks, and one of them fails
bos@558 620 to apply, you will have: an unmodified
bos@558 621 <filename>foo.orig</filename>, a <filename>foo.rej</filename>
bos@558 622 containing one hunk, and <filename>foo</filename>, containing
bos@558 623 the changes made by the five successful hunks.</para>
bos@558 624
bos@558 625 </sect2>
bos@558 626 <sect2>
bos@558 627 <title>Some quirks of patch representation</title>
bos@558 628
bos@558 629 <para>There are a few useful things to know about how
bos@558 630 <command>patch</command> works with files.</para>
bos@558 631 <itemizedlist>
bos@558 632 <listitem><para>This should already be obvious, but
bos@558 633 <command>patch</command> cannot handle binary
bos@559 634 files.</para>
bos@559 635 </listitem>
bos@558 636 <listitem><para>Neither does it care about the executable bit;
bos@558 637 it creates new files as readable, but not
bos@559 638 executable.</para>
bos@559 639 </listitem>
bos@558 640 <listitem><para><command>patch</command> treats the removal of
bos@558 641 a file as a diff between the file to be removed and the
bos@558 642 empty file. So your idea of <quote>I deleted this
bos@558 643 file</quote> looks like <quote>every line of this file
bos@559 644 was deleted</quote> in a patch.</para>
bos@559 645 </listitem>
bos@558 646 <listitem><para>It treats the addition of a file as a diff
bos@558 647 between the empty file and the file to be added. So in a
bos@558 648 patch, your idea of <quote>I added this file</quote> looks
bos@558 649 like <quote>every line of this file was
bos@559 650 added</quote>.</para>
bos@559 651 </listitem>
bos@558 652 <listitem><para>It treats a renamed file as the removal of the
bos@558 653 old name, and the addition of the new name. This means
bos@558 654 that renamed files have a big footprint in patches. (Note
bos@558 655 also that Mercurial does not currently try to infer when
bos@559 656 files have been renamed or copied in a patch.)</para>
bos@559 657 </listitem>
bos@558 658 <listitem><para><command>patch</command> cannot represent
bos@558 659 empty files, so you cannot use a patch to represent the
bos@558 660 notion <quote>I added this empty file to the
bos@559 661 tree</quote>.</para>
bos@559 662 </listitem></itemizedlist>
bos@559 663 </sect2>
bos@558 664 <sect2>
bos@558 665 <title>Beware the fuzz</title>
bos@558 666
bos@558 667 <para>While applying a hunk at an offset, or with a fuzz factor,
bos@558 668 will often be completely successful, these inexact techniques
bos@558 669 naturally leave open the possibility of corrupting the patched
bos@558 670 file. The most common cases typically involve applying a
bos@558 671 patch twice, or at an incorrect location in the file. If
bos@558 672 <command>patch</command> or <command
bos@559 673 role="hg-ext-mq">qpush</command> ever mentions an offset or
bos@558 674 fuzz factor, you should make sure that the modified files are
bos@558 675 correct afterwards.</para>
bos@558 676
bos@558 677 <para>It's often a good idea to refresh a patch that has applied
bos@558 678 with an offset or fuzz factor; refreshing the patch generates
bos@558 679 new context information that will make it apply cleanly. I
bos@558 680 say <quote>often,</quote> not <quote>always,</quote> because
bos@558 681 sometimes refreshing a patch will make it fail to apply
bos@558 682 against a different revision of the underlying files. In some
bos@558 683 cases, such as when you're maintaining a patch that must sit
bos@558 684 on top of multiple versions of a source tree, it's acceptable
bos@558 685 to have a patch apply with some fuzz, provided you've verified
bos@558 686 the results of the patching process in such cases.</para>
bos@558 687
bos@558 688 </sect2>
bos@558 689 <sect2>
bos@558 690 <title>Handling rejection</title>
bos@558 691
bos@559 692 <para>If <command role="hg-ext-mq">qpush</command> fails to
bos@558 693 apply a patch, it will print an error message and exit. If it
bos@558 694 has left <filename role="special">.rej</filename> files
bos@558 695 behind, it is usually best to fix up the rejected hunks before
bos@558 696 you push more patches or do any further work.</para>
bos@558 697
bos@558 698 <para>If your patch <emphasis>used to</emphasis> apply cleanly,
bos@558 699 and no longer does because you've changed the underlying code
bos@558 700 that your patches are based on, Mercurial Queues can help; see
bos@559 701 section <xref
bos@559 702 linkend="sec:mq:merge"/> for details.</para>
bos@558 703
bos@558 704 <para>Unfortunately, there aren't any great techniques for
bos@558 705 dealing with rejected hunks. Most often, you'll need to view
bos@558 706 the <filename role="special">.rej</filename> file and edit the
bos@558 707 target file, applying the rejected hunks by hand.</para>
bos@558 708
bos@558 709 <para>If you're feeling adventurous, Neil Brown, a Linux kernel
bos@558 710 hacker, wrote a tool called <command>wiggle</command>
bos@558 711 <citation>web:wiggle</citation>, which is more vigorous than
bos@558 712 <command>patch</command> in its attempts to make a patch
bos@558 713 apply.</para>
bos@558 714
bos@558 715 <para>Another Linux kernel hacker, Chris Mason (the author of
bos@558 716 Mercurial Queues), wrote a similar tool called
bos@558 717 <command>mpatch</command> <citation>web:mpatch</citation>,
bos@558 718 which takes a simple approach to automating the application of
bos@558 719 hunks rejected by <command>patch</command>. The
bos@558 720 <command>mpatch</command> command can help with four common
bos@558 721 reasons that a hunk may be rejected:</para>
bos@558 722
bos@558 723 <itemizedlist>
bos@558 724 <listitem><para>The context in the middle of a hunk has
bos@559 725 changed.</para>
bos@559 726 </listitem>
bos@558 727 <listitem><para>A hunk is missing some context at the
bos@559 728 beginning or end.</para>
bos@559 729 </listitem>
bos@558 730 <listitem><para>A large hunk might apply better&emdash;either
bos@559 731 entirely or in part&emdash;if it was broken up into
bos@559 732 smaller hunks.</para>
bos@559 733 </listitem>
bos@558 734 <listitem><para>A hunk removes lines with slightly different
bos@559 735 content than those currently present in the file.</para>
bos@559 736 </listitem></itemizedlist>
bos@558 737
bos@558 738 <para>If you use <command>wiggle</command> or
bos@558 739 <command>mpatch</command>, you should be doubly careful to
bos@558 740 check your results when you're done. In fact,
bos@558 741 <command>mpatch</command> enforces this method of
bos@558 742 double-checking the tool's output, by automatically dropping
bos@558 743 you into a merge program when it has done its job, so that you
bos@558 744 can verify its work and finish off any remaining
bos@558 745 merges.</para>
bos@558 746
bos@558 747 </sect2>
bos@558 748 </sect1>
bos@558 749 <sect1 id="sec:mq:perf">
bos@558 750 <title>Getting the best performance out of MQ</title>
bos@558 751
bos@558 752 <para>MQ is very efficient at handling a large number of patches.
bos@558 753 I ran some performance experiments in mid-2006 for a talk that I
bos@558 754 gave at the 2006 EuroPython conference
bos@558 755 <citation>web:europython</citation>. I used as my data set the
bos@558 756 Linux 2.6.17-mm1 patch series, which consists of 1,738 patches.
bos@558 757 I applied these on top of a Linux kernel repository containing
bos@558 758 all 27,472 revisions between Linux 2.6.12-rc2 and Linux
bos@558 759 2.6.17.</para>
bos@558 760
bos@558 761 <para>On my old, slow laptop, I was able to <command
bos@558 762 role="hg-cmd">hg qpush <option
bos@559 763 role="hg-ext-mq-cmd-qpush-opt">hg -a</option></command> all
bos@558 764 1,738 patches in 3.5 minutes, and <command role="hg-cmd">hg qpop
bos@559 765 <option role="hg-ext-mq-cmd-qpop-opt">hg -a</option></command>
bos@558 766 them all in 30 seconds. (On a newer laptop, the time to push
bos@558 767 all patches dropped to two minutes.) I could <command
bos@559 768 role="hg-ext-mq">qrefresh</command> one of the biggest patches
bos@558 769 (which made 22,779 lines of changes to 287 files) in 6.6
bos@558 770 seconds.</para>
bos@558 771
bos@558 772 <para>Clearly, MQ is well suited to working in large trees, but
bos@558 773 there are a few tricks you can use to get the best performance
bos@558 774 of it.</para>
bos@558 775
bos@558 776 <para>First of all, try to <quote>batch</quote> operations
bos@558 777 together. Every time you run <command
bos@559 778 role="hg-ext-mq">qpush</command> or <command
bos@559 779 role="hg-ext-mq">qpop</command>, these commands scan the
bos@558 780 working directory once to make sure you haven't made some
bos@558 781 changes and then forgotten to run <command
bos@559 782 role="hg-ext-mq">qrefresh</command>. On a small tree, the
bos@558 783 time that this scan takes is unnoticeable. However, on a
bos@558 784 medium-sized tree (containing tens of thousands of files), it
bos@558 785 can take a second or more.</para>
bos@558 786
bos@559 787 <para>The <command role="hg-ext-mq">qpush</command> and <command
bos@559 788 role="hg-ext-mq">qpop</command> commands allow you to push and
bos@558 789 pop multiple patches at a time. You can identify the
bos@558 790 <quote>destination patch</quote> that you want to end up at.
bos@559 791 When you <command role="hg-ext-mq">qpush</command> with a
bos@558 792 destination specified, it will push patches until that patch is
bos@558 793 at the top of the applied stack. When you <command
bos@559 794 role="hg-ext-mq">qpop</command> to a destination, MQ will pop
bos@558 795 patches until the destination patch is at the top.</para>
bos@558 796
bos@558 797 <para>You can identify a destination patch using either the name
bos@558 798 of the patch, or by number. If you use numeric addressing,
bos@558 799 patches are counted from zero; this means that the first patch
bos@558 800 is zero, the second is one, and so on.</para>
bos@558 801
bos@558 802 </sect1>
bos@558 803 <sect1 id="sec:mq:merge">
bos@558 804 <title>Updating your patches when the underlying code
bos@558 805 changes</title>
bos@558 806
bos@558 807 <para>It's common to have a stack of patches on top of an
bos@558 808 underlying repository that you don't modify directly. If you're
bos@558 809 working on changes to third-party code, or on a feature that is
bos@558 810 taking longer to develop than the rate of change of the code
bos@558 811 beneath, you will often need to sync up with the underlying
bos@558 812 code, and fix up any hunks in your patches that no longer apply.
bos@558 813 This is called <emphasis>rebasing</emphasis> your patch
bos@558 814 series.</para>
bos@558 815
bos@558 816 <para>The simplest way to do this is to <command role="hg-cmd">hg
bos@559 817 qpop <option role="hg-ext-mq-cmd-qpop-opt">hg
bos@559 818 -a</option></command> your patches, then <command
bos@559 819 role="hg-cmd">hg pull</command> changes into the underlying
bos@559 820 repository, and finally <command role="hg-cmd">hg qpush <option
bos@559 821 role="hg-ext-mq-cmd-qpop-opt">hg -a</option></command> your
bos@558 822 patches again. MQ will stop pushing any time it runs across a
bos@558 823 patch that fails to apply during conflicts, allowing you to fix
bos@559 824 your conflicts, <command role="hg-ext-mq">qrefresh</command> the
bos@558 825 affected patch, and continue pushing until you have fixed your
bos@558 826 entire stack.</para>
bos@558 827
bos@558 828 <para>This approach is easy to use and works well if you don't
bos@558 829 expect changes to the underlying code to affect how well your
bos@558 830 patches apply. If your patch stack touches code that is modified
bos@558 831 frequently or invasively in the underlying repository, however,
bos@558 832 fixing up rejected hunks by hand quickly becomes
bos@558 833 tiresome.</para>
bos@558 834
bos@559 835 <para>It's possible to partially automate the rebasing process.
bos@559 836 If your patches apply cleanly against some revision of the
bos@558 837 underlying repo, MQ can use this information to help you to
bos@558 838 resolve conflicts between your patches and a different
bos@558 839 revision.</para>
bos@558 840
bos@558 841 <para>The process is a little involved.</para>
bos@558 842 <orderedlist>
bos@558 843 <listitem><para>To begin, <command role="hg-cmd">hg qpush
bos@558 844 -a</command> all of your patches on top of the revision
bos@559 845 where you know that they apply cleanly.</para>
bos@559 846 </listitem>
bos@558 847 <listitem><para>Save a backup copy of your patch directory using
bos@558 848 <command role="hg-cmd">hg qsave <option
bos@559 849 role="hg-ext-mq-cmd-qsave-opt">hg -e</option> <option
bos@559 850 role="hg-ext-mq-cmd-qsave-opt">hg -c</option></command>.
bos@558 851 This prints the name of the directory that it has saved the
bos@558 852 patches in. It will save the patches to a directory called
bos@558 853 <filename role="special"
bos@558 854 class="directory">.hg/patches.N</filename>, where
bos@558 855 <literal>N</literal> is a small integer. It also commits a
bos@558 856 <quote>save changeset</quote> on top of your applied
bos@558 857 patches; this is for internal book-keeping, and records the
bos@558 858 states of the <filename role="special">series</filename> and
bos@559 859 <filename role="special">status</filename> files.</para>
bos@559 860 </listitem>
bos@559 861 <listitem><para>Use <command role="hg-cmd">hg pull</command> to
bos@559 862 bring new changes into the underlying repository. (Don't
bos@559 863 run <command role="hg-cmd">hg pull -u</command>; see below
bos@559 864 for why.)</para>
bos@559 865 </listitem>
bos@558 866 <listitem><para>Update to the new tip revision, using <command
bos@558 867 role="hg-cmd">hg update <option
bos@558 868 role="hg-opt-update">-C</option></command> to override
bos@559 869 the patches you have pushed.</para>
bos@559 870 </listitem>
bos@558 871 <listitem><para>Merge all patches using
bos@559 872 \hgcmdargs{qpush}{<option role="hg-ext-mq-cmd-qpush-opt">hg
bos@559 873 -m</option> <option role="hg-ext-mq-cmd-qpush-opt">hg
bos@559 874 -a</option>}. The <option
bos@559 875 role="hg-ext-mq-cmd-qpush-opt">hg -m</option> option to
bos@559 876 <command role="hg-ext-mq">qpush</command> tells MQ to
bos@558 877 perform a three-way merge if the patch fails to
bos@559 878 apply.</para>
bos@559 879 </listitem></orderedlist>
bos@558 880
bos@558 881 <para>During the <command role="hg-cmd">hg qpush <option
bos@559 882 role="hg-ext-mq-cmd-qpush-opt">hg -m</option></command>,
bos@559 883 each patch in the <filename role="special">series</filename>
bos@559 884 file is applied normally. If a patch applies with fuzz or
bos@559 885 rejects, MQ looks at the queue you <command
bos@559 886 role="hg-ext-mq">qsave</command>d, and performs a three-way
bos@558 887 merge with the corresponding changeset. This merge uses
bos@558 888 Mercurial's normal merge machinery, so it may pop up a GUI merge
bos@558 889 tool to help you to resolve problems.</para>
bos@558 890
bos@558 891 <para>When you finish resolving the effects of a patch, MQ
bos@558 892 refreshes your patch based on the result of the merge.</para>
bos@558 893
bos@558 894 <para>At the end of this process, your repository will have one
bos@558 895 extra head from the old patch queue, and a copy of the old patch
bos@558 896 queue will be in <filename role="special"
bos@558 897 class="directory">.hg/patches.N</filename>. You can remove the
bos@559 898 extra head using <command role="hg-cmd">hg qpop -a -n
bos@558 899 patches.N</command> or <command role="hg-cmd">hg
bos@558 900 strip</command>. You can delete <filename role="special"
bos@558 901 class="directory">.hg/patches.N</filename> once you are sure
bos@558 902 that you no longer need it as a backup.</para>
bos@558 903
bos@558 904 </sect1>
bos@558 905 <sect1>
bos@558 906 <title>Identifying patches</title>
bos@558 907
bos@558 908 <para>MQ commands that work with patches let you refer to a patch
bos@558 909 either by using its name or by a number. By name is obvious
bos@558 910 enough; pass the name <filename>foo.patch</filename> to <command
bos@559 911 role="hg-ext-mq">qpush</command>, for example, and it will
bos@558 912 push patches until <filename>foo.patch</filename> is
bos@558 913 applied.</para>
bos@558 914
bos@558 915 <para>As a shortcut, you can refer to a patch using both a name
bos@558 916 and a numeric offset; <literal>foo.patch-2</literal> means
bos@558 917 <quote>two patches before <literal>foo.patch</literal></quote>,
bos@558 918 while <literal>bar.patch+4</literal> means <quote>four patches
bos@558 919 after <literal>bar.patch</literal></quote>.</para>
bos@558 920
bos@558 921 <para>Referring to a patch by index isn't much different. The
bos@558 922 first patch printed in the output of <command
bos@559 923 role="hg-ext-mq">qseries</command> is patch zero (yes, it's
bos@558 924 one of those start-at-zero counting systems); the second is
bos@558 925 patch one; and so on.</para>
bos@558 926
bos@558 927 <para>MQ also makes it easy to work with patches when you are
bos@558 928 using normal Mercurial commands. Every command that accepts a
bos@558 929 changeset ID will also accept the name of an applied patch. MQ
bos@558 930 augments the tags normally in the repository with an eponymous
bos@558 931 one for each applied patch. In addition, the special tags
bos@558 932 \index{tags!special tag
bos@558 933 names!<literal>qbase</literal>}<literal>qbase</literal> and
bos@558 934 \index{tags!special tag
bos@558 935 names!<literal>qtip</literal>}<literal>qtip</literal> identify
bos@558 936 the <quote>bottom-most</quote> and topmost applied patches,
bos@558 937 respectively.</para>
bos@558 938
bos@558 939 <para>These additions to Mercurial's normal tagging capabilities
bos@558 940 make dealing with patches even more of a breeze.</para>
bos@558 941 <itemizedlist>
bos@558 942 <listitem><para>Want to patchbomb a mailing list with your
bos@558 943 latest series of changes?</para>
bos@559 944 <programlisting>hg email qbase:qtip
bos@559 945 </programlisting>
bos@559 946 <para> (Don't know what <quote>patchbombing</quote> is? See
bos@559 947 section <xref linkend="sec:hgext:patchbomb"/>.)</para>
bos@559 948 </listitem>
bos@558 949 <listitem><para>Need to see all of the patches since
bos@558 950 <literal>foo.patch</literal> that have touched files in a
bos@558 951 subdirectory of your tree?</para>
bos@558 952 <programlisting>
bos@558 953 hg log -r foo.patch:qtip <emphasis>subdir</emphasis>
bos@559 954 </programlisting>
bos@559 955 </listitem>
bos@558 956 </itemizedlist>
bos@558 957
bos@558 958 <para>Because MQ makes the names of patches available to the rest
bos@558 959 of Mercurial through its normal internal tag machinery, you
bos@558 960 don't need to type in the entire name of a patch when you want
bos@558 961 to identify it by name.</para>
bos@558 962
bos@558 963 <para>Another nice consequence of representing patch names as tags
bos@558 964 is that when you run the <command role="hg-cmd">hg log</command>
bos@558 965 command, it will display a patch's name as a tag, simply as part
bos@558 966 of its normal output. This makes it easy to visually
bos@558 967 distinguish applied patches from underlying
bos@559 968 <quote>normal</quote> revisions. The following example shows a
bos@559 969 few normal Mercurial commands in use with applied
bos@559 970 patches.</para>
bos@559 971
bos@559 972 <!-- &interaction.mq.id.output; -->
bos@558 973
bos@558 974 </sect1>
bos@558 975 <sect1>
bos@558 976 <title>Useful things to know about</title>
bos@558 977
bos@558 978 <para>There are a number of aspects of MQ usage that don't fit
bos@558 979 tidily into sections of their own, but that are good to know.
bos@558 980 Here they are, in one place.</para>
bos@558 981
bos@558 982 <itemizedlist>
bos@558 983 <listitem><para>Normally, when you <command
bos@559 984 role="hg-ext-mq">qpop</command> a patch and <command
bos@559 985 role="hg-ext-mq">qpush</command> it again, the changeset
bos@558 986 that represents the patch after the pop/push will have a
bos@558 987 <emphasis>different identity</emphasis> than the changeset
bos@558 988 that represented the hash beforehand. See section <xref
bos@559 989 linkend="sec:mqref:cmd:qpush"/> for
bos@559 990 information as to why this is.</para>
bos@559 991 </listitem>
bos@558 992 <listitem><para>It's not a good idea to <command
bos@558 993 role="hg-cmd">hg merge</command> changes from another
bos@558 994 branch with a patch changeset, at least if you want to
bos@558 995 maintain the <quote>patchiness</quote> of that changeset and
bos@558 996 changesets below it on the patch stack. If you try to do
bos@558 997 this, it will appear to succeed, but MQ will become
bos@559 998 confused.</para>
bos@559 999 </listitem></itemizedlist>
bos@559 1000
bos@559 1001 </sect1>
bos@558 1002 <sect1 id="sec:mq:repo">
bos@558 1003 <title>Managing patches in a repository</title>
bos@558 1004
bos@558 1005 <para>Because MQ's <filename role="special"
bos@558 1006 class="directory">.hg/patches</filename> directory resides
bos@558 1007 outside a Mercurial repository's working directory, the
bos@558 1008 <quote>underlying</quote> Mercurial repository knows nothing
bos@558 1009 about the management or presence of patches.</para>
bos@558 1010
bos@558 1011 <para>This presents the interesting possibility of managing the
bos@558 1012 contents of the patch directory as a Mercurial repository in its
bos@558 1013 own right. This can be a useful way to work. For example, you
bos@558 1014 can work on a patch for a while, <command
bos@559 1015 role="hg-ext-mq">qrefresh</command> it, then <command
bos@558 1016 role="hg-cmd">hg commit</command> the current state of the
bos@558 1017 patch. This lets you <quote>roll back</quote> to that version
bos@558 1018 of the patch later on.</para>
bos@558 1019
bos@558 1020 <para>You can then share different versions of the same patch
bos@558 1021 stack among multiple underlying repositories. I use this when I
bos@558 1022 am developing a Linux kernel feature. I have a pristine copy of
bos@558 1023 my kernel sources for each of several CPU architectures, and a
bos@558 1024 cloned repository under each that contains the patches I am
bos@558 1025 working on. When I want to test a change on a different
bos@558 1026 architecture, I push my current patches to the patch repository
bos@558 1027 associated with that kernel tree, pop and push all of my
bos@558 1028 patches, and build and test that kernel.</para>
bos@558 1029
bos@558 1030 <para>Managing patches in a repository makes it possible for
bos@558 1031 multiple developers to work on the same patch series without
bos@558 1032 colliding with each other, all on top of an underlying source
bos@558 1033 base that they may or may not control.</para>
bos@558 1034
bos@558 1035 <sect2>
bos@558 1036 <title>MQ support for patch repositories</title>
bos@558 1037
bos@558 1038 <para>MQ helps you to work with the <filename role="special"
bos@558 1039 class="directory">.hg/patches</filename> directory as a
bos@558 1040 repository; when you prepare a repository for working with
bos@559 1041 patches using <command role="hg-ext-mq">qinit</command>, you
bos@559 1042 can pass the <option role="hg-ext-mq-cmd-qinit-opt">hg
bos@559 1043 -c</option> option to create the <filename role="special"
bos@558 1044 class="directory">.hg/patches</filename> directory as a
bos@558 1045 Mercurial repository.</para>
bos@558 1046
bos@558 1047 <note>
bos@558 1048 <para> If you forget to use the <option
bos@559 1049 role="hg-ext-mq-cmd-qinit-opt">hg -c</option> option, you
bos@559 1050 can simply go into the <filename role="special"
bos@558 1051 class="directory">.hg/patches</filename> directory at any
bos@559 1052 time and run <command role="hg-cmd">hg init</command>.
bos@559 1053 Don't forget to add an entry for the <filename
bos@558 1054 role="special">status</filename> file to the <filename
bos@558 1055 role="special">.hgignore</filename> file, though</para>
bos@558 1056
bos@558 1057 <para> (<command role="hg-cmd">hg qinit <option
bos@559 1058 role="hg-ext-mq-cmd-qinit-opt">hg -c</option></command>
bos@558 1059 does this for you automatically); you
bos@558 1060 <emphasis>really</emphasis> don't want to manage the
bos@558 1061 <filename role="special">status</filename> file.</para>
bos@558 1062 </note>
bos@558 1063
bos@558 1064 <para>As a convenience, if MQ notices that the <filename
bos@558 1065 class="directory">.hg/patches</filename> directory is a
bos@558 1066 repository, it will automatically <command role="hg-cmd">hg
bos@558 1067 add</command> every patch that you create and import.</para>
bos@558 1068
bos@558 1069 <para>MQ provides a shortcut command, <command
bos@559 1070 role="hg-ext-mq">qcommit</command>, that runs <command
bos@558 1071 role="hg-cmd">hg commit</command> in the <filename
bos@558 1072 role="special" class="directory">.hg/patches</filename>
bos@558 1073 directory. This saves some bothersome typing.</para>
bos@558 1074
bos@558 1075 <para>Finally, as a convenience to manage the patch directory,
bos@558 1076 you can define the alias <command>mq</command> on Unix
bos@558 1077 systems. For example, on Linux systems using the
bos@558 1078 <command>bash</command> shell, you can include the following
bos@559 1079 snippet in your <filename
bos@559 1080 role="home">~/.bashrc</filename>.</para>
bos@559 1081
bos@559 1082 <programlisting>alias mq=`hg -R $(hg
bos@559 1083 root)/.hg/patches'</programlisting>
bos@558 1084
bos@558 1085 <para>You can then issue commands of the form <command>mq
bos@558 1086 pull</command> from the main repository.</para>
bos@558 1087
bos@558 1088 </sect2>
bos@558 1089 <sect2>
bos@558 1090 <title>A few things to watch out for</title>
bos@558 1091
bos@558 1092 <para>MQ's support for working with a repository full of patches
bos@558 1093 is limited in a few small respects.</para>
bos@558 1094
bos@558 1095 <para>MQ cannot automatically detect changes that you make to
bos@558 1096 the patch directory. If you <command role="hg-cmd">hg
bos@558 1097 pull</command>, manually edit, or <command role="hg-cmd">hg
bos@558 1098 update</command> changes to patches or the <filename
bos@558 1099 role="special">series</filename> file, you will have to
bos@558 1100 <command role="hg-cmd">hg qpop <option
bos@559 1101 role="hg-ext-mq-cmd-qpop-opt">hg -a</option></command> and
bos@558 1102 then <command role="hg-cmd">hg qpush <option
bos@559 1103 role="hg-ext-mq-cmd-qpush-opt">hg -a</option></command> in
bos@558 1104 the underlying repository to see those changes show up there.
bos@558 1105 If you forget to do this, you can confuse MQ's idea of which
bos@558 1106 patches are applied.</para>
bos@558 1107
bos@558 1108 </sect2>
bos@558 1109 </sect1>
bos@558 1110 <sect1 id="sec:mq:tools">
bos@558 1111 <title>Third party tools for working with patches</title>
bos@558 1112
bos@558 1113 <para>Once you've been working with patches for a while, you'll
bos@558 1114 find yourself hungry for tools that will help you to understand
bos@558 1115 and manipulate the patches you're dealing with.</para>
bos@558 1116
bos@558 1117 <para>The <command>diffstat</command> command
bos@558 1118 <citation>web:diffstat</citation> generates a histogram of the
bos@558 1119 modifications made to each file in a patch. It provides a good
bos@559 1120 way to <quote>get a sense of</quote> a patch&emdash;which files
bos@559 1121 it affects, and how much change it introduces to each file and
bos@559 1122 as a whole. (I find that it's a good idea to use
bos@558 1123 <command>diffstat</command>'s <option
bos@558 1124 role="cmd-opt-diffstat">-p</option> option as a matter of
bos@558 1125 course, as otherwise it will try to do clever things with
bos@558 1126 prefixes of file names that inevitably confuse at least
bos@558 1127 me.)</para>
bos@558 1128
bos@558 1129 <!-- &interaction.mq.tools.tools; -->
bos@558 1130
bos@558 1131 <para>The <literal role="package">patchutils</literal> package
bos@558 1132 <citation>web:patchutils</citation> is invaluable. It provides a
bos@558 1133 set of small utilities that follow the <quote>Unix
bos@558 1134 philosophy;</quote> each does one useful thing with a patch.
bos@558 1135 The <literal role="package">patchutils</literal> command I use
bos@558 1136 most is <command>filterdiff</command>, which extracts subsets
bos@558 1137 from a patch file. For example, given a patch that modifies
bos@558 1138 hundreds of files across dozens of directories, a single
bos@558 1139 invocation of <command>filterdiff</command> can generate a
bos@558 1140 smaller patch that only touches files whose names match a
bos@558 1141 particular glob pattern. See section <xref
bos@558 1142 linkend="mq-collab:tips:interdiff"/> for another
bos@558 1143 example.</para>
bos@558 1144
bos@558 1145 </sect1>
bos@558 1146 <sect1>
bos@558 1147 <title>Good ways to work with patches</title>
bos@558 1148
bos@558 1149 <para>Whether you are working on a patch series to submit to a
bos@558 1150 free software or open source project, or a series that you
bos@558 1151 intend to treat as a sequence of regular changesets when you're
bos@558 1152 done, you can use some simple techniques to keep your work well
bos@558 1153 organised.</para>
bos@558 1154
bos@558 1155 <para>Give your patches descriptive names. A good name for a
bos@558 1156 patch might be <filename>rework-device-alloc.patch</filename>,
bos@558 1157 because it will immediately give you a hint what the purpose of
bos@558 1158 the patch is. Long names shouldn't be a problem; you won't be
bos@558 1159 typing the names often, but you <emphasis>will</emphasis> be
bos@558 1160 running commands like <command
bos@559 1161 role="hg-ext-mq">qapplied</command> and <command
bos@559 1162 role="hg-ext-mq">qtop</command> over and over. Good naming
bos@558 1163 becomes especially important when you have a number of patches
bos@558 1164 to work with, or if you are juggling a number of different tasks
bos@558 1165 and your patches only get a fraction of your attention.</para>
bos@558 1166
bos@558 1167 <para>Be aware of what patch you're working on. Use the <command
bos@559 1168 role="hg-ext-mq">qtop</command> command and skim over the text
bos@558 1169 of your patches frequently&emdash;for example, using <command
bos@558 1170 role="hg-cmd">hg tip <option
bos@559 1171 role="hg-opt-tip">-p</option></command>)&emdash;to be sure
bos@559 1172 of where you stand. I have several times worked on and <command
bos@559 1173 role="hg-ext-mq">qrefresh</command>ed a patch other than the
bos@558 1174 one I intended, and it's often tricky to migrate changes into
bos@558 1175 the right patch after making them in the wrong one.</para>
bos@558 1176
bos@558 1177 <para>For this reason, it is very much worth investing a little
bos@558 1178 time to learn how to use some of the third-party tools I
bos@558 1179 described in section <xref linkend="sec:mq:tools"/>,
bos@559 1180 particularly
bos@559 1181 <command>diffstat</command> and <command>filterdiff</command>.
bos@559 1182 The former will give you a quick idea of what changes your patch
bos@559 1183 is making, while the latter makes it easy to splice hunks
bos@559 1184 selectively out of one patch and into another.</para>
bos@558 1185
bos@558 1186 </sect1>
bos@558 1187 <sect1>
bos@558 1188 <title>MQ cookbook</title>
bos@558 1189
bos@558 1190 <sect2>
bos@558 1191 <title>Manage <quote>trivial</quote> patches</title>
bos@558 1192
bos@558 1193 <para>Because the overhead of dropping files into a new
bos@558 1194 Mercurial repository is so low, it makes a lot of sense to
bos@558 1195 manage patches this way even if you simply want to make a few
bos@558 1196 changes to a source tarball that you downloaded.</para>
bos@558 1197
bos@558 1198 <para>Begin by downloading and unpacking the source tarball, and
bos@558 1199 turning it into a Mercurial repository. <!--
bos@558 1200 &interaction.mq.tarball.download; --></para>
bos@558 1201
bos@558 1202 <para>Continue by creating a patch stack and making your
bos@558 1203 changes. <!-- &interaction.mq.tarball.qinit; --></para>
bos@558 1204
bos@558 1205 <para>Let's say a few weeks or months pass, and your package
bos@558 1206 author releases a new version. First, bring their changes
bos@558 1207 into the repository. <!-- &interaction.mq.tarball.newsource;
bos@558 1208 --> The pipeline starting with <command role="hg-cmd">hg
bos@558 1209 locate</command> above deletes all files in the working
bos@558 1210 directory, so that <command role="hg-cmd">hg
bos@558 1211 commit</command>'s <option
bos@558 1212 role="hg-opt-commit">--addremove</option> option can
bos@558 1213 actually tell which files have really been removed in the
bos@558 1214 newer version of the source.</para>
bos@558 1215
bos@558 1216 <para>Finally, you can apply your patches on top of the new
bos@558 1217 tree. <!-- &interaction.mq.tarball.repush; --></para>
bos@558 1218
bos@558 1219 </sect2>
bos@558 1220 <sect2 id="sec:mq:combine">
bos@558 1221 <title>Combining entire patches</title>
bos@558 1222
bos@558 1223 <para>MQ provides a command, <command
bos@559 1224 role="hg-ext-mq">qfold</command> that lets you combine
bos@558 1225 entire patches. This <quote>folds</quote> the patches you
bos@558 1226 name, in the order you name them, into the topmost applied
bos@558 1227 patch, and concatenates their descriptions onto the end of its
bos@558 1228 description. The patches that you fold must be unapplied
bos@558 1229 before you fold them.</para>
bos@558 1230
bos@558 1231 <para>The order in which you fold patches matters. If your
bos@558 1232 topmost applied patch is <literal>foo</literal>, and you
bos@559 1233 <command role="hg-ext-mq">qfold</command>
bos@558 1234 <literal>bar</literal> and <literal>quux</literal> into it,
bos@558 1235 you will end up with a patch that has the same effect as if
bos@558 1236 you applied first <literal>foo</literal>, then
bos@558 1237 <literal>bar</literal>, followed by
bos@558 1238 <literal>quux</literal>.</para>
bos@558 1239
bos@558 1240 </sect2>
bos@558 1241 <sect2>
bos@558 1242 <title>Merging part of one patch into another</title>
bos@558 1243
bos@558 1244 <para>Merging <emphasis>part</emphasis> of one patch into
bos@558 1245 another is more difficult than combining entire
bos@558 1246 patches.</para>
bos@558 1247
bos@558 1248 <para>If you want to move changes to entire files, you can use
bos@558 1249 <command>filterdiff</command>'s <option
bos@558 1250 role="cmd-opt-filterdiff">-i</option> and <option
bos@558 1251 role="cmd-opt-filterdiff">-x</option> options to choose the
bos@558 1252 modifications to snip out of one patch, concatenating its
bos@558 1253 output onto the end of the patch you want to merge into. You
bos@558 1254 usually won't need to modify the patch you've merged the
bos@558 1255 changes from. Instead, MQ will report some rejected hunks
bos@559 1256 when you <command role="hg-ext-mq">qpush</command> it (from
bos@558 1257 the hunks you moved into the other patch), and you can simply
bos@559 1258 <command role="hg-ext-mq">qrefresh</command> the patch to drop
bos@558 1259 the duplicate hunks.</para>
bos@558 1260
bos@558 1261 <para>If you have a patch that has multiple hunks modifying a
bos@558 1262 file, and you only want to move a few of those hunks, the job
bos@558 1263 becomes more messy, but you can still partly automate it. Use
bos@558 1264 <command>lsdiff -nvv</command> to print some metadata about
bos@558 1265 the patch. <!-- &interaction.mq.tools.lsdiff; --></para>
bos@558 1266
bos@558 1267 <para>This command prints three different kinds of
bos@558 1268 number:</para>
bos@558 1269 <itemizedlist>
bos@558 1270 <listitem><para>(in the first column) a <emphasis>file
bos@558 1271 number</emphasis> to identify each file modified in the
bos@559 1272 patch;</para>
bos@559 1273 </listitem>
bos@558 1274 <listitem><para>(on the next line, indented) the line number
bos@559 1275 within a modified file where a hunk starts; and</para>
bos@559 1276 </listitem>
bos@558 1277 <listitem><para>(on the same line) a <emphasis>hunk
bos@559 1278 number</emphasis> to identify that hunk.</para>
bos@559 1279 </listitem></itemizedlist>
bos@558 1280
bos@558 1281 <para>You'll have to use some visual inspection, and reading of
bos@558 1282 the patch, to identify the file and hunk numbers you'll want,
bos@558 1283 but you can then pass them to to
bos@558 1284 <command>filterdiff</command>'s <option
bos@558 1285 role="cmd-opt-filterdiff">--files</option> and <option
bos@558 1286 role="cmd-opt-filterdiff">--hunks</option> options, to
bos@558 1287 select exactly the file and hunk you want to extract.</para>
bos@558 1288
bos@558 1289 <para>Once you have this hunk, you can concatenate it onto the
bos@558 1290 end of your destination patch and continue with the remainder
bos@558 1291 of section <xref linkend="sec:mq:combine"/>.</para>
bos@558 1292
bos@558 1293 </sect2>
bos@558 1294 </sect1>
bos@558 1295 <sect1>
bos@558 1296 <title>Differences between quilt and MQ</title>
bos@558 1297
bos@558 1298 <para>If you are already familiar with quilt, MQ provides a
bos@558 1299 similar command set. There are a few differences in the way
bos@558 1300 that it works.</para>
bos@558 1301
bos@558 1302 <para>You will already have noticed that most quilt commands have
bos@558 1303 MQ counterparts that simply begin with a
bos@558 1304 <quote><literal>q</literal></quote>. The exceptions are quilt's
bos@558 1305 <literal>add</literal> and <literal>remove</literal> commands,
bos@558 1306 the counterparts for which are the normal Mercurial <command
bos@558 1307 role="hg-cmd">hg add</command> and <command role="hg-cmd">hg
bos@558 1308 remove</command> commands. There is no MQ equivalent of the
bos@558 1309 quilt <literal>edit</literal> command.</para>
bos@558 1310
bos@558 1311 </sect1>
bos@558 1312 </chapter>
bos@558 1313
bos@558 1314 <!--
bos@558 1315 local variables:
bos@558 1316 sgml-parent-document: ("00book.xml" "book" "chapter")
bos@558 1317 end:
bos@558 1318 -->