hgbook

annotate en/ch12-mq.xml @ 647:d0160b0b1a9e

Merge with http://hg.serpentine.com/mercurial/book
author Dongsheng Song <dongsheng.song@gmail.com>
date Wed Mar 18 20:32:37 2009 +0800 (2009-03-18)
parents a13813534ccd 8366882f67f2
children
rev   line source
bos@558 1 <!-- vim: set filetype=docbkxml shiftwidth=2 autoindent expandtab tw=77 : -->
bos@558 2
dongsheng@625 3 <chapter id="chap.mq">
bos@572 4 <?dbhtml filename="managing-change-with-mercurial-queues.html"?>
bos@558 5 <title>Managing change with Mercurial Queues</title>
bos@558 6
dongsheng@625 7 <sect1 id="sec.mq.patch-mgmt">
bos@558 8 <title>The patch management problem</title>
bos@558 9
bos@558 10 <para>Here is a common scenario: you need to install a software
bos@558 11 package from source, but you find a bug that you must fix in the
bos@558 12 source before you can start using the package. You make your
bos@558 13 changes, forget about the package for a while, and a few months
bos@558 14 later you need to upgrade to a newer version of the package. If
bos@558 15 the newer version of the package still has the bug, you must
bos@558 16 extract your fix from the older source tree and apply it against
bos@558 17 the newer version. This is a tedious task, and it's easy to
bos@558 18 make mistakes.</para>
bos@558 19
bos@558 20 <para>This is a simple case of the <quote>patch management</quote>
bos@558 21 problem. You have an <quote>upstream</quote> source tree that
bos@558 22 you can't change; you need to make some local changes on top of
bos@558 23 the upstream tree; and you'd like to be able to keep those
bos@558 24 changes separate, so that you can apply them to newer versions
bos@558 25 of the upstream source.</para>
bos@558 26
bos@558 27 <para>The patch management problem arises in many situations.
bos@558 28 Probably the most visible is that a user of an open source
bos@558 29 software project will contribute a bug fix or new feature to the
bos@558 30 project's maintainers in the form of a patch.</para>
bos@558 31
bos@558 32 <para>Distributors of operating systems that include open source
bos@558 33 software often need to make changes to the packages they
bos@558 34 distribute so that they will build properly in their
bos@558 35 environments.</para>
bos@558 36
bos@558 37 <para>When you have few changes to maintain, it is easy to manage
bos@558 38 a single patch using the standard <command>diff</command> and
bos@558 39 <command>patch</command> programs (see section <xref
dongsheng@625 40 linkend="sec.mq.patch"/> for a discussion of these
bos@559 41 tools). Once the number of changes grows, it starts to make
bos@559 42 sense to maintain patches as discrete <quote>chunks of
bos@559 43 work,</quote> so that for example a single patch will contain
bos@559 44 only one bug fix (the patch might modify several files, but it's
bos@559 45 doing <quote>only one thing</quote>), and you may have a number
bos@559 46 of such patches for different bugs you need fixed and local
bos@559 47 changes you require. In this situation, if you submit a bug fix
bos@559 48 patch to the upstream maintainers of a package and they include
bos@559 49 your fix in a subsequent release, you can simply drop that
bos@559 50 single patch when you're updating to the newer release.</para>
bos@558 51
bos@558 52 <para>Maintaining a single patch against an upstream tree is a
bos@558 53 little tedious and error-prone, but not difficult. However, the
bos@558 54 complexity of the problem grows rapidly as the number of patches
bos@558 55 you have to maintain increases. With more than a tiny number of
bos@558 56 patches in hand, understanding which ones you have applied and
bos@558 57 maintaining them moves from messy to overwhelming.</para>
bos@558 58
bos@558 59 <para>Fortunately, Mercurial includes a powerful extension,
bos@558 60 Mercurial Queues (or simply <quote>MQ</quote>), that massively
bos@558 61 simplifies the patch management problem.</para>
bos@558 62
bos@558 63 </sect1>
dongsheng@625 64 <sect1 id="sec.mq.history">
bos@558 65 <title>The prehistory of Mercurial Queues</title>
bos@558 66
bos@558 67 <para>During the late 1990s, several Linux kernel developers
bos@558 68 started to maintain <quote>patch series</quote> that modified
bos@558 69 the behaviour of the Linux kernel. Some of these series were
bos@558 70 focused on stability, some on feature coverage, and others were
bos@558 71 more speculative.</para>
bos@558 72
bos@558 73 <para>The sizes of these patch series grew rapidly. In 2002,
bos@558 74 Andrew Morton published some shell scripts he had been using to
bos@558 75 automate the task of managing his patch queues. Andrew was
bos@558 76 successfully using these scripts to manage hundreds (sometimes
bos@558 77 thousands) of patches on top of the Linux kernel.</para>
bos@558 78
dongsheng@625 79 <sect2 id="sec.mq.quilt">
bos@558 80 <title>A patchwork quilt</title>
bos@558 81
bos@558 82 <para>In early 2003, Andreas Gruenbacher and Martin Quinson
bos@558 83 borrowed the approach of Andrew's scripts and published a tool
bos@558 84 called <quote>patchwork quilt</quote>
bos@558 85 <citation>web:quilt</citation>, or simply <quote>quilt</quote>
bos@558 86 (see <citation>gruenbacher:2005</citation> for a paper
bos@558 87 describing it). Because quilt substantially automated patch
bos@558 88 management, it rapidly gained a large following among open
bos@558 89 source software developers.</para>
bos@558 90
bos@558 91 <para>Quilt manages a <emphasis>stack of patches</emphasis> on
bos@558 92 top of a directory tree. To begin, you tell quilt to manage a
bos@558 93 directory tree, and tell it which files you want to manage; it
bos@558 94 stores away the names and contents of those files. To fix a
bos@558 95 bug, you create a new patch (using a single command), edit the
bos@558 96 files you need to fix, then <quote>refresh</quote> the
bos@558 97 patch.</para>
bos@558 98
bos@558 99 <para>The refresh step causes quilt to scan the directory tree;
bos@558 100 it updates the patch with all of the changes you have made.
bos@558 101 You can create another patch on top of the first, which will
bos@558 102 track the changes required to modify the tree from <quote>tree
bos@558 103 with one patch applied</quote> to <quote>tree with two
bos@558 104 patches applied</quote>.</para>
bos@558 105
bos@558 106 <para>You can <emphasis>change</emphasis> which patches are
bos@558 107 applied to the tree. If you <quote>pop</quote> a patch, the
bos@558 108 changes made by that patch will vanish from the directory
bos@558 109 tree. Quilt remembers which patches you have popped, though,
bos@558 110 so you can <quote>push</quote> a popped patch again, and the
bos@558 111 directory tree will be restored to contain the modifications
bos@558 112 in the patch. Most importantly, you can run the
bos@558 113 <quote>refresh</quote> command at any time, and the topmost
bos@558 114 applied patch will be updated. This means that you can, at
bos@558 115 any time, change both which patches are applied and what
bos@558 116 modifications those patches make.</para>
bos@558 117
bos@558 118 <para>Quilt knows nothing about revision control tools, so it
bos@558 119 works equally well on top of an unpacked tarball or a
bos@558 120 Subversion working copy.</para>
bos@558 121
bos@558 122 </sect2>
dongsheng@625 123 <sect2 id="sec.mq.quilt-mq">
bos@558 124 <title>From patchwork quilt to Mercurial Queues</title>
bos@558 125
bos@558 126 <para>In mid-2005, Chris Mason took the features of quilt and
bos@558 127 wrote an extension that he called Mercurial Queues, which
bos@558 128 added quilt-like behaviour to Mercurial.</para>
bos@558 129
bos@558 130 <para>The key difference between quilt and MQ is that quilt
bos@558 131 knows nothing about revision control systems, while MQ is
bos@558 132 <emphasis>integrated</emphasis> into Mercurial. Each patch
bos@558 133 that you push is represented as a Mercurial changeset. Pop a
bos@558 134 patch, and the changeset goes away.</para>
bos@558 135
bos@558 136 <para>Because quilt does not care about revision control tools,
bos@558 137 it is still a tremendously useful piece of software to know
bos@558 138 about for situations where you cannot use Mercurial and
bos@558 139 MQ.</para>
bos@558 140
bos@558 141 </sect2>
bos@558 142 </sect1>
bos@558 143 <sect1>
bos@558 144 <title>The huge advantage of MQ</title>
bos@558 145
bos@558 146 <para>I cannot overstate the value that MQ offers through the
bos@558 147 unification of patches and revision control.</para>
bos@558 148
bos@558 149 <para>A major reason that patches have persisted in the free
bos@559 150 software and open source world&emdash;in spite of the
bos@559 151 availability of increasingly capable revision control tools over
bos@559 152 the years&emdash;is the <emphasis>agility</emphasis> they
bos@559 153 offer.</para>
bos@558 154
bos@558 155 <para>Traditional revision control tools make a permanent,
bos@558 156 irreversible record of everything that you do. While this has
bos@558 157 great value, it's also somewhat stifling. If you want to
bos@558 158 perform a wild-eyed experiment, you have to be careful in how
bos@558 159 you go about it, or you risk leaving unneeded&emdash;or worse,
bos@559 160 misleading or destabilising&emdash;traces of your missteps and
bos@559 161 errors in the permanent revision record.</para>
bos@558 162
bos@558 163 <para>By contrast, MQ's marriage of distributed revision control
bos@558 164 with patches makes it much easier to isolate your work. Your
bos@558 165 patches live on top of normal revision history, and you can make
bos@558 166 them disappear or reappear at will. If you don't like a patch,
bos@558 167 you can drop it. If a patch isn't quite as you want it to be,
bos@559 168 simply fix it&emdash;as many times as you need to, until you
bos@559 169 have refined it into the form you desire.</para>
bos@558 170
bos@558 171 <para>As an example, the integration of patches with revision
bos@558 172 control makes understanding patches and debugging their
bos@558 173 effects&emdash;and their interplay with the code they're based
bos@559 174 on&emdash;<emphasis>enormously</emphasis> easier. Since every
bos@559 175 applied patch has an associated changeset, you can give <command
bos@559 176 role="hg-cmd">hg log</command> a file name to see which
bos@559 177 changesets and patches affected the file. You can use the
bos@559 178 <command role="hg-cmd">hg bisect</command> command to
bos@559 179 binary-search through all changesets and applied patches to see
bos@559 180 where a bug got introduced or fixed. You can use the <command
bos@558 181 role="hg-cmd">hg annotate</command> command to see which
bos@558 182 changeset or patch modified a particular line of a source file.
bos@558 183 And so on.</para>
bos@558 184
bos@558 185 </sect1>
dongsheng@625 186 <sect1 id="sec.mq.patch">
bos@558 187 <title>Understanding patches</title>
bos@558 188
bos@558 189 <para>Because MQ doesn't hide its patch-oriented nature, it is
bos@558 190 helpful to understand what patches are, and a little about the
bos@558 191 tools that work with them.</para>
bos@558 192
bos@558 193 <para>The traditional Unix <command>diff</command> command
bos@558 194 compares two files, and prints a list of differences between
bos@558 195 them. The <command>patch</command> command understands these
bos@558 196 differences as <emphasis>modifications</emphasis> to make to a
bos@558 197 file. Take a look below for a simple example of these commands
bos@558 198 in action.</para>
bos@558 199
bos@567 200 &interaction.mq.dodiff.diff;
bos@558 201
bos@558 202 <para>The type of file that <command>diff</command> generates (and
bos@558 203 <command>patch</command> takes as input) is called a
bos@558 204 <quote>patch</quote> or a <quote>diff</quote>; there is no
bos@558 205 difference between a patch and a diff. (We'll use the term
bos@558 206 <quote>patch</quote>, since it's more commonly used.)</para>
bos@558 207
bos@558 208 <para>A patch file can start with arbitrary text; the
bos@558 209 <command>patch</command> command ignores this text, but MQ uses
bos@558 210 it as the commit message when creating changesets. To find the
bos@558 211 beginning of the patch content, <command>patch</command>
bos@558 212 searches for the first line that starts with the string
bos@558 213 <quote><literal>diff -</literal></quote>.</para>
bos@558 214
bos@558 215 <para>MQ works with <emphasis>unified</emphasis> diffs
bos@558 216 (<command>patch</command> can accept several other diff formats,
bos@558 217 but MQ doesn't). A unified diff contains two kinds of header.
bos@558 218 The <emphasis>file header</emphasis> describes the file being
bos@558 219 modified; it contains the name of the file to modify. When
bos@558 220 <command>patch</command> sees a new file header, it looks for a
bos@558 221 file with that name to start modifying.</para>
bos@558 222
bos@558 223 <para>After the file header comes a series of
bos@558 224 <emphasis>hunks</emphasis>. Each hunk starts with a header;
bos@558 225 this identifies the range of line numbers within the file that
bos@558 226 the hunk should modify. Following the header, a hunk starts and
bos@558 227 ends with a few (usually three) lines of text from the
bos@558 228 unmodified file; these are called the
bos@558 229 <emphasis>context</emphasis> for the hunk. If there's only a
bos@558 230 small amount of context between successive hunks,
bos@558 231 <command>diff</command> doesn't print a new hunk header; it just
bos@558 232 runs the hunks together, with a few lines of context between
bos@558 233 modifications.</para>
bos@558 234
bos@558 235 <para>Each line of context begins with a space character. Within
bos@558 236 the hunk, a line that begins with
bos@558 237 <quote><literal>-</literal></quote> means <quote>remove this
bos@558 238 line,</quote> while a line that begins with
bos@558 239 <quote><literal>+</literal></quote> means <quote>insert this
bos@558 240 line.</quote> For example, a line that is modified is
bos@558 241 represented by one deletion and one insertion.</para>
bos@558 242
bos@558 243 <para>We will return to some of the more subtle aspects of patches
dongsheng@625 244 later (in section <xref linkend="sec.mq.adv-patch"/>), but you
bos@559 245 should have
bos@559 246 enough information now to use MQ.</para>
bos@558 247
bos@558 248 </sect1>
dongsheng@625 249 <sect1 id="sec.mq.start">
bos@558 250 <title>Getting started with Mercurial Queues</title>
bos@558 251
bos@558 252 <para>Because MQ is implemented as an extension, you must
bos@558 253 explicitly enable before you can use it. (You don't need to
bos@558 254 download anything; MQ ships with the standard Mercurial
bos@559 255 distribution.) To enable MQ, edit your <filename
bos@559 256 role="home">~/.hgrc</filename> file, and add the lines
bos@559 257 below.</para>
bos@558 258
bos@580 259 <programlisting>[extensions]
bos@580 260 hgext.mq =</programlisting>
bos@558 261
bos@558 262 <para>Once the extension is enabled, it will make a number of new
bos@558 263 commands available. To verify that the extension is working,
bos@558 264 you can use <command role="hg-cmd">hg help</command> to see if
bos@559 265 the <command role="hg-ext-mq">qinit</command> command is now
bos@558 266 available.</para>
bos@558 267
bos@567 268 &interaction.mq.qinit-help.help;
bos@558 269
bos@558 270 <para>You can use MQ with <emphasis>any</emphasis> Mercurial
bos@558 271 repository, and its commands only operate within that
bos@558 272 repository. To get started, simply prepare the repository using
bos@559 273 the <command role="hg-ext-mq">qinit</command> command.</para>
bos@559 274
bos@567 275 &interaction.mq.tutorial.qinit;
bos@559 276
bos@559 277 <para>This command creates an empty directory called <filename
bos@558 278 role="special" class="directory">.hg/patches</filename>, where
bos@558 279 MQ will keep its metadata. As with many Mercurial commands, the
bos@559 280 <command role="hg-ext-mq">qinit</command> command prints nothing
bos@558 281 if it succeeds.</para>
bos@558 282
bos@558 283 <sect2>
bos@558 284 <title>Creating a new patch</title>
bos@558 285
bos@558 286 <para>To begin work on a new patch, use the <command
bos@559 287 role="hg-ext-mq">qnew</command> command. This command takes
bos@559 288 one argument, the name of the patch to create.</para>
bos@559 289
bos@559 290 <para>MQ will use this as the name of an actual file in the
bos@559 291 <filename role="special"
bos@559 292 class="directory">.hg/patches</filename> directory, as you
bos@559 293 can see below.</para>
bos@559 294
bos@567 295 &interaction.mq.tutorial.qnew;
bos@558 296
bos@558 297 <para>Also newly present in the <filename role="special"
bos@558 298 class="directory">.hg/patches</filename> directory are two
bos@558 299 other files, <filename role="special">series</filename> and
bos@558 300 <filename role="special">status</filename>. The <filename
bos@558 301 role="special">series</filename> file lists all of the
bos@558 302 patches that MQ knows about for this repository, with one
bos@558 303 patch per line. Mercurial uses the <filename
bos@558 304 role="special">status</filename> file for internal
bos@558 305 book-keeping; it tracks all of the patches that MQ has
bos@558 306 <emphasis>applied</emphasis> in this repository.</para>
bos@558 307
bos@558 308 <note>
bos@558 309 <para> You may sometimes want to edit the <filename
bos@558 310 role="special">series</filename> file by hand; for
bos@558 311 example, to change the sequence in which some patches are
bos@558 312 applied. However, manually editing the <filename
bos@558 313 role="special">status</filename> file is almost always a
bos@558 314 bad idea, as it's easy to corrupt MQ's idea of what is
bos@558 315 happening.</para>
bos@558 316 </note>
bos@558 317
bos@558 318 <para>Once you have created your new patch, you can edit files
bos@558 319 in the working directory as you usually would. All of the
bos@558 320 normal Mercurial commands, such as <command role="hg-cmd">hg
bos@558 321 diff</command> and <command role="hg-cmd">hg
bos@558 322 annotate</command>, work exactly as they did before.</para>
bos@558 323
bos@558 324 </sect2>
bos@558 325 <sect2>
bos@558 326 <title>Refreshing a patch</title>
bos@558 327
bos@558 328 <para>When you reach a point where you want to save your work,
bos@559 329 use the <command role="hg-ext-mq">qrefresh</command> command
bos@559 330 to update the patch you are working on.</para>
bos@558 331
bos@567 332 &interaction.mq.tutorial.qrefresh;
bos@558 333
bos@559 334 <para>This command folds the changes you have made in the
bos@559 335 working directory into your patch, and updates its
bos@559 336 corresponding changeset to contain those changes.</para>
bos@559 337
bos@559 338 <para>You can run <command role="hg-ext-mq">qrefresh</command>
bos@559 339 as often as you like, so it's a good way to
bos@559 340 <quote>checkpoint</quote> your work. Refresh your patch at an
bos@559 341 opportune time; try an experiment; and if the experiment
bos@558 342 doesn't work out, <command role="hg-cmd">hg revert</command>
bos@558 343 your modifications back to the last time you refreshed.</para>
bos@558 344
bos@567 345 &interaction.mq.tutorial.qrefresh2;
bos@558 346
bos@558 347 </sect2>
bos@558 348 <sect2>
bos@558 349 <title>Stacking and tracking patches</title>
bos@558 350
bos@558 351 <para>Once you have finished working on a patch, or need to work
bos@559 352 on another, you can use the <command
bos@559 353 role="hg-ext-mq">qnew</command> command again to create a
bos@559 354 new patch. Mercurial will apply this patch on top of your
bos@559 355 existing patch.</para>
bos@559 356
bos@567 357 &interaction.mq.tutorial.qnew2;
bos@559 358 <para>Notice that the patch contains the changes in our prior
bos@559 359 patch as part of its context (you can see this more clearly in
bos@559 360 the output of <command role="hg-cmd">hg
bos@558 361 annotate</command>).</para>
bos@558 362
bos@558 363 <para>So far, with the exception of <command
bos@559 364 role="hg-ext-mq">qnew</command> and <command
bos@559 365 role="hg-ext-mq">qrefresh</command>, we've been careful to
bos@558 366 only use regular Mercurial commands. However, MQ provides
bos@558 367 many commands that are easier to use when you are thinking
bos@559 368 about patches, as illustrated below.</para>
bos@559 369
bos@567 370 &interaction.mq.tutorial.qseries;
bos@558 371
bos@558 372 <itemizedlist>
bos@558 373 <listitem><para>The <command
bos@559 374 role="hg-ext-mq">qseries</command> command lists every
bos@558 375 patch that MQ knows about in this repository, from oldest
bos@558 376 to newest (most recently
bos@559 377 <emphasis>created</emphasis>).</para>
bos@559 378 </listitem>
bos@558 379 <listitem><para>The <command
bos@559 380 role="hg-ext-mq">qapplied</command> command lists every
bos@558 381 patch that MQ has <emphasis>applied</emphasis> in this
bos@558 382 repository, again from oldest to newest (most recently
bos@559 383 applied).</para>
bos@559 384 </listitem></itemizedlist>
bos@558 385
bos@558 386 </sect2>
bos@558 387 <sect2>
bos@558 388 <title>Manipulating the patch stack</title>
bos@558 389
bos@558 390 <para>The previous discussion implied that there must be a
bos@558 391 difference between <quote>known</quote> and
bos@558 392 <quote>applied</quote> patches, and there is. MQ can manage a
bos@558 393 patch without it being applied in the repository.</para>
bos@558 394
bos@558 395 <para>An <emphasis>applied</emphasis> patch has a corresponding
bos@558 396 changeset in the repository, and the effects of the patch and
bos@558 397 changeset are visible in the working directory. You can undo
bos@558 398 the application of a patch using the <command
bos@559 399 role="hg-ext-mq">qpop</command> command. MQ still
bos@558 400 <emphasis>knows about</emphasis>, or manages, a popped patch,
bos@558 401 but the patch no longer has a corresponding changeset in the
bos@558 402 repository, and the working directory does not contain the
bos@558 403 changes made by the patch. Figure <xref
dongsheng@640 404 endterm="fig.mq.stack.caption" linkend="fig.mq.stack"/> illustrates
bos@559 405 the difference between applied and tracked patches.</para>
bos@558 406
dongsheng@625 407 <informalfigure id="fig.mq.stack">
dongsheng@640 408 <mediaobject>
dongsheng@640 409 <imageobject><imagedata fileref="images/mq-stack.png"/></imageobject>
dongsheng@640 410 <textobject><phrase>XXX add text</phrase></textobject>
dongsheng@640 411 <caption><para id="fig.mq.stack.caption">Applied and unapplied patches
dongsheng@640 412 in the MQ patch stack</para></caption>
dongsheng@640 413 </mediaobject>
bos@558 414 </informalfigure>
bos@558 415
bos@558 416 <para>You can reapply an unapplied, or popped, patch using the
bos@559 417 <command role="hg-ext-mq">qpush</command> command. This
bos@558 418 creates a new changeset to correspond to the patch, and the
bos@558 419 patch's changes once again become present in the working
bos@559 420 directory. See below for examples of <command
bos@559 421 role="hg-ext-mq">qpop</command> and <command
bos@559 422 role="hg-ext-mq">qpush</command> in action.</para>
bos@567 423 &interaction.mq.tutorial.qpop;
bos@558 424
bos@559 425 <para>Notice that once we have popped a patch or two patches,
bos@559 426 the output of <command role="hg-ext-mq">qseries</command>
bos@559 427 remains the same, while that of <command
bos@559 428 role="hg-ext-mq">qapplied</command> has changed.</para>
bos@559 429
bos@559 430
bos@558 431 </sect2>
bos@558 432 <sect2>
bos@558 433 <title>Pushing and popping many patches</title>
bos@558 434
bos@559 435 <para>While <command role="hg-ext-mq">qpush</command> and
bos@559 436 <command role="hg-ext-mq">qpop</command> each operate on a
bos@558 437 single patch at a time by default, you can push and pop many
bos@558 438 patches in one go. The <option
bos@559 439 role="hg-ext-mq-cmd-qpush-opt">hg -a</option> option to
bos@559 440 <command role="hg-ext-mq">qpush</command> causes it to push
bos@558 441 all unapplied patches, while the <option
bos@558 442 role="hg-ext-mq-cmd-qpop-opt">-a</option> option to <command
bos@559 443 role="hg-ext-mq">qpop</command> causes it to pop all applied
bos@558 444 patches. (For some more ways to push and pop many patches,
dongsheng@625 445 see section <xref linkend="sec.mq.perf"/>
bos@559 446 below.)</para>
bos@558 447
bos@567 448 &interaction.mq.tutorial.qpush-a;
bos@558 449
bos@558 450 </sect2>
bos@558 451 <sect2>
bos@558 452 <title>Safety checks, and overriding them</title>
bos@558 453
bos@558 454 <para>Several MQ commands check the working directory before
bos@558 455 they do anything, and fail if they find any modifications.
bos@558 456 They do this to ensure that you won't lose any changes that
bos@559 457 you have made, but not yet incorporated into a patch. The
bos@559 458 example below illustrates this; the <command
bos@559 459 role="hg-ext-mq">qnew</command> command will not create a
bos@558 460 new patch if there are outstanding changes, caused in this
bos@558 461 case by the <command role="hg-cmd">hg add</command> of
bos@558 462 <filename>file3</filename>.</para>
bos@558 463
bos@567 464 &interaction.mq.tutorial.add;
bos@558 465
bos@558 466 <para>Commands that check the working directory all take an
bos@558 467 <quote>I know what I'm doing</quote> option, which is always
bos@558 468 named <option>-f</option>. The exact meaning of
bos@558 469 <option>-f</option> depends on the command. For example,
bos@558 470 <command role="hg-cmd">hg qnew <option
bos@559 471 role="hg-ext-mq-cmd-qnew-opt">hg -f</option></command>
bos@559 472 will incorporate any outstanding changes into the new patch it
bos@558 473 creates, but <command role="hg-cmd">hg qpop <option
bos@559 474 role="hg-ext-mq-cmd-qpop-opt">hg -f</option></command>
bos@559 475 will revert modifications to any files affected by the patch
bos@559 476 that it is popping. Be sure to read the documentation for a
bos@558 477 command's <option>-f</option> option before you use it!</para>
bos@558 478
bos@558 479 </sect2>
bos@558 480 <sect2>
bos@558 481 <title>Working on several patches at once</title>
bos@558 482
bos@559 483 <para>The <command role="hg-ext-mq">qrefresh</command> command
bos@558 484 always refreshes the <emphasis>topmost</emphasis> applied
bos@558 485 patch. This means that you can suspend work on one patch (by
bos@558 486 refreshing it), pop or push to make a different patch the top,
bos@558 487 and work on <emphasis>that</emphasis> patch for a
bos@558 488 while.</para>
bos@558 489
bos@558 490 <para>Here's an example that illustrates how you can use this
bos@558 491 ability. Let's say you're developing a new feature as two
bos@558 492 patches. The first is a change to the core of your software,
bos@559 493 and the second&emdash;layered on top of the
bos@559 494 first&emdash;changes the user interface to use the code you
bos@559 495 just added to the core. If you notice a bug in the core while
bos@559 496 you're working on the UI patch, it's easy to fix the core.
bos@559 497 Simply <command role="hg-ext-mq">qrefresh</command> the UI
bos@559 498 patch to save your in-progress changes, and <command
bos@559 499 role="hg-ext-mq">qpop</command> down to the core patch. Fix
bos@559 500 the core bug, <command role="hg-ext-mq">qrefresh</command> the
bos@559 501 core patch, and <command role="hg-ext-mq">qpush</command> back
bos@558 502 to the UI patch to continue where you left off.</para>
bos@558 503
bos@558 504 </sect2>
bos@558 505 </sect1>
dongsheng@625 506 <sect1 id="sec.mq.adv-patch">
bos@558 507 <title>More about patches</title>
bos@558 508
bos@558 509 <para>MQ uses the GNU <command>patch</command> command to apply
bos@558 510 patches, so it's helpful to know a few more detailed aspects of
bos@558 511 how <command>patch</command> works, and about patches
bos@558 512 themselves.</para>
bos@558 513
bos@558 514 <sect2>
bos@558 515 <title>The strip count</title>
bos@558 516
bos@558 517 <para>If you look at the file headers in a patch, you will
bos@558 518 notice that the pathnames usually have an extra component on
bos@558 519 the front that isn't present in the actual path name. This is
bos@558 520 a holdover from the way that people used to generate patches
bos@558 521 (people still do this, but it's somewhat rare with modern
bos@558 522 revision control tools).</para>
bos@558 523
bos@558 524 <para>Alice would unpack a tarball, edit her files, then decide
bos@558 525 that she wanted to create a patch. So she'd rename her
bos@558 526 working directory, unpack the tarball again (hence the need
bos@558 527 for the rename), and use the <option
bos@558 528 role="cmd-opt-diff">-r</option> and <option
bos@558 529 role="cmd-opt-diff">-N</option> options to
bos@558 530 <command>diff</command> to recursively generate a patch
bos@558 531 between the unmodified directory and the modified one. The
bos@558 532 result would be that the name of the unmodified directory
bos@558 533 would be at the front of the left-hand path in every file
bos@558 534 header, and the name of the modified directory would be at the
bos@558 535 front of the right-hand path.</para>
bos@558 536
bos@558 537 <para>Since someone receiving a patch from the Alices of the net
bos@558 538 would be unlikely to have unmodified and modified directories
bos@558 539 with exactly the same names, the <command>patch</command>
bos@558 540 command has a <option role="cmd-opt-patch">-p</option> option
bos@558 541 that indicates the number of leading path name components to
bos@558 542 strip when trying to apply a patch. This number is called the
bos@558 543 <emphasis>strip count</emphasis>.</para>
bos@558 544
bos@558 545 <para>An option of <quote><literal>-p1</literal></quote> means
bos@558 546 <quote>use a strip count of one</quote>. If
bos@558 547 <command>patch</command> sees a file name
bos@558 548 <filename>foo/bar/baz</filename> in a file header, it will
bos@558 549 strip <filename>foo</filename> and try to patch a file named
bos@558 550 <filename>bar/baz</filename>. (Strictly speaking, the strip
bos@558 551 count refers to the number of <emphasis>path
bos@558 552 separators</emphasis> (and the components that go with them
bos@558 553 ) to strip. A strip count of one will turn
bos@558 554 <filename>foo/bar</filename> into <filename>bar</filename>,
bos@558 555 but <filename>/foo/bar</filename> (notice the extra leading
bos@558 556 slash) into <filename>foo/bar</filename>.)</para>
bos@558 557
bos@558 558 <para>The <quote>standard</quote> strip count for patches is
bos@558 559 one; almost all patches contain one leading path name
bos@558 560 component that needs to be stripped. Mercurial's <command
bos@558 561 role="hg-cmd">hg diff</command> command generates path names
bos@558 562 in this form, and the <command role="hg-cmd">hg
bos@558 563 import</command> command and MQ expect patches to have a
bos@558 564 strip count of one.</para>
bos@558 565
bos@558 566 <para>If you receive a patch from someone that you want to add
bos@558 567 to your patch queue, and the patch needs a strip count other
bos@558 568 than one, you cannot just <command
bos@559 569 role="hg-ext-mq">qimport</command> the patch, because
bos@559 570 <command role="hg-ext-mq">qimport</command> does not yet have
bos@558 571 a <literal>-p</literal> option (see <ulink role="hg-bug"
bos@558 572 url="http://www.selenic.com/mercurial/bts/issue311">issue
bos@558 573 311</ulink>). Your best bet is to <command
bos@559 574 role="hg-ext-mq">qnew</command> a patch of your own, then
bos@558 575 use <command>patch -pN</command> to apply their patch,
bos@558 576 followed by <command role="hg-cmd">hg addremove</command> to
bos@558 577 pick up any files added or removed by the patch, followed by
bos@559 578 <command role="hg-ext-mq">hg qrefresh</command>. This
bos@559 579 complexity may become unnecessary; see <ulink role="hg-bug"
bos@558 580 url="http://www.selenic.com/mercurial/bts/issue311">issue
bos@559 581 311</ulink> for details.
bos@559 582 </para>
bos@558 583 </sect2>
bos@558 584 <sect2>
bos@558 585 <title>Strategies for applying a patch</title>
bos@558 586
bos@558 587 <para>When <command>patch</command> applies a hunk, it tries a
bos@558 588 handful of successively less accurate strategies to try to
bos@558 589 make the hunk apply. This falling-back technique often makes
bos@558 590 it possible to take a patch that was generated against an old
bos@558 591 version of a file, and apply it against a newer version of
bos@558 592 that file.</para>
bos@558 593
bos@558 594 <para>First, <command>patch</command> tries an exact match,
bos@558 595 where the line numbers, the context, and the text to be
bos@558 596 modified must apply exactly. If it cannot make an exact
bos@558 597 match, it tries to find an exact match for the context,
bos@558 598 without honouring the line numbering information. If this
bos@558 599 succeeds, it prints a line of output saying that the hunk was
bos@558 600 applied, but at some <emphasis>offset</emphasis> from the
bos@558 601 original line number.</para>
bos@558 602
bos@558 603 <para>If a context-only match fails, <command>patch</command>
bos@558 604 removes the first and last lines of the context, and tries a
bos@558 605 <emphasis>reduced</emphasis> context-only match. If the hunk
bos@558 606 with reduced context succeeds, it prints a message saying that
bos@558 607 it applied the hunk with a <emphasis>fuzz factor</emphasis>
bos@558 608 (the number after the fuzz factor indicates how many lines of
bos@558 609 context <command>patch</command> had to trim before the patch
bos@558 610 applied).</para>
bos@558 611
bos@558 612 <para>When neither of these techniques works,
bos@558 613 <command>patch</command> prints a message saying that the hunk
bos@558 614 in question was rejected. It saves rejected hunks (also
bos@558 615 simply called <quote>rejects</quote>) to a file with the same
bos@558 616 name, and an added <filename role="special">.rej</filename>
bos@558 617 extension. It also saves an unmodified copy of the file with
bos@558 618 a <filename role="special">.orig</filename> extension; the
bos@558 619 copy of the file without any extensions will contain any
bos@558 620 changes made by hunks that <emphasis>did</emphasis> apply
bos@558 621 cleanly. If you have a patch that modifies
bos@558 622 <filename>foo</filename> with six hunks, and one of them fails
bos@558 623 to apply, you will have: an unmodified
bos@558 624 <filename>foo.orig</filename>, a <filename>foo.rej</filename>
bos@558 625 containing one hunk, and <filename>foo</filename>, containing
bos@558 626 the changes made by the five successful hunks.</para>
bos@558 627
bos@558 628 </sect2>
bos@558 629 <sect2>
bos@558 630 <title>Some quirks of patch representation</title>
bos@558 631
bos@558 632 <para>There are a few useful things to know about how
bos@558 633 <command>patch</command> works with files.</para>
bos@558 634 <itemizedlist>
bos@558 635 <listitem><para>This should already be obvious, but
bos@558 636 <command>patch</command> cannot handle binary
bos@559 637 files.</para>
bos@559 638 </listitem>
bos@558 639 <listitem><para>Neither does it care about the executable bit;
bos@558 640 it creates new files as readable, but not
bos@559 641 executable.</para>
bos@559 642 </listitem>
bos@558 643 <listitem><para><command>patch</command> treats the removal of
bos@558 644 a file as a diff between the file to be removed and the
bos@558 645 empty file. So your idea of <quote>I deleted this
bos@558 646 file</quote> looks like <quote>every line of this file
bos@559 647 was deleted</quote> in a patch.</para>
bos@559 648 </listitem>
bos@558 649 <listitem><para>It treats the addition of a file as a diff
bos@558 650 between the empty file and the file to be added. So in a
bos@558 651 patch, your idea of <quote>I added this file</quote> looks
bos@558 652 like <quote>every line of this file was
bos@559 653 added</quote>.</para>
bos@559 654 </listitem>
bos@558 655 <listitem><para>It treats a renamed file as the removal of the
bos@558 656 old name, and the addition of the new name. This means
bos@558 657 that renamed files have a big footprint in patches. (Note
bos@558 658 also that Mercurial does not currently try to infer when
bos@559 659 files have been renamed or copied in a patch.)</para>
bos@559 660 </listitem>
bos@558 661 <listitem><para><command>patch</command> cannot represent
bos@558 662 empty files, so you cannot use a patch to represent the
bos@558 663 notion <quote>I added this empty file to the
bos@559 664 tree</quote>.</para>
bos@559 665 </listitem></itemizedlist>
bos@559 666 </sect2>
bos@558 667 <sect2>
bos@558 668 <title>Beware the fuzz</title>
bos@558 669
bos@558 670 <para>While applying a hunk at an offset, or with a fuzz factor,
bos@558 671 will often be completely successful, these inexact techniques
bos@558 672 naturally leave open the possibility of corrupting the patched
bos@558 673 file. The most common cases typically involve applying a
bos@558 674 patch twice, or at an incorrect location in the file. If
bos@558 675 <command>patch</command> or <command
bos@559 676 role="hg-ext-mq">qpush</command> ever mentions an offset or
bos@558 677 fuzz factor, you should make sure that the modified files are
bos@558 678 correct afterwards.</para>
bos@558 679
bos@558 680 <para>It's often a good idea to refresh a patch that has applied
bos@558 681 with an offset or fuzz factor; refreshing the patch generates
bos@558 682 new context information that will make it apply cleanly. I
bos@558 683 say <quote>often,</quote> not <quote>always,</quote> because
bos@558 684 sometimes refreshing a patch will make it fail to apply
bos@558 685 against a different revision of the underlying files. In some
bos@558 686 cases, such as when you're maintaining a patch that must sit
bos@558 687 on top of multiple versions of a source tree, it's acceptable
bos@558 688 to have a patch apply with some fuzz, provided you've verified
bos@558 689 the results of the patching process in such cases.</para>
bos@558 690
bos@558 691 </sect2>
bos@558 692 <sect2>
bos@558 693 <title>Handling rejection</title>
bos@558 694
bos@559 695 <para>If <command role="hg-ext-mq">qpush</command> fails to
bos@558 696 apply a patch, it will print an error message and exit. If it
bos@558 697 has left <filename role="special">.rej</filename> files
bos@558 698 behind, it is usually best to fix up the rejected hunks before
bos@558 699 you push more patches or do any further work.</para>
bos@558 700
bos@558 701 <para>If your patch <emphasis>used to</emphasis> apply cleanly,
bos@558 702 and no longer does because you've changed the underlying code
bos@558 703 that your patches are based on, Mercurial Queues can help; see
bos@559 704 section <xref
dongsheng@625 705 linkend="sec.mq.merge"/> for details.</para>
bos@558 706
bos@558 707 <para>Unfortunately, there aren't any great techniques for
bos@558 708 dealing with rejected hunks. Most often, you'll need to view
bos@558 709 the <filename role="special">.rej</filename> file and edit the
bos@558 710 target file, applying the rejected hunks by hand.</para>
bos@558 711
bos@558 712 <para>If you're feeling adventurous, Neil Brown, a Linux kernel
bos@558 713 hacker, wrote a tool called <command>wiggle</command>
bos@558 714 <citation>web:wiggle</citation>, which is more vigorous than
bos@558 715 <command>patch</command> in its attempts to make a patch
bos@558 716 apply.</para>
bos@558 717
bos@558 718 <para>Another Linux kernel hacker, Chris Mason (the author of
bos@558 719 Mercurial Queues), wrote a similar tool called
bos@558 720 <command>mpatch</command> <citation>web:mpatch</citation>,
bos@558 721 which takes a simple approach to automating the application of
bos@558 722 hunks rejected by <command>patch</command>. The
bos@558 723 <command>mpatch</command> command can help with four common
bos@558 724 reasons that a hunk may be rejected:</para>
bos@558 725
bos@558 726 <itemizedlist>
bos@558 727 <listitem><para>The context in the middle of a hunk has
bos@559 728 changed.</para>
bos@559 729 </listitem>
bos@558 730 <listitem><para>A hunk is missing some context at the
bos@559 731 beginning or end.</para>
bos@559 732 </listitem>
bos@558 733 <listitem><para>A large hunk might apply better&emdash;either
bos@559 734 entirely or in part&emdash;if it was broken up into
bos@559 735 smaller hunks.</para>
bos@559 736 </listitem>
bos@558 737 <listitem><para>A hunk removes lines with slightly different
bos@559 738 content than those currently present in the file.</para>
bos@559 739 </listitem></itemizedlist>
bos@558 740
bos@558 741 <para>If you use <command>wiggle</command> or
bos@558 742 <command>mpatch</command>, you should be doubly careful to
bos@558 743 check your results when you're done. In fact,
bos@558 744 <command>mpatch</command> enforces this method of
bos@558 745 double-checking the tool's output, by automatically dropping
bos@558 746 you into a merge program when it has done its job, so that you
bos@558 747 can verify its work and finish off any remaining
bos@558 748 merges.</para>
bos@558 749
bos@558 750 </sect2>
bos@558 751 </sect1>
dongsheng@625 752 <sect1 id="sec.mq.perf">
bos@558 753 <title>Getting the best performance out of MQ</title>
bos@558 754
bos@558 755 <para>MQ is very efficient at handling a large number of patches.
bos@558 756 I ran some performance experiments in mid-2006 for a talk that I
bos@558 757 gave at the 2006 EuroPython conference
bos@558 758 <citation>web:europython</citation>. I used as my data set the
bos@558 759 Linux 2.6.17-mm1 patch series, which consists of 1,738 patches.
bos@558 760 I applied these on top of a Linux kernel repository containing
bos@558 761 all 27,472 revisions between Linux 2.6.12-rc2 and Linux
bos@558 762 2.6.17.</para>
bos@558 763
bos@558 764 <para>On my old, slow laptop, I was able to <command
bos@558 765 role="hg-cmd">hg qpush <option
bos@559 766 role="hg-ext-mq-cmd-qpush-opt">hg -a</option></command> all
bos@558 767 1,738 patches in 3.5 minutes, and <command role="hg-cmd">hg qpop
bos@559 768 <option role="hg-ext-mq-cmd-qpop-opt">hg -a</option></command>
bos@558 769 them all in 30 seconds. (On a newer laptop, the time to push
bos@558 770 all patches dropped to two minutes.) I could <command
bos@559 771 role="hg-ext-mq">qrefresh</command> one of the biggest patches
bos@558 772 (which made 22,779 lines of changes to 287 files) in 6.6
bos@558 773 seconds.</para>
bos@558 774
bos@558 775 <para>Clearly, MQ is well suited to working in large trees, but
bos@558 776 there are a few tricks you can use to get the best performance
bos@558 777 of it.</para>
bos@558 778
bos@558 779 <para>First of all, try to <quote>batch</quote> operations
bos@558 780 together. Every time you run <command
bos@559 781 role="hg-ext-mq">qpush</command> or <command
bos@559 782 role="hg-ext-mq">qpop</command>, these commands scan the
bos@558 783 working directory once to make sure you haven't made some
bos@558 784 changes and then forgotten to run <command
bos@559 785 role="hg-ext-mq">qrefresh</command>. On a small tree, the
bos@558 786 time that this scan takes is unnoticeable. However, on a
bos@558 787 medium-sized tree (containing tens of thousands of files), it
bos@558 788 can take a second or more.</para>
bos@558 789
bos@559 790 <para>The <command role="hg-ext-mq">qpush</command> and <command
bos@559 791 role="hg-ext-mq">qpop</command> commands allow you to push and
bos@558 792 pop multiple patches at a time. You can identify the
bos@558 793 <quote>destination patch</quote> that you want to end up at.
bos@559 794 When you <command role="hg-ext-mq">qpush</command> with a
bos@558 795 destination specified, it will push patches until that patch is
bos@558 796 at the top of the applied stack. When you <command
bos@559 797 role="hg-ext-mq">qpop</command> to a destination, MQ will pop
bos@558 798 patches until the destination patch is at the top.</para>
bos@558 799
bos@558 800 <para>You can identify a destination patch using either the name
bos@558 801 of the patch, or by number. If you use numeric addressing,
bos@558 802 patches are counted from zero; this means that the first patch
bos@558 803 is zero, the second is one, and so on.</para>
bos@558 804
bos@558 805 </sect1>
dongsheng@625 806 <sect1 id="sec.mq.merge">
bos@558 807 <title>Updating your patches when the underlying code
bos@558 808 changes</title>
bos@558 809
bos@558 810 <para>It's common to have a stack of patches on top of an
bos@558 811 underlying repository that you don't modify directly. If you're
bos@558 812 working on changes to third-party code, or on a feature that is
bos@558 813 taking longer to develop than the rate of change of the code
bos@558 814 beneath, you will often need to sync up with the underlying
bos@558 815 code, and fix up any hunks in your patches that no longer apply.
bos@558 816 This is called <emphasis>rebasing</emphasis> your patch
bos@558 817 series.</para>
bos@558 818
bos@558 819 <para>The simplest way to do this is to <command role="hg-cmd">hg
bos@559 820 qpop <option role="hg-ext-mq-cmd-qpop-opt">hg
bos@559 821 -a</option></command> your patches, then <command
bos@559 822 role="hg-cmd">hg pull</command> changes into the underlying
bos@559 823 repository, and finally <command role="hg-cmd">hg qpush <option
bos@559 824 role="hg-ext-mq-cmd-qpop-opt">hg -a</option></command> your
bos@558 825 patches again. MQ will stop pushing any time it runs across a
bos@558 826 patch that fails to apply during conflicts, allowing you to fix
bos@559 827 your conflicts, <command role="hg-ext-mq">qrefresh</command> the
bos@558 828 affected patch, and continue pushing until you have fixed your
bos@558 829 entire stack.</para>
bos@558 830
bos@558 831 <para>This approach is easy to use and works well if you don't
bos@558 832 expect changes to the underlying code to affect how well your
bos@558 833 patches apply. If your patch stack touches code that is modified
bos@558 834 frequently or invasively in the underlying repository, however,
bos@558 835 fixing up rejected hunks by hand quickly becomes
bos@558 836 tiresome.</para>
bos@558 837
bos@559 838 <para>It's possible to partially automate the rebasing process.
bos@559 839 If your patches apply cleanly against some revision of the
bos@558 840 underlying repo, MQ can use this information to help you to
bos@558 841 resolve conflicts between your patches and a different
bos@558 842 revision.</para>
bos@558 843
bos@558 844 <para>The process is a little involved.</para>
bos@558 845 <orderedlist>
bos@558 846 <listitem><para>To begin, <command role="hg-cmd">hg qpush
bos@558 847 -a</command> all of your patches on top of the revision
bos@559 848 where you know that they apply cleanly.</para>
bos@559 849 </listitem>
bos@558 850 <listitem><para>Save a backup copy of your patch directory using
bos@558 851 <command role="hg-cmd">hg qsave <option
bos@559 852 role="hg-ext-mq-cmd-qsave-opt">hg -e</option> <option
bos@559 853 role="hg-ext-mq-cmd-qsave-opt">hg -c</option></command>.
bos@558 854 This prints the name of the directory that it has saved the
bos@558 855 patches in. It will save the patches to a directory called
bos@558 856 <filename role="special"
bos@558 857 class="directory">.hg/patches.N</filename>, where
bos@558 858 <literal>N</literal> is a small integer. It also commits a
bos@558 859 <quote>save changeset</quote> on top of your applied
bos@558 860 patches; this is for internal book-keeping, and records the
bos@558 861 states of the <filename role="special">series</filename> and
bos@559 862 <filename role="special">status</filename> files.</para>
bos@559 863 </listitem>
bos@559 864 <listitem><para>Use <command role="hg-cmd">hg pull</command> to
bos@559 865 bring new changes into the underlying repository. (Don't
bos@559 866 run <command role="hg-cmd">hg pull -u</command>; see below
bos@559 867 for why.)</para>
bos@559 868 </listitem>
bos@558 869 <listitem><para>Update to the new tip revision, using <command
bos@558 870 role="hg-cmd">hg update <option
bos@558 871 role="hg-opt-update">-C</option></command> to override
bos@559 872 the patches you have pushed.</para>
bos@559 873 </listitem>
bos@580 874 <listitem><para>Merge all patches using <command>hg qpush -m
bos@580 875 -a</command>. The <option
bos@580 876 role="hg-ext-mq-cmd-qpush-opt">-m</option> option to
bos@559 877 <command role="hg-ext-mq">qpush</command> tells MQ to
bos@558 878 perform a three-way merge if the patch fails to
bos@559 879 apply.</para>
bos@559 880 </listitem></orderedlist>
bos@558 881
bos@558 882 <para>During the <command role="hg-cmd">hg qpush <option
bos@559 883 role="hg-ext-mq-cmd-qpush-opt">hg -m</option></command>,
bos@559 884 each patch in the <filename role="special">series</filename>
bos@559 885 file is applied normally. If a patch applies with fuzz or
bos@559 886 rejects, MQ looks at the queue you <command
bos@559 887 role="hg-ext-mq">qsave</command>d, and performs a three-way
bos@558 888 merge with the corresponding changeset. This merge uses
bos@558 889 Mercurial's normal merge machinery, so it may pop up a GUI merge
bos@558 890 tool to help you to resolve problems.</para>
bos@558 891
bos@558 892 <para>When you finish resolving the effects of a patch, MQ
bos@558 893 refreshes your patch based on the result of the merge.</para>
bos@558 894
bos@558 895 <para>At the end of this process, your repository will have one
bos@558 896 extra head from the old patch queue, and a copy of the old patch
bos@558 897 queue will be in <filename role="special"
bos@558 898 class="directory">.hg/patches.N</filename>. You can remove the
bos@559 899 extra head using <command role="hg-cmd">hg qpop -a -n
bos@558 900 patches.N</command> or <command role="hg-cmd">hg
bos@558 901 strip</command>. You can delete <filename role="special"
bos@558 902 class="directory">.hg/patches.N</filename> once you are sure
bos@558 903 that you no longer need it as a backup.</para>
bos@558 904
bos@558 905 </sect1>
bos@558 906 <sect1>
bos@558 907 <title>Identifying patches</title>
bos@558 908
bos@558 909 <para>MQ commands that work with patches let you refer to a patch
bos@558 910 either by using its name or by a number. By name is obvious
bos@558 911 enough; pass the name <filename>foo.patch</filename> to <command
bos@559 912 role="hg-ext-mq">qpush</command>, for example, and it will
bos@558 913 push patches until <filename>foo.patch</filename> is
bos@558 914 applied.</para>
bos@558 915
bos@558 916 <para>As a shortcut, you can refer to a patch using both a name
bos@558 917 and a numeric offset; <literal>foo.patch-2</literal> means
bos@558 918 <quote>two patches before <literal>foo.patch</literal></quote>,
bos@558 919 while <literal>bar.patch+4</literal> means <quote>four patches
bos@558 920 after <literal>bar.patch</literal></quote>.</para>
bos@558 921
bos@558 922 <para>Referring to a patch by index isn't much different. The
bos@558 923 first patch printed in the output of <command
bos@559 924 role="hg-ext-mq">qseries</command> is patch zero (yes, it's
bos@558 925 one of those start-at-zero counting systems); the second is
bos@558 926 patch one; and so on.</para>
bos@558 927
bos@558 928 <para>MQ also makes it easy to work with patches when you are
bos@558 929 using normal Mercurial commands. Every command that accepts a
bos@558 930 changeset ID will also accept the name of an applied patch. MQ
bos@558 931 augments the tags normally in the repository with an eponymous
bos@558 932 one for each applied patch. In addition, the special tags
bos@580 933 <literal role="tag">qbase</literal> and
bos@580 934 <literal role="tag">qtip</literal> identify
bos@558 935 the <quote>bottom-most</quote> and topmost applied patches,
bos@558 936 respectively.</para>
bos@558 937
bos@558 938 <para>These additions to Mercurial's normal tagging capabilities
bos@558 939 make dealing with patches even more of a breeze.</para>
bos@558 940 <itemizedlist>
bos@558 941 <listitem><para>Want to patchbomb a mailing list with your
bos@558 942 latest series of changes?</para>
bos@580 943 <programlisting>hg email qbase:qtip</programlisting>
bos@559 944 <para> (Don't know what <quote>patchbombing</quote> is? See
dongsheng@625 945 section <xref linkend="sec.hgext.patchbomb"/>.)</para>
bos@559 946 </listitem>
bos@558 947 <listitem><para>Need to see all of the patches since
bos@558 948 <literal>foo.patch</literal> that have touched files in a
bos@558 949 subdirectory of your tree?</para>
bos@580 950 <programlisting>hg log -r foo.patch:qtip subdir</programlisting>
bos@559 951 </listitem>
bos@558 952 </itemizedlist>
bos@558 953
bos@558 954 <para>Because MQ makes the names of patches available to the rest
bos@558 955 of Mercurial through its normal internal tag machinery, you
bos@558 956 don't need to type in the entire name of a patch when you want
bos@558 957 to identify it by name.</para>
bos@558 958
bos@558 959 <para>Another nice consequence of representing patch names as tags
bos@558 960 is that when you run the <command role="hg-cmd">hg log</command>
bos@558 961 command, it will display a patch's name as a tag, simply as part
bos@558 962 of its normal output. This makes it easy to visually
bos@558 963 distinguish applied patches from underlying
bos@559 964 <quote>normal</quote> revisions. The following example shows a
bos@559 965 few normal Mercurial commands in use with applied
bos@559 966 patches.</para>
bos@559 967
bos@567 968 &interaction.mq.id.output;
bos@558 969
bos@558 970 </sect1>
bos@558 971 <sect1>
bos@558 972 <title>Useful things to know about</title>
bos@558 973
bos@558 974 <para>There are a number of aspects of MQ usage that don't fit
bos@558 975 tidily into sections of their own, but that are good to know.
bos@558 976 Here they are, in one place.</para>
bos@558 977
bos@558 978 <itemizedlist>
bos@558 979 <listitem><para>Normally, when you <command
bos@559 980 role="hg-ext-mq">qpop</command> a patch and <command
bos@559 981 role="hg-ext-mq">qpush</command> it again, the changeset
bos@558 982 that represents the patch after the pop/push will have a
bos@558 983 <emphasis>different identity</emphasis> than the changeset
bos@558 984 that represented the hash beforehand. See section <xref
dongsheng@625 985 linkend="sec.mqref.cmd.qpush"/> for
bos@559 986 information as to why this is.</para>
bos@559 987 </listitem>
bos@558 988 <listitem><para>It's not a good idea to <command
bos@558 989 role="hg-cmd">hg merge</command> changes from another
bos@558 990 branch with a patch changeset, at least if you want to
bos@558 991 maintain the <quote>patchiness</quote> of that changeset and
bos@558 992 changesets below it on the patch stack. If you try to do
bos@558 993 this, it will appear to succeed, but MQ will become
bos@559 994 confused.</para>
bos@559 995 </listitem></itemizedlist>
bos@559 996
bos@559 997 </sect1>
dongsheng@625 998 <sect1 id="sec.mq.repo">
bos@558 999 <title>Managing patches in a repository</title>
bos@558 1000
bos@558 1001 <para>Because MQ's <filename role="special"
bos@558 1002 class="directory">.hg/patches</filename> directory resides
bos@558 1003 outside a Mercurial repository's working directory, the
bos@558 1004 <quote>underlying</quote> Mercurial repository knows nothing
bos@558 1005 about the management or presence of patches.</para>
bos@558 1006
bos@558 1007 <para>This presents the interesting possibility of managing the
bos@558 1008 contents of the patch directory as a Mercurial repository in its
bos@558 1009 own right. This can be a useful way to work. For example, you
bos@558 1010 can work on a patch for a while, <command
bos@559 1011 role="hg-ext-mq">qrefresh</command> it, then <command
bos@558 1012 role="hg-cmd">hg commit</command> the current state of the
bos@558 1013 patch. This lets you <quote>roll back</quote> to that version
bos@558 1014 of the patch later on.</para>
bos@558 1015
bos@558 1016 <para>You can then share different versions of the same patch
bos@558 1017 stack among multiple underlying repositories. I use this when I
bos@558 1018 am developing a Linux kernel feature. I have a pristine copy of
bos@558 1019 my kernel sources for each of several CPU architectures, and a
bos@558 1020 cloned repository under each that contains the patches I am
bos@558 1021 working on. When I want to test a change on a different
bos@558 1022 architecture, I push my current patches to the patch repository
bos@558 1023 associated with that kernel tree, pop and push all of my
bos@558 1024 patches, and build and test that kernel.</para>
bos@558 1025
bos@558 1026 <para>Managing patches in a repository makes it possible for
bos@558 1027 multiple developers to work on the same patch series without
bos@558 1028 colliding with each other, all on top of an underlying source
bos@558 1029 base that they may or may not control.</para>
bos@558 1030
bos@558 1031 <sect2>
bos@558 1032 <title>MQ support for patch repositories</title>
bos@558 1033
bos@558 1034 <para>MQ helps you to work with the <filename role="special"
bos@558 1035 class="directory">.hg/patches</filename> directory as a
bos@558 1036 repository; when you prepare a repository for working with
bos@559 1037 patches using <command role="hg-ext-mq">qinit</command>, you
bos@559 1038 can pass the <option role="hg-ext-mq-cmd-qinit-opt">hg
bos@559 1039 -c</option> option to create the <filename role="special"
bos@558 1040 class="directory">.hg/patches</filename> directory as a
bos@558 1041 Mercurial repository.</para>
bos@558 1042
bos@558 1043 <note>
bos@558 1044 <para> If you forget to use the <option
bos@559 1045 role="hg-ext-mq-cmd-qinit-opt">hg -c</option> option, you
bos@559 1046 can simply go into the <filename role="special"
bos@558 1047 class="directory">.hg/patches</filename> directory at any
bos@559 1048 time and run <command role="hg-cmd">hg init</command>.
bos@559 1049 Don't forget to add an entry for the <filename
bos@558 1050 role="special">status</filename> file to the <filename
bos@558 1051 role="special">.hgignore</filename> file, though</para>
bos@558 1052
bos@558 1053 <para> (<command role="hg-cmd">hg qinit <option
bos@559 1054 role="hg-ext-mq-cmd-qinit-opt">hg -c</option></command>
bos@558 1055 does this for you automatically); you
bos@558 1056 <emphasis>really</emphasis> don't want to manage the
bos@558 1057 <filename role="special">status</filename> file.</para>
bos@558 1058 </note>
bos@558 1059
bos@558 1060 <para>As a convenience, if MQ notices that the <filename
bos@558 1061 class="directory">.hg/patches</filename> directory is a
bos@558 1062 repository, it will automatically <command role="hg-cmd">hg
bos@558 1063 add</command> every patch that you create and import.</para>
bos@558 1064
bos@558 1065 <para>MQ provides a shortcut command, <command
bos@559 1066 role="hg-ext-mq">qcommit</command>, that runs <command
bos@558 1067 role="hg-cmd">hg commit</command> in the <filename
bos@558 1068 role="special" class="directory">.hg/patches</filename>
bos@558 1069 directory. This saves some bothersome typing.</para>
bos@558 1070
bos@558 1071 <para>Finally, as a convenience to manage the patch directory,
bos@558 1072 you can define the alias <command>mq</command> on Unix
bos@558 1073 systems. For example, on Linux systems using the
bos@558 1074 <command>bash</command> shell, you can include the following
bos@559 1075 snippet in your <filename
bos@559 1076 role="home">~/.bashrc</filename>.</para>
bos@559 1077
bos@580 1078 <programlisting>alias mq=`hg -R $(hg root)/.hg/patches'</programlisting>
bos@558 1079
bos@558 1080 <para>You can then issue commands of the form <command>mq
bos@558 1081 pull</command> from the main repository.</para>
bos@558 1082
bos@558 1083 </sect2>
bos@558 1084 <sect2>
bos@558 1085 <title>A few things to watch out for</title>
bos@558 1086
bos@558 1087 <para>MQ's support for working with a repository full of patches
bos@558 1088 is limited in a few small respects.</para>
bos@558 1089
bos@558 1090 <para>MQ cannot automatically detect changes that you make to
bos@558 1091 the patch directory. If you <command role="hg-cmd">hg
bos@558 1092 pull</command>, manually edit, or <command role="hg-cmd">hg
bos@558 1093 update</command> changes to patches or the <filename
bos@558 1094 role="special">series</filename> file, you will have to
bos@558 1095 <command role="hg-cmd">hg qpop <option
bos@559 1096 role="hg-ext-mq-cmd-qpop-opt">hg -a</option></command> and
bos@558 1097 then <command role="hg-cmd">hg qpush <option
bos@559 1098 role="hg-ext-mq-cmd-qpush-opt">hg -a</option></command> in
bos@558 1099 the underlying repository to see those changes show up there.
bos@558 1100 If you forget to do this, you can confuse MQ's idea of which
bos@558 1101 patches are applied.</para>
bos@558 1102
bos@558 1103 </sect2>
bos@558 1104 </sect1>
dongsheng@625 1105 <sect1 id="sec.mq.tools">
bos@558 1106 <title>Third party tools for working with patches</title>
bos@558 1107
bos@558 1108 <para>Once you've been working with patches for a while, you'll
bos@558 1109 find yourself hungry for tools that will help you to understand
bos@558 1110 and manipulate the patches you're dealing with.</para>
bos@558 1111
bos@558 1112 <para>The <command>diffstat</command> command
bos@558 1113 <citation>web:diffstat</citation> generates a histogram of the
bos@558 1114 modifications made to each file in a patch. It provides a good
bos@559 1115 way to <quote>get a sense of</quote> a patch&emdash;which files
bos@559 1116 it affects, and how much change it introduces to each file and
bos@559 1117 as a whole. (I find that it's a good idea to use
bos@558 1118 <command>diffstat</command>'s <option
bos@558 1119 role="cmd-opt-diffstat">-p</option> option as a matter of
bos@558 1120 course, as otherwise it will try to do clever things with
bos@558 1121 prefixes of file names that inevitably confuse at least
bos@558 1122 me.)</para>
bos@558 1123
bos@567 1124 &interaction.mq.tools.tools;
bos@558 1125
bos@558 1126 <para>The <literal role="package">patchutils</literal> package
bos@558 1127 <citation>web:patchutils</citation> is invaluable. It provides a
bos@558 1128 set of small utilities that follow the <quote>Unix
bos@558 1129 philosophy;</quote> each does one useful thing with a patch.
bos@558 1130 The <literal role="package">patchutils</literal> command I use
bos@558 1131 most is <command>filterdiff</command>, which extracts subsets
bos@558 1132 from a patch file. For example, given a patch that modifies
bos@558 1133 hundreds of files across dozens of directories, a single
bos@558 1134 invocation of <command>filterdiff</command> can generate a
bos@558 1135 smaller patch that only touches files whose names match a
bos@558 1136 particular glob pattern. See section <xref
dongsheng@625 1137 linkend="mq-collab.tips.interdiff"/> for another
bos@558 1138 example.</para>
bos@558 1139
bos@558 1140 </sect1>
bos@558 1141 <sect1>
bos@558 1142 <title>Good ways to work with patches</title>
bos@558 1143
bos@558 1144 <para>Whether you are working on a patch series to submit to a
bos@558 1145 free software or open source project, or a series that you
bos@558 1146 intend to treat as a sequence of regular changesets when you're
bos@558 1147 done, you can use some simple techniques to keep your work well
bos@558 1148 organised.</para>
bos@558 1149
bos@558 1150 <para>Give your patches descriptive names. A good name for a
bos@558 1151 patch might be <filename>rework-device-alloc.patch</filename>,
bos@558 1152 because it will immediately give you a hint what the purpose of
bos@558 1153 the patch is. Long names shouldn't be a problem; you won't be
bos@558 1154 typing the names often, but you <emphasis>will</emphasis> be
bos@558 1155 running commands like <command
bos@559 1156 role="hg-ext-mq">qapplied</command> and <command
bos@559 1157 role="hg-ext-mq">qtop</command> over and over. Good naming
bos@558 1158 becomes especially important when you have a number of patches
bos@558 1159 to work with, or if you are juggling a number of different tasks
bos@558 1160 and your patches only get a fraction of your attention.</para>
bos@558 1161
bos@558 1162 <para>Be aware of what patch you're working on. Use the <command
bos@559 1163 role="hg-ext-mq">qtop</command> command and skim over the text
bos@558 1164 of your patches frequently&emdash;for example, using <command
bos@558 1165 role="hg-cmd">hg tip <option
bos@559 1166 role="hg-opt-tip">-p</option></command>)&emdash;to be sure
bos@559 1167 of where you stand. I have several times worked on and <command
bos@559 1168 role="hg-ext-mq">qrefresh</command>ed a patch other than the
bos@558 1169 one I intended, and it's often tricky to migrate changes into
bos@558 1170 the right patch after making them in the wrong one.</para>
bos@558 1171
bos@558 1172 <para>For this reason, it is very much worth investing a little
bos@558 1173 time to learn how to use some of the third-party tools I
dongsheng@625 1174 described in section <xref linkend="sec.mq.tools"/>,
bos@559 1175 particularly
bos@559 1176 <command>diffstat</command> and <command>filterdiff</command>.
bos@559 1177 The former will give you a quick idea of what changes your patch
bos@559 1178 is making, while the latter makes it easy to splice hunks
bos@559 1179 selectively out of one patch and into another.</para>
bos@558 1180
bos@558 1181 </sect1>
bos@558 1182 <sect1>
bos@558 1183 <title>MQ cookbook</title>
bos@558 1184
bos@558 1185 <sect2>
bos@558 1186 <title>Manage <quote>trivial</quote> patches</title>
bos@558 1187
bos@558 1188 <para>Because the overhead of dropping files into a new
bos@558 1189 Mercurial repository is so low, it makes a lot of sense to
bos@558 1190 manage patches this way even if you simply want to make a few
bos@558 1191 changes to a source tarball that you downloaded.</para>
bos@558 1192
bos@558 1193 <para>Begin by downloading and unpacking the source tarball, and
bos@567 1194 turning it into a Mercurial repository.</para>
bos@567 1195
bos@567 1196 &interaction.mq.tarball.download;
bos@558 1197
bos@558 1198 <para>Continue by creating a patch stack and making your
bos@567 1199 changes.</para>
bos@567 1200
bos@567 1201 &interaction.mq.tarball.qinit;
bos@558 1202
bos@558 1203 <para>Let's say a few weeks or months pass, and your package
bos@558 1204 author releases a new version. First, bring their changes
bos@567 1205 into the repository.</para>
bos@567 1206
bos@567 1207 &interaction.mq.tarball.newsource;
bos@567 1208
bos@567 1209 <para>The pipeline starting with <command role="hg-cmd">hg
bos@558 1210 locate</command> above deletes all files in the working
bos@558 1211 directory, so that <command role="hg-cmd">hg
bos@558 1212 commit</command>'s <option
bos@558 1213 role="hg-opt-commit">--addremove</option> option can
bos@558 1214 actually tell which files have really been removed in the
bos@558 1215 newer version of the source.</para>
bos@558 1216
bos@558 1217 <para>Finally, you can apply your patches on top of the new
bos@567 1218 tree.</para>
bos@567 1219
bos@567 1220 &interaction.mq.tarball.repush;
bos@558 1221
bos@558 1222 </sect2>
dongsheng@625 1223 <sect2 id="sec.mq.combine">
bos@558 1224 <title>Combining entire patches</title>
bos@558 1225
bos@558 1226 <para>MQ provides a command, <command
bos@559 1227 role="hg-ext-mq">qfold</command> that lets you combine
bos@558 1228 entire patches. This <quote>folds</quote> the patches you
bos@558 1229 name, in the order you name them, into the topmost applied
bos@558 1230 patch, and concatenates their descriptions onto the end of its
bos@558 1231 description. The patches that you fold must be unapplied
bos@558 1232 before you fold them.</para>
bos@558 1233
bos@558 1234 <para>The order in which you fold patches matters. If your
bos@558 1235 topmost applied patch is <literal>foo</literal>, and you
bos@559 1236 <command role="hg-ext-mq">qfold</command>
bos@558 1237 <literal>bar</literal> and <literal>quux</literal> into it,
bos@558 1238 you will end up with a patch that has the same effect as if
bos@558 1239 you applied first <literal>foo</literal>, then
bos@558 1240 <literal>bar</literal>, followed by
bos@558 1241 <literal>quux</literal>.</para>
bos@558 1242
bos@558 1243 </sect2>
bos@558 1244 <sect2>
bos@558 1245 <title>Merging part of one patch into another</title>
bos@558 1246
bos@558 1247 <para>Merging <emphasis>part</emphasis> of one patch into
bos@558 1248 another is more difficult than combining entire
bos@558 1249 patches.</para>
bos@558 1250
bos@558 1251 <para>If you want to move changes to entire files, you can use
bos@558 1252 <command>filterdiff</command>'s <option
bos@558 1253 role="cmd-opt-filterdiff">-i</option> and <option
bos@558 1254 role="cmd-opt-filterdiff">-x</option> options to choose the
bos@558 1255 modifications to snip out of one patch, concatenating its
bos@558 1256 output onto the end of the patch you want to merge into. You
bos@558 1257 usually won't need to modify the patch you've merged the
bos@558 1258 changes from. Instead, MQ will report some rejected hunks
bos@559 1259 when you <command role="hg-ext-mq">qpush</command> it (from
bos@558 1260 the hunks you moved into the other patch), and you can simply
bos@559 1261 <command role="hg-ext-mq">qrefresh</command> the patch to drop
bos@558 1262 the duplicate hunks.</para>
bos@558 1263
bos@558 1264 <para>If you have a patch that has multiple hunks modifying a
bos@558 1265 file, and you only want to move a few of those hunks, the job
bos@558 1266 becomes more messy, but you can still partly automate it. Use
bos@558 1267 <command>lsdiff -nvv</command> to print some metadata about
bos@567 1268 the patch.</para>
bos@567 1269
bos@567 1270 &interaction.mq.tools.lsdiff;
bos@558 1271
bos@558 1272 <para>This command prints three different kinds of
bos@558 1273 number:</para>
bos@558 1274 <itemizedlist>
bos@558 1275 <listitem><para>(in the first column) a <emphasis>file
bos@558 1276 number</emphasis> to identify each file modified in the
bos@559 1277 patch;</para>
bos@559 1278 </listitem>
bos@558 1279 <listitem><para>(on the next line, indented) the line number
bos@559 1280 within a modified file where a hunk starts; and</para>
bos@559 1281 </listitem>
bos@558 1282 <listitem><para>(on the same line) a <emphasis>hunk
bos@559 1283 number</emphasis> to identify that hunk.</para>
bos@559 1284 </listitem></itemizedlist>
bos@558 1285
bos@558 1286 <para>You'll have to use some visual inspection, and reading of
bos@558 1287 the patch, to identify the file and hunk numbers you'll want,
bos@558 1288 but you can then pass them to to
bos@558 1289 <command>filterdiff</command>'s <option
bos@558 1290 role="cmd-opt-filterdiff">--files</option> and <option
bos@558 1291 role="cmd-opt-filterdiff">--hunks</option> options, to
bos@558 1292 select exactly the file and hunk you want to extract.</para>
bos@558 1293
bos@558 1294 <para>Once you have this hunk, you can concatenate it onto the
bos@558 1295 end of your destination patch and continue with the remainder
dongsheng@625 1296 of section <xref linkend="sec.mq.combine"/>.</para>
bos@558 1297
bos@558 1298 </sect2>
bos@558 1299 </sect1>
bos@558 1300 <sect1>
bos@558 1301 <title>Differences between quilt and MQ</title>
bos@558 1302
bos@558 1303 <para>If you are already familiar with quilt, MQ provides a
bos@558 1304 similar command set. There are a few differences in the way
bos@558 1305 that it works.</para>
bos@558 1306
bos@558 1307 <para>You will already have noticed that most quilt commands have
bos@558 1308 MQ counterparts that simply begin with a
bos@558 1309 <quote><literal>q</literal></quote>. The exceptions are quilt's
bos@558 1310 <literal>add</literal> and <literal>remove</literal> commands,
bos@558 1311 the counterparts for which are the normal Mercurial <command
bos@558 1312 role="hg-cmd">hg add</command> and <command role="hg-cmd">hg
bos@558 1313 remove</command> commands. There is no MQ equivalent of the
bos@558 1314 quilt <literal>edit</literal> command.</para>
bos@558 1315
bos@558 1316 </sect1>
bos@558 1317 </chapter>
bos@558 1318
bos@558 1319 <!--
bos@558 1320 local variables:
bos@558 1321 sgml-parent-document: ("00book.xml" "book" "chapter")
bos@558 1322 end:
bos@558 1323 -->