hgbook

annotate en/ch12-mq.xml @ 558:8631da51309b

Slow progress on XML conversion
author Bryan O'Sullivan <bos@serpentine.com>
date Mon Feb 09 23:25:40 2009 -0800 (2009-02-09)
parents en/ch12-mq.tex@5cd47f721686
children b90b024729f1
rev   line source
bos@558 1 <!-- vim: set filetype=docbkxml shiftwidth=2 autoindent expandtab tw=77 : -->
bos@558 2
bos@558 3 <chapter id="chap:mq">
bos@558 4 <title>Managing change with Mercurial Queues</title>
bos@558 5
bos@558 6 <sect1 id="sec:mq:patch-mgmt">
bos@558 7 <title>The patch management problem</title>
bos@558 8
bos@558 9 <para>Here is a common scenario: you need to install a software
bos@558 10 package from source, but you find a bug that you must fix in the
bos@558 11 source before you can start using the package. You make your
bos@558 12 changes, forget about the package for a while, and a few months
bos@558 13 later you need to upgrade to a newer version of the package. If
bos@558 14 the newer version of the package still has the bug, you must
bos@558 15 extract your fix from the older source tree and apply it against
bos@558 16 the newer version. This is a tedious task, and it's easy to
bos@558 17 make mistakes.</para>
bos@558 18
bos@558 19 <para>This is a simple case of the <quote>patch management</quote>
bos@558 20 problem. You have an <quote>upstream</quote> source tree that
bos@558 21 you can't change; you need to make some local changes on top of
bos@558 22 the upstream tree; and you'd like to be able to keep those
bos@558 23 changes separate, so that you can apply them to newer versions
bos@558 24 of the upstream source.</para>
bos@558 25
bos@558 26 <para>The patch management problem arises in many situations.
bos@558 27 Probably the most visible is that a user of an open source
bos@558 28 software project will contribute a bug fix or new feature to the
bos@558 29 project's maintainers in the form of a patch.</para>
bos@558 30
bos@558 31 <para>Distributors of operating systems that include open source
bos@558 32 software often need to make changes to the packages they
bos@558 33 distribute so that they will build properly in their
bos@558 34 environments.</para>
bos@558 35
bos@558 36 <para>When you have few changes to maintain, it is easy to manage
bos@558 37 a single patch using the standard <command>diff</command> and
bos@558 38 <command>patch</command> programs (see section <xref
bos@558 39 linkend="sec:mq:patch"/> for a discussion of these tools).
bos@558 40 Once the number of changes grows, it starts to make sense to
bos@558 41 maintain patches as discrete <quote>chunks of work,</quote> so
bos@558 42 that for example a single patch will contain only one bug fix
bos@558 43 (the patch might modify several files, but it's doing
bos@558 44 <quote>only one thing</quote>), and you may have a number of
bos@558 45 such patches for different bugs you need fixed and local changes
bos@558 46 you require. In this situation, if you submit a bug fix patch
bos@558 47 to the upstream maintainers of a package and they include your
bos@558 48 fix in a subsequent release, you can simply drop that single
bos@558 49 patch when you're updating to the newer release.</para>
bos@558 50
bos@558 51 <para>Maintaining a single patch against an upstream tree is a
bos@558 52 little tedious and error-prone, but not difficult. However, the
bos@558 53 complexity of the problem grows rapidly as the number of patches
bos@558 54 you have to maintain increases. With more than a tiny number of
bos@558 55 patches in hand, understanding which ones you have applied and
bos@558 56 maintaining them moves from messy to overwhelming.</para>
bos@558 57
bos@558 58 <para>Fortunately, Mercurial includes a powerful extension,
bos@558 59 Mercurial Queues (or simply <quote>MQ</quote>), that massively
bos@558 60 simplifies the patch management problem.</para>
bos@558 61
bos@558 62 </sect1>
bos@558 63 <sect1 id="sec:mq:history">
bos@558 64 <title>The prehistory of Mercurial Queues</title>
bos@558 65
bos@558 66 <para>During the late 1990s, several Linux kernel developers
bos@558 67 started to maintain <quote>patch series</quote> that modified
bos@558 68 the behaviour of the Linux kernel. Some of these series were
bos@558 69 focused on stability, some on feature coverage, and others were
bos@558 70 more speculative.</para>
bos@558 71
bos@558 72 <para>The sizes of these patch series grew rapidly. In 2002,
bos@558 73 Andrew Morton published some shell scripts he had been using to
bos@558 74 automate the task of managing his patch queues. Andrew was
bos@558 75 successfully using these scripts to manage hundreds (sometimes
bos@558 76 thousands) of patches on top of the Linux kernel.</para>
bos@558 77
bos@558 78 <sect2 id="sec:mq:quilt">
bos@558 79 <title>A patchwork quilt</title>
bos@558 80
bos@558 81 <para>In early 2003, Andreas Gruenbacher and Martin Quinson
bos@558 82 borrowed the approach of Andrew's scripts and published a tool
bos@558 83 called <quote>patchwork quilt</quote>
bos@558 84 <citation>web:quilt</citation>, or simply <quote>quilt</quote>
bos@558 85 (see <citation>gruenbacher:2005</citation> for a paper
bos@558 86 describing it). Because quilt substantially automated patch
bos@558 87 management, it rapidly gained a large following among open
bos@558 88 source software developers.</para>
bos@558 89
bos@558 90 <para>Quilt manages a <emphasis>stack of patches</emphasis> on
bos@558 91 top of a directory tree. To begin, you tell quilt to manage a
bos@558 92 directory tree, and tell it which files you want to manage; it
bos@558 93 stores away the names and contents of those files. To fix a
bos@558 94 bug, you create a new patch (using a single command), edit the
bos@558 95 files you need to fix, then <quote>refresh</quote> the
bos@558 96 patch.</para>
bos@558 97
bos@558 98 <para>The refresh step causes quilt to scan the directory tree;
bos@558 99 it updates the patch with all of the changes you have made.
bos@558 100 You can create another patch on top of the first, which will
bos@558 101 track the changes required to modify the tree from <quote>tree
bos@558 102 with one patch applied</quote> to <quote>tree with two
bos@558 103 patches applied</quote>.</para>
bos@558 104
bos@558 105 <para>You can <emphasis>change</emphasis> which patches are
bos@558 106 applied to the tree. If you <quote>pop</quote> a patch, the
bos@558 107 changes made by that patch will vanish from the directory
bos@558 108 tree. Quilt remembers which patches you have popped, though,
bos@558 109 so you can <quote>push</quote> a popped patch again, and the
bos@558 110 directory tree will be restored to contain the modifications
bos@558 111 in the patch. Most importantly, you can run the
bos@558 112 <quote>refresh</quote> command at any time, and the topmost
bos@558 113 applied patch will be updated. This means that you can, at
bos@558 114 any time, change both which patches are applied and what
bos@558 115 modifications those patches make.</para>
bos@558 116
bos@558 117 <para>Quilt knows nothing about revision control tools, so it
bos@558 118 works equally well on top of an unpacked tarball or a
bos@558 119 Subversion working copy.</para>
bos@558 120
bos@558 121 </sect2>
bos@558 122 <sect2 id="sec:mq:quilt-mq">
bos@558 123 <title>From patchwork quilt to Mercurial Queues</title>
bos@558 124
bos@558 125 <para>In mid-2005, Chris Mason took the features of quilt and
bos@558 126 wrote an extension that he called Mercurial Queues, which
bos@558 127 added quilt-like behaviour to Mercurial.</para>
bos@558 128
bos@558 129 <para>The key difference between quilt and MQ is that quilt
bos@558 130 knows nothing about revision control systems, while MQ is
bos@558 131 <emphasis>integrated</emphasis> into Mercurial. Each patch
bos@558 132 that you push is represented as a Mercurial changeset. Pop a
bos@558 133 patch, and the changeset goes away.</para>
bos@558 134
bos@558 135 <para>Because quilt does not care about revision control tools,
bos@558 136 it is still a tremendously useful piece of software to know
bos@558 137 about for situations where you cannot use Mercurial and
bos@558 138 MQ.</para>
bos@558 139
bos@558 140 </sect2>
bos@558 141 </sect1>
bos@558 142 <sect1>
bos@558 143 <title>The huge advantage of MQ</title>
bos@558 144
bos@558 145 <para>I cannot overstate the value that MQ offers through the
bos@558 146 unification of patches and revision control.</para>
bos@558 147
bos@558 148 <para>A major reason that patches have persisted in the free
bos@558 149 software and open source world&emdash;in spite of the availability of
bos@558 150 increasingly capable revision control tools over the years&emdash;is
bos@558 151 the <emphasis>agility</emphasis> they offer.</para>
bos@558 152
bos@558 153 <para>Traditional revision control tools make a permanent,
bos@558 154 irreversible record of everything that you do. While this has
bos@558 155 great value, it's also somewhat stifling. If you want to
bos@558 156 perform a wild-eyed experiment, you have to be careful in how
bos@558 157 you go about it, or you risk leaving unneeded&emdash;or worse,
bos@558 158 misleading or destabilising&emdash;traces of your missteps and errors
bos@558 159 in the permanent revision record.</para>
bos@558 160
bos@558 161 <para>By contrast, MQ's marriage of distributed revision control
bos@558 162 with patches makes it much easier to isolate your work. Your
bos@558 163 patches live on top of normal revision history, and you can make
bos@558 164 them disappear or reappear at will. If you don't like a patch,
bos@558 165 you can drop it. If a patch isn't quite as you want it to be,
bos@558 166 simply fix it&emdash;as many times as you need to, until you have
bos@558 167 refined it into the form you desire.</para>
bos@558 168
bos@558 169 <para>As an example, the integration of patches with revision
bos@558 170 control makes understanding patches and debugging their
bos@558 171 effects&emdash;and their interplay with the code they're based
bos@558 172 on&emdash;<emphasis>enormously</emphasis> easier. Since every applied
bos@558 173 patch has an associated changeset, you can use <command
bos@558 174 role="hg-cmd">hg log</command> to see which changesets and
bos@558 175 patches affected a file. You can use the <literal
bos@558 176 role="hg-ext">bisect</literal> command to binary-search
bos@558 177 through all changesets and applied patches to see where a bug
bos@558 178 got introduced or fixed. You can use the <command
bos@558 179 role="hg-cmd">hg annotate</command> command to see which
bos@558 180 changeset or patch modified a particular line of a source file.
bos@558 181 And so on.</para>
bos@558 182
bos@558 183 </sect1>
bos@558 184
bos@558 185 <sect1 id="sec:mq:patch">
bos@558 186 <title>Understanding patches</title>
bos@558 187
bos@558 188 <para>Because MQ doesn't hide its patch-oriented nature, it is
bos@558 189 helpful to understand what patches are, and a little about the
bos@558 190 tools that work with them.</para>
bos@558 191
bos@558 192 <para>The traditional Unix <command>diff</command> command
bos@558 193 compares two files, and prints a list of differences between
bos@558 194 them. The <command>patch</command> command understands these
bos@558 195 differences as <emphasis>modifications</emphasis> to make to a
bos@558 196 file. Take a look below for a simple example of these commands
bos@558 197 in action.</para>
bos@558 198
bos@558 199 <!-- &interaction.mq.dodiff.diff; -->
bos@558 200
bos@558 201 <para>The type of file that <command>diff</command> generates (and
bos@558 202 <command>patch</command> takes as input) is called a
bos@558 203 <quote>patch</quote> or a <quote>diff</quote>; there is no
bos@558 204 difference between a patch and a diff. (We'll use the term
bos@558 205 <quote>patch</quote>, since it's more commonly used.)</para>
bos@558 206
bos@558 207 <para>A patch file can start with arbitrary text; the
bos@558 208 <command>patch</command> command ignores this text, but MQ uses
bos@558 209 it as the commit message when creating changesets. To find the
bos@558 210 beginning of the patch content, <command>patch</command>
bos@558 211 searches for the first line that starts with the string
bos@558 212 <quote><literal>diff -</literal></quote>.</para>
bos@558 213
bos@558 214 <para>MQ works with <emphasis>unified</emphasis> diffs
bos@558 215 (<command>patch</command> can accept several other diff formats,
bos@558 216 but MQ doesn't). A unified diff contains two kinds of header.
bos@558 217 The <emphasis>file header</emphasis> describes the file being
bos@558 218 modified; it contains the name of the file to modify. When
bos@558 219 <command>patch</command> sees a new file header, it looks for a
bos@558 220 file with that name to start modifying.</para>
bos@558 221
bos@558 222 <para>After the file header comes a series of
bos@558 223 <emphasis>hunks</emphasis>. Each hunk starts with a header;
bos@558 224 this identifies the range of line numbers within the file that
bos@558 225 the hunk should modify. Following the header, a hunk starts and
bos@558 226 ends with a few (usually three) lines of text from the
bos@558 227 unmodified file; these are called the
bos@558 228 <emphasis>context</emphasis> for the hunk. If there's only a
bos@558 229 small amount of context between successive hunks,
bos@558 230 <command>diff</command> doesn't print a new hunk header; it just
bos@558 231 runs the hunks together, with a few lines of context between
bos@558 232 modifications.</para>
bos@558 233
bos@558 234 <para>Each line of context begins with a space character. Within
bos@558 235 the hunk, a line that begins with
bos@558 236 <quote><literal>-</literal></quote> means <quote>remove this
bos@558 237 line,</quote> while a line that begins with
bos@558 238 <quote><literal>+</literal></quote> means <quote>insert this
bos@558 239 line.</quote> For example, a line that is modified is
bos@558 240 represented by one deletion and one insertion.</para>
bos@558 241
bos@558 242 <para>We will return to some of the more subtle aspects of patches
bos@558 243 later (in section <xref linkend="sec:mq:adv-patch"/>), but you
bos@558 244 should have enough information now to use MQ.</para>
bos@558 245
bos@558 246 </sect1>
bos@558 247 <sect1 id="sec:mq:start">
bos@558 248 <title>Getting started with Mercurial Queues</title>
bos@558 249
bos@558 250 <para>Because MQ is implemented as an extension, you must
bos@558 251 explicitly enable before you can use it. (You don't need to
bos@558 252 download anything; MQ ships with the standard Mercurial
bos@558 253 distribution.) To enable MQ, edit your <filename role="home">
bos@558 254 /.hgrc</filename> file, and add the lines below.</para>
bos@558 255
bos@558 256 <programlisting>[extensions] hgext.mq =</programlisting>
bos@558 257
bos@558 258 <para>Once the extension is enabled, it will make a number of new
bos@558 259 commands available. To verify that the extension is working,
bos@558 260 you can use <command role="hg-cmd">hg help</command> to see if
bos@558 261 the <command role="hg-ext-mq">hg qinit</command> command is now
bos@558 262 available.</para>
bos@558 263
bos@558 264 <!-- &interaction.mq.qinit-help.help; -->
bos@558 265
bos@558 266 <para>You can use MQ with <emphasis>any</emphasis> Mercurial
bos@558 267 repository, and its commands only operate within that
bos@558 268 repository. To get started, simply prepare the repository using
bos@558 269 the <command role="hg-ext-mq">hg qinit</command> command. This
bos@558 270 command creates an empty directory called <filename
bos@558 271 role="special" class="directory">.hg/patches</filename>, where
bos@558 272 MQ will keep its metadata. As with many Mercurial commands, the
bos@558 273 <command role="hg-ext-mq">hg qinit</command> command prints nothing
bos@558 274 if it succeeds.</para>
bos@558 275 <!-- &interaction.mq.tutorial.qinit; -->
bos@558 276
bos@558 277
bos@558 278 <sect2>
bos@558 279 <title>Creating a new patch</title>
bos@558 280
bos@558 281 <para>To begin work on a new patch, use the <command
bos@558 282 role="hg-ext-mq">hg qnew</command> command. This command takes
bos@558 283 one argument, the name of the patch to create. MQ will use
bos@558 284 this as the name of an actual file in the <filename
bos@558 285 role="special" class="directory">.hg/patches</filename>
bos@558 286 directory, as you can see in figure <xref
bos@558 287 linkend="ex:mq:qnew"/>.</para>
bos@558 288
bos@558 289 <para>Also newly present in the <filename role="special"
bos@558 290 class="directory">.hg/patches</filename> directory are two
bos@558 291 other files, <filename role="special">series</filename> and
bos@558 292 <filename role="special">status</filename>. The <filename
bos@558 293 role="special">series</filename> file lists all of the
bos@558 294 patches that MQ knows about for this repository, with one
bos@558 295 patch per line. Mercurial uses the <filename
bos@558 296 role="special">status</filename> file for internal
bos@558 297 book-keeping; it tracks all of the patches that MQ has
bos@558 298 <emphasis>applied</emphasis> in this repository.</para>
bos@558 299
bos@558 300 <note>
bos@558 301 <para> You may sometimes want to edit the <filename
bos@558 302 role="special">series</filename> file by hand; for
bos@558 303 example, to change the sequence in which some patches are
bos@558 304 applied. However, manually editing the <filename
bos@558 305 role="special">status</filename> file is almost always a
bos@558 306 bad idea, as it's easy to corrupt MQ's idea of what is
bos@558 307 happening.</para>
bos@558 308 </note>
bos@558 309
bos@558 310 <para>Once you have created your new patch, you can edit files
bos@558 311 in the working directory as you usually would. All of the
bos@558 312 normal Mercurial commands, such as <command role="hg-cmd">hg
bos@558 313 diff</command> and <command role="hg-cmd">hg
bos@558 314 annotate</command>, work exactly as they did before.</para>
bos@558 315
bos@558 316 </sect2>
bos@558 317 <sect2>
bos@558 318 <title>Refreshing a patch</title>
bos@558 319
bos@558 320 <para>When you reach a point where you want to save your work,
bos@558 321 use the <command role="hg-ext-mq">hg qrefresh</command>
bos@558 322 command to update the patch you are working on. This command
bos@558 323 folds the changes you have made in the working directory into
bos@558 324 your patch, and updates its corresponding changeset to contain
bos@558 325 those changes.</para>
bos@558 326
bos@558 327 <!-- &interaction.mq.tutorial.qrefresh; -->
bos@558 328
bos@558 329 <para>You can run <command role="hg-ext-mq">hg hg
bos@558 330 qrefresh</command> as often as you like, so it's a good way
bos@558 331 to <quote>checkpoint</quote> your work. Refresh your patch at
bos@558 332 an opportune time; try an experiment; and if the experiment
bos@558 333 doesn't work out, <command role="hg-cmd">hg revert</command>
bos@558 334 your modifications back to the last time you refreshed.</para>
bos@558 335
bos@558 336 <!-- &interaction.mq.tutorial.qrefresh2; -->
bos@558 337
bos@558 338 </sect2>
bos@558 339 <sect2>
bos@558 340 <title>Stacking and tracking patches</title>
bos@558 341
bos@558 342 <para>Once you have finished working on a patch, or need to work
bos@558 343 on another, you can use the <command role="hg-ext-mq">hg hg
bos@558 344 qnew</command> command again to create a new patch.
bos@558 345 Mercurial will apply this patch on top of your existing patch.
bos@558 346 See figure <xref linkend="ex:mq:qnew2"/> for an example.
bos@558 347 Notice that the patch contains the changes in our prior patch
bos@558 348 as part of its context (you can see this more clearly in the
bos@558 349 output of <command role="hg-cmd">hg
bos@558 350 annotate</command>).</para>
bos@558 351
bos@558 352 <!-- &interaction.mq.tutorial.qnew2; -->
bos@558 353
bos@558 354 <para>So far, with the exception of <command
bos@558 355 role="hg-ext-mq">hg qnew</command> and <command
bos@558 356 role="hg-ext-mq">hg qrefresh</command>, we've been careful to
bos@558 357 only use regular Mercurial commands. However, MQ provides
bos@558 358 many commands that are easier to use when you are thinking
bos@558 359 about patches, as illustrated in figure <xref
bos@558 360 linkend="ex:mq:qseries"/>:</para>
bos@558 361
bos@558 362 <itemizedlist>
bos@558 363 <listitem><para>The <command
bos@558 364 role="hg-ext-mq">hg qseries</command> command lists every
bos@558 365 patch that MQ knows about in this repository, from oldest
bos@558 366 to newest (most recently
bos@558 367 <emphasis>created</emphasis>).</para></listitem>
bos@558 368 <listitem><para>The <command
bos@558 369 role="hg-ext-mq">hg qapplied</command> command lists every
bos@558 370 patch that MQ has <emphasis>applied</emphasis> in this
bos@558 371 repository, again from oldest to newest (most recently
bos@558 372 applied).</para></listitem></itemizedlist>
bos@558 373
bos@558 374 <!-- &interaction.mq.tutorial.qseries; -->
bos@558 375
bos@558 376 </sect2>
bos@558 377 <sect2>
bos@558 378 <title>Manipulating the patch stack</title>
bos@558 379
bos@558 380 <para>The previous discussion implied that there must be a
bos@558 381 difference between <quote>known</quote> and
bos@558 382 <quote>applied</quote> patches, and there is. MQ can manage a
bos@558 383 patch without it being applied in the repository.</para>
bos@558 384
bos@558 385 <para>An <emphasis>applied</emphasis> patch has a corresponding
bos@558 386 changeset in the repository, and the effects of the patch and
bos@558 387 changeset are visible in the working directory. You can undo
bos@558 388 the application of a patch using the <command
bos@558 389 role="hg-ext-mq">hg qpop</command> command. MQ still
bos@558 390 <emphasis>knows about</emphasis>, or manages, a popped patch,
bos@558 391 but the patch no longer has a corresponding changeset in the
bos@558 392 repository, and the working directory does not contain the
bos@558 393 changes made by the patch. Figure <xref
bos@558 394 linkend="fig:mq:stack"/> illustrates the difference between
bos@558 395 applied and tracked patches.</para>
bos@558 396
bos@558 397 <informalfigure id="fig:mq:stack">
bos@558 398 <mediaobject>
bos@558 399 <imageobject><imagedata fileref="mq-stack"/></imageobject>
bos@558 400 <textobject><phrase>XXX add text</phrase></textobject>
bos@558 401 <caption><para>Applied and unapplied patches in the MQ patch
bos@558 402 stack</para></caption>
bos@558 403 </mediaobject>
bos@558 404 </informalfigure>
bos@558 405
bos@558 406 <para>You can reapply an unapplied, or popped, patch using the
bos@558 407 <command role="hg-ext-mq">hg qpush</command> command. This
bos@558 408 creates a new changeset to correspond to the patch, and the
bos@558 409 patch's changes once again become present in the working
bos@558 410 directory. See figure <xref linkend="ex:mq:qpop"/> for
bos@558 411 examples of <command role="hg-ext-mq">hg qpop</command> and
bos@558 412 <command role="hg-ext-mq">hg qpush</command> in action. Notice
bos@558 413 that once we have popped a patch or two patches, the output of
bos@558 414 <command role="hg-ext-mq">hg qseries</command> remains the same,
bos@558 415 while that of <command role="hg-ext-mq">hg qapplied</command> has
bos@558 416 changed.</para>
bos@558 417
bos@558 418 <!-- &interaction.mq.tutorial.qpop; -->
bos@558 419
bos@558 420 </sect2>
bos@558 421 <sect2>
bos@558 422 <title>Pushing and popping many patches</title>
bos@558 423
bos@558 424 <para>While <command role="hg-ext-mq">hg qpush</command> and
bos@558 425 <command role="hg-ext-mq">hg qpop</command> each operate on a
bos@558 426 single patch at a time by default, you can push and pop many
bos@558 427 patches in one go. The <option
bos@558 428 role="hg-ext-mq-cmd-qpush-opt">-a</option> option to
bos@558 429 <command role="hg-ext-mq">hg qpush</command> causes it to push
bos@558 430 all unapplied patches, while the <option
bos@558 431 role="hg-ext-mq-cmd-qpop-opt">-a</option> option to <command
bos@558 432 role="hg-ext-mq">hg qpop</command> causes it to pop all applied
bos@558 433 patches. (For some more ways to push and pop many patches,
bos@558 434 see section <xref linkend="sec:mq:perf"/> below.)</para>
bos@558 435
bos@558 436 <!-- &interaction.mq.tutorial.qpush-a; -->
bos@558 437
bos@558 438 </sect2>
bos@558 439 <sect2>
bos@558 440 <title>Safety checks, and overriding them</title>
bos@558 441
bos@558 442 <para>Several MQ commands check the working directory before
bos@558 443 they do anything, and fail if they find any modifications.
bos@558 444 They do this to ensure that you won't lose any changes that
bos@558 445 you have made, but not yet incorporated into a patch. Figure
bos@558 446 <xref linkend="ex:mq:add"/> illustrates this; the <command
bos@558 447 role="hg-ext-mq">hg qnew</command> command will not create a
bos@558 448 new patch if there are outstanding changes, caused in this
bos@558 449 case by the <command role="hg-cmd">hg add</command> of
bos@558 450 <filename>file3</filename>.</para>
bos@558 451
bos@558 452 <!-- &interaction.mq.tutorial.add; -->
bos@558 453
bos@558 454 <para>Commands that check the working directory all take an
bos@558 455 <quote>I know what I'm doing</quote> option, which is always
bos@558 456 named <option>-f</option>. The exact meaning of
bos@558 457 <option>-f</option> depends on the command. For example,
bos@558 458 <command role="hg-cmd">hg qnew <option
bos@558 459 role="hg-ext-mq-cmd-qnew-opt">-f</option></command> will
bos@558 460 incorporate any outstanding changes into the new patch it
bos@558 461 creates, but <command role="hg-cmd">hg qpop <option
bos@558 462 role="hg-ext-mq-cmd-qpop-opt">-f</option></command> will
bos@558 463 revert modifications to any files affected by the patch that
bos@558 464 it is popping. Be sure to read the documentation for a
bos@558 465 command's <option>-f</option> option before you use it!</para>
bos@558 466
bos@558 467 </sect2>
bos@558 468 <sect2>
bos@558 469 <title>Working on several patches at once</title>
bos@558 470
bos@558 471 <para>The <command role="hg-ext-mq">hg qrefresh</command> command
bos@558 472 always refreshes the <emphasis>topmost</emphasis> applied
bos@558 473 patch. This means that you can suspend work on one patch (by
bos@558 474 refreshing it), pop or push to make a different patch the top,
bos@558 475 and work on <emphasis>that</emphasis> patch for a
bos@558 476 while.</para>
bos@558 477
bos@558 478 <para>Here's an example that illustrates how you can use this
bos@558 479 ability. Let's say you're developing a new feature as two
bos@558 480 patches. The first is a change to the core of your software,
bos@558 481 and the second&emdash;layered on top of the first&emdash;changes the
bos@558 482 user interface to use the code you just added to the core. If
bos@558 483 you notice a bug in the core while you're working on the UI
bos@558 484 patch, it's easy to fix the core. Simply <command
bos@558 485 role="hg-ext-mq">hg qrefresh</command> the UI patch to save
bos@558 486 your in-progress changes, and <command
bos@558 487 role="hg-ext-mq">hg qpop</command> down to the core patch. Fix
bos@558 488 the core bug, <command role="hg-ext-mq">hg qrefresh</command> the
bos@558 489 core patch, and <command role="hg-ext-mq">hg qpush</command> back
bos@558 490 to the UI patch to continue where you left off.</para>
bos@558 491
bos@558 492 </sect2>
bos@558 493 </sect1>
bos@558 494
bos@558 495 <sect1 id="sec:mq:adv-patch">
bos@558 496 <title>More about patches</title>
bos@558 497
bos@558 498 <para>MQ uses the GNU <command>patch</command> command to apply
bos@558 499 patches, so it's helpful to know a few more detailed aspects of
bos@558 500 how <command>patch</command> works, and about patches
bos@558 501 themselves.</para>
bos@558 502
bos@558 503 <sect2>
bos@558 504 <title>The strip count</title>
bos@558 505
bos@558 506 <para>If you look at the file headers in a patch, you will
bos@558 507 notice that the pathnames usually have an extra component on
bos@558 508 the front that isn't present in the actual path name. This is
bos@558 509 a holdover from the way that people used to generate patches
bos@558 510 (people still do this, but it's somewhat rare with modern
bos@558 511 revision control tools).</para>
bos@558 512
bos@558 513 <para>Alice would unpack a tarball, edit her files, then decide
bos@558 514 that she wanted to create a patch. So she'd rename her
bos@558 515 working directory, unpack the tarball again (hence the need
bos@558 516 for the rename), and use the <option
bos@558 517 role="cmd-opt-diff">-r</option> and <option
bos@558 518 role="cmd-opt-diff">-N</option> options to
bos@558 519 <command>diff</command> to recursively generate a patch
bos@558 520 between the unmodified directory and the modified one. The
bos@558 521 result would be that the name of the unmodified directory
bos@558 522 would be at the front of the left-hand path in every file
bos@558 523 header, and the name of the modified directory would be at the
bos@558 524 front of the right-hand path.</para>
bos@558 525
bos@558 526 <para>Since someone receiving a patch from the Alices of the net
bos@558 527 would be unlikely to have unmodified and modified directories
bos@558 528 with exactly the same names, the <command>patch</command>
bos@558 529 command has a <option role="cmd-opt-patch">-p</option> option
bos@558 530 that indicates the number of leading path name components to
bos@558 531 strip when trying to apply a patch. This number is called the
bos@558 532 <emphasis>strip count</emphasis>.</para>
bos@558 533
bos@558 534 <para>An option of <quote><literal>-p1</literal></quote> means
bos@558 535 <quote>use a strip count of one</quote>. If
bos@558 536 <command>patch</command> sees a file name
bos@558 537 <filename>foo/bar/baz</filename> in a file header, it will
bos@558 538 strip <filename>foo</filename> and try to patch a file named
bos@558 539 <filename>bar/baz</filename>. (Strictly speaking, the strip
bos@558 540 count refers to the number of <emphasis>path
bos@558 541 separators</emphasis> (and the components that go with them
bos@558 542 ) to strip. A strip count of one will turn
bos@558 543 <filename>foo/bar</filename> into <filename>bar</filename>,
bos@558 544 but <filename>/foo/bar</filename> (notice the extra leading
bos@558 545 slash) into <filename>foo/bar</filename>.)</para>
bos@558 546
bos@558 547 <para>The <quote>standard</quote> strip count for patches is
bos@558 548 one; almost all patches contain one leading path name
bos@558 549 component that needs to be stripped. Mercurial's <command
bos@558 550 role="hg-cmd">hg diff</command> command generates path names
bos@558 551 in this form, and the <command role="hg-cmd">hg
bos@558 552 import</command> command and MQ expect patches to have a
bos@558 553 strip count of one.</para>
bos@558 554
bos@558 555 <para>If you receive a patch from someone that you want to add
bos@558 556 to your patch queue, and the patch needs a strip count other
bos@558 557 than one, you cannot just <command
bos@558 558 role="hg-ext-mq">hg qimport</command> the patch, because
bos@558 559 <command role="hg-ext-mq">hg qimport</command> does not yet have
bos@558 560 a <literal>-p</literal> option (see <ulink role="hg-bug"
bos@558 561 url="http://www.selenic.com/mercurial/bts/issue311">issue
bos@558 562 311</ulink>). Your best bet is to <command
bos@558 563 role="hg-ext-mq">hg qnew</command> a patch of your own, then
bos@558 564 use <command>patch -pN</command> to apply their patch,
bos@558 565 followed by <command role="hg-cmd">hg addremove</command> to
bos@558 566 pick up any files added or removed by the patch, followed by
bos@558 567 <command role="hg-ext-mq">hg qrefresh</command>. This complexity
bos@558 568 may become unnecessary; see <ulink role="hg-bug"
bos@558 569 url="http://www.selenic.com/mercurial/bts/issue311">issue
bos@558 570 311</ulink> for details.</para>
bos@558 571 </sect2>
bos@558 572 <sect2>
bos@558 573 <title>Strategies for applying a patch</title>
bos@558 574
bos@558 575 <para>When <command>patch</command> applies a hunk, it tries a
bos@558 576 handful of successively less accurate strategies to try to
bos@558 577 make the hunk apply. This falling-back technique often makes
bos@558 578 it possible to take a patch that was generated against an old
bos@558 579 version of a file, and apply it against a newer version of
bos@558 580 that file.</para>
bos@558 581
bos@558 582 <para>First, <command>patch</command> tries an exact match,
bos@558 583 where the line numbers, the context, and the text to be
bos@558 584 modified must apply exactly. If it cannot make an exact
bos@558 585 match, it tries to find an exact match for the context,
bos@558 586 without honouring the line numbering information. If this
bos@558 587 succeeds, it prints a line of output saying that the hunk was
bos@558 588 applied, but at some <emphasis>offset</emphasis> from the
bos@558 589 original line number.</para>
bos@558 590
bos@558 591 <para>If a context-only match fails, <command>patch</command>
bos@558 592 removes the first and last lines of the context, and tries a
bos@558 593 <emphasis>reduced</emphasis> context-only match. If the hunk
bos@558 594 with reduced context succeeds, it prints a message saying that
bos@558 595 it applied the hunk with a <emphasis>fuzz factor</emphasis>
bos@558 596 (the number after the fuzz factor indicates how many lines of
bos@558 597 context <command>patch</command> had to trim before the patch
bos@558 598 applied).</para>
bos@558 599
bos@558 600 <para>When neither of these techniques works,
bos@558 601 <command>patch</command> prints a message saying that the hunk
bos@558 602 in question was rejected. It saves rejected hunks (also
bos@558 603 simply called <quote>rejects</quote>) to a file with the same
bos@558 604 name, and an added <filename role="special">.rej</filename>
bos@558 605 extension. It also saves an unmodified copy of the file with
bos@558 606 a <filename role="special">.orig</filename> extension; the
bos@558 607 copy of the file without any extensions will contain any
bos@558 608 changes made by hunks that <emphasis>did</emphasis> apply
bos@558 609 cleanly. If you have a patch that modifies
bos@558 610 <filename>foo</filename> with six hunks, and one of them fails
bos@558 611 to apply, you will have: an unmodified
bos@558 612 <filename>foo.orig</filename>, a <filename>foo.rej</filename>
bos@558 613 containing one hunk, and <filename>foo</filename>, containing
bos@558 614 the changes made by the five successful hunks.</para>
bos@558 615
bos@558 616 </sect2>
bos@558 617 <sect2>
bos@558 618 <title>Some quirks of patch representation</title>
bos@558 619
bos@558 620 <para>There are a few useful things to know about how
bos@558 621 <command>patch</command> works with files.</para>
bos@558 622
bos@558 623 <itemizedlist>
bos@558 624 <listitem><para>This should already be obvious, but
bos@558 625 <command>patch</command> cannot handle binary
bos@558 626 files.</para></listitem>
bos@558 627
bos@558 628 <listitem><para>Neither does it care about the executable bit;
bos@558 629 it creates new files as readable, but not
bos@558 630 executable.</para></listitem>
bos@558 631
bos@558 632 <listitem><para><command>patch</command> treats the removal of
bos@558 633 a file as a diff between the file to be removed and the
bos@558 634 empty file. So your idea of <quote>I deleted this
bos@558 635 file</quote> looks like <quote>every line of this file
bos@558 636 was deleted</quote> in a patch.</para></listitem>
bos@558 637
bos@558 638 <listitem><para>It treats the addition of a file as a diff
bos@558 639 between the empty file and the file to be added. So in a
bos@558 640 patch, your idea of <quote>I added this file</quote> looks
bos@558 641 like <quote>every line of this file was
bos@558 642 added</quote>.</para></listitem>
bos@558 643
bos@558 644 <listitem><para>It treats a renamed file as the removal of the
bos@558 645 old name, and the addition of the new name. This means
bos@558 646 that renamed files have a big footprint in patches. (Note
bos@558 647 also that Mercurial does not currently try to infer when
bos@558 648 files have been renamed or copied in a
bos@558 649 patch.)</para></listitem>
bos@558 650
bos@558 651 <listitem><para><command>patch</command> cannot represent
bos@558 652 empty files, so you cannot use a patch to represent the
bos@558 653 notion <quote>I added this empty file to the
bos@558 654 tree</quote>.</para></listitem>
bos@558 655 </itemizedlist>
bos@558 656 </sect2>
bos@558 657
bos@558 658 <sect2>
bos@558 659 <title>Beware the fuzz</title>
bos@558 660
bos@558 661 <para>While applying a hunk at an offset, or with a fuzz factor,
bos@558 662 will often be completely successful, these inexact techniques
bos@558 663 naturally leave open the possibility of corrupting the patched
bos@558 664 file. The most common cases typically involve applying a
bos@558 665 patch twice, or at an incorrect location in the file. If
bos@558 666 <command>patch</command> or <command
bos@558 667 role="hg-ext-mq">hg qpush</command> ever mentions an offset or
bos@558 668 fuzz factor, you should make sure that the modified files are
bos@558 669 correct afterwards.</para>
bos@558 670
bos@558 671 <para>It's often a good idea to refresh a patch that has applied
bos@558 672 with an offset or fuzz factor; refreshing the patch generates
bos@558 673 new context information that will make it apply cleanly. I
bos@558 674 say <quote>often,</quote> not <quote>always,</quote> because
bos@558 675 sometimes refreshing a patch will make it fail to apply
bos@558 676 against a different revision of the underlying files. In some
bos@558 677 cases, such as when you're maintaining a patch that must sit
bos@558 678 on top of multiple versions of a source tree, it's acceptable
bos@558 679 to have a patch apply with some fuzz, provided you've verified
bos@558 680 the results of the patching process in such cases.</para>
bos@558 681
bos@558 682 </sect2>
bos@558 683 <sect2>
bos@558 684 <title>Handling rejection</title>
bos@558 685
bos@558 686 <para>If <command role="hg-ext-mq">hg qpush</command> fails to
bos@558 687 apply a patch, it will print an error message and exit. If it
bos@558 688 has left <filename role="special">.rej</filename> files
bos@558 689 behind, it is usually best to fix up the rejected hunks before
bos@558 690 you push more patches or do any further work.</para>
bos@558 691
bos@558 692 <para>If your patch <emphasis>used to</emphasis> apply cleanly,
bos@558 693 and no longer does because you've changed the underlying code
bos@558 694 that your patches are based on, Mercurial Queues can help; see
bos@558 695 section <xref linkend="sec:mq:merge"/> for details.</para>
bos@558 696
bos@558 697 <para>Unfortunately, there aren't any great techniques for
bos@558 698 dealing with rejected hunks. Most often, you'll need to view
bos@558 699 the <filename role="special">.rej</filename> file and edit the
bos@558 700 target file, applying the rejected hunks by hand.</para>
bos@558 701
bos@558 702 <para>If you're feeling adventurous, Neil Brown, a Linux kernel
bos@558 703 hacker, wrote a tool called <command>wiggle</command>
bos@558 704 <citation>web:wiggle</citation>, which is more vigorous than
bos@558 705 <command>patch</command> in its attempts to make a patch
bos@558 706 apply.</para>
bos@558 707
bos@558 708 <para>Another Linux kernel hacker, Chris Mason (the author of
bos@558 709 Mercurial Queues), wrote a similar tool called
bos@558 710 <command>mpatch</command> <citation>web:mpatch</citation>,
bos@558 711 which takes a simple approach to automating the application of
bos@558 712 hunks rejected by <command>patch</command>. The
bos@558 713 <command>mpatch</command> command can help with four common
bos@558 714 reasons that a hunk may be rejected:</para>
bos@558 715
bos@558 716 <itemizedlist>
bos@558 717 <listitem><para>The context in the middle of a hunk has
bos@558 718 changed.</para></listitem>
bos@558 719
bos@558 720 <listitem><para>A hunk is missing some context at the
bos@558 721 beginning or end.</para></listitem>
bos@558 722
bos@558 723 <listitem><para>A large hunk might apply better&emdash;either
bos@558 724 entirely or in part&emdash;if it was broken up into smaller
bos@558 725 hunks.</para></listitem>
bos@558 726
bos@558 727 <listitem><para>A hunk removes lines with slightly different
bos@558 728 content than those currently present in the
bos@558 729 file.</para></listitem>
bos@558 730 </itemizedlist>
bos@558 731
bos@558 732 <para>If you use <command>wiggle</command> or
bos@558 733 <command>mpatch</command>, you should be doubly careful to
bos@558 734 check your results when you're done. In fact,
bos@558 735 <command>mpatch</command> enforces this method of
bos@558 736 double-checking the tool's output, by automatically dropping
bos@558 737 you into a merge program when it has done its job, so that you
bos@558 738 can verify its work and finish off any remaining
bos@558 739 merges.</para>
bos@558 740
bos@558 741 </sect2>
bos@558 742 </sect1>
bos@558 743 <sect1 id="sec:mq:perf">
bos@558 744 <title>Getting the best performance out of MQ</title>
bos@558 745
bos@558 746 <para>MQ is very efficient at handling a large number of patches.
bos@558 747 I ran some performance experiments in mid-2006 for a talk that I
bos@558 748 gave at the 2006 EuroPython conference
bos@558 749 <citation>web:europython</citation>. I used as my data set the
bos@558 750 Linux 2.6.17-mm1 patch series, which consists of 1,738 patches.
bos@558 751 I applied these on top of a Linux kernel repository containing
bos@558 752 all 27,472 revisions between Linux 2.6.12-rc2 and Linux
bos@558 753 2.6.17.</para>
bos@558 754
bos@558 755 <para>On my old, slow laptop, I was able to <command
bos@558 756 role="hg-cmd">hg qpush <option
bos@558 757 role="hg-ext-mq-cmd-qpush-opt">-a</option></command> all
bos@558 758 1,738 patches in 3.5 minutes, and <command role="hg-cmd">hg qpop
bos@558 759 <option role="hg-ext-mq-cmd-qpop-opt">-a</option></command>
bos@558 760 them all in 30 seconds. (On a newer laptop, the time to push
bos@558 761 all patches dropped to two minutes.) I could <command
bos@558 762 role="hg-ext-mq">hg qrefresh</command> one of the biggest patches
bos@558 763 (which made 22,779 lines of changes to 287 files) in 6.6
bos@558 764 seconds.</para>
bos@558 765
bos@558 766 <para>Clearly, MQ is well suited to working in large trees, but
bos@558 767 there are a few tricks you can use to get the best performance
bos@558 768 of it.</para>
bos@558 769
bos@558 770 <para>First of all, try to <quote>batch</quote> operations
bos@558 771 together. Every time you run <command
bos@558 772 role="hg-ext-mq">hg qpush</command> or <command
bos@558 773 role="hg-ext-mq">hg qpop</command>, these commands scan the
bos@558 774 working directory once to make sure you haven't made some
bos@558 775 changes and then forgotten to run <command
bos@558 776 role="hg-ext-mq">hg qrefresh</command>. On a small tree, the
bos@558 777 time that this scan takes is unnoticeable. However, on a
bos@558 778 medium-sized tree (containing tens of thousands of files), it
bos@558 779 can take a second or more.</para>
bos@558 780
bos@558 781 <para>The <command role="hg-ext-mq">hg qpush</command> and <command
bos@558 782 role="hg-ext-mq">hg qpop</command> commands allow you to push and
bos@558 783 pop multiple patches at a time. You can identify the
bos@558 784 <quote>destination patch</quote> that you want to end up at.
bos@558 785 When you <command role="hg-ext-mq">hg qpush</command> with a
bos@558 786 destination specified, it will push patches until that patch is
bos@558 787 at the top of the applied stack. When you <command
bos@558 788 role="hg-ext-mq">hg qpop</command> to a destination, MQ will pop
bos@558 789 patches until the destination patch is at the top.</para>
bos@558 790
bos@558 791 <para>You can identify a destination patch using either the name
bos@558 792 of the patch, or by number. If you use numeric addressing,
bos@558 793 patches are counted from zero; this means that the first patch
bos@558 794 is zero, the second is one, and so on.</para>
bos@558 795
bos@558 796 </sect1>
bos@558 797
bos@558 798 <sect1 id="sec:mq:merge">
bos@558 799 <title>Updating your patches when the underlying code
bos@558 800 changes</title>
bos@558 801
bos@558 802 <para>It's common to have a stack of patches on top of an
bos@558 803 underlying repository that you don't modify directly. If you're
bos@558 804 working on changes to third-party code, or on a feature that is
bos@558 805 taking longer to develop than the rate of change of the code
bos@558 806 beneath, you will often need to sync up with the underlying
bos@558 807 code, and fix up any hunks in your patches that no longer apply.
bos@558 808 This is called <emphasis>rebasing</emphasis> your patch
bos@558 809 series.</para>
bos@558 810
bos@558 811 <para>The simplest way to do this is to <command role="hg-cmd">hg
bos@558 812 qpop <option
bos@558 813 role="hg-ext-mq-cmd-qpop-opt">-a</option></command> your
bos@558 814 patches, then <command role="hg-cmd">hg pull</command> changes
bos@558 815 into the underlying repository, and finally <command
bos@558 816 role="hg-cmd">hg qpush <option
bos@558 817 role="hg-ext-mq-cmd-qpop-opt">-a</option></command> your
bos@558 818 patches again. MQ will stop pushing any time it runs across a
bos@558 819 patch that fails to apply during conflicts, allowing you to fix
bos@558 820 your conflicts, <command role="hg-ext-mq">hg qrefresh</command> the
bos@558 821 affected patch, and continue pushing until you have fixed your
bos@558 822 entire stack.</para>
bos@558 823
bos@558 824 <para>This approach is easy to use and works well if you don't
bos@558 825 expect changes to the underlying code to affect how well your
bos@558 826 patches apply. If your patch stack touches code that is modified
bos@558 827 frequently or invasively in the underlying repository, however,
bos@558 828 fixing up rejected hunks by hand quickly becomes
bos@558 829 tiresome.</para>
bos@558 830
bos@558 831 <para>It's possible to partially automate the rebasing process. If
bos@558 832 your patches apply cleanly against some revision of the
bos@558 833 underlying repo, MQ can use this information to help you to
bos@558 834 resolve conflicts between your patches and a different
bos@558 835 revision.</para>
bos@558 836
bos@558 837 <para>The process is a little involved.</para>
bos@558 838 <orderedlist>
bos@558 839 <listitem><para>To begin, <command role="hg-cmd">hg qpush
bos@558 840 -a</command> all of your patches on top of the revision
bos@558 841 where you know that they apply cleanly.</para></listitem>
bos@558 842
bos@558 843 <listitem><para>Save a backup copy of your patch directory using
bos@558 844 <command role="hg-cmd">hg qsave <option
bos@558 845 role="hg-ext-mq-cmd-qsave-opt">-e</option> <option
bos@558 846 role="hg-ext-mq-cmd-qsave-opt">-c</option></command>.
bos@558 847 This prints the name of the directory that it has saved the
bos@558 848 patches in. It will save the patches to a directory called
bos@558 849 <filename role="special"
bos@558 850 class="directory">.hg/patches.N</filename>, where
bos@558 851 <literal>N</literal> is a small integer. It also commits a
bos@558 852 <quote>save changeset</quote> on top of your applied
bos@558 853 patches; this is for internal book-keeping, and records the
bos@558 854 states of the <filename role="special">series</filename> and
bos@558 855 <filename role="special">status</filename>
bos@558 856 files.</para></listitem>
bos@558 857
bos@558 858 <listitem>
bos@558 859 <para>Use <command role="hg-cmd">hg pull</command> to bring
bos@558 860 new changes into the underlying repository. (Don't run
bos@558 861 <command role="hg-cmd">hg pull -u</command>; see below for
bos@558 862 why.)</para></listitem>
bos@558 863
bos@558 864 <listitem><para>Update to the new tip revision, using <command
bos@558 865 role="hg-cmd">hg update <option
bos@558 866 role="hg-opt-update">-C</option></command> to override
bos@558 867 the patches you have pushed.</para></listitem>
bos@558 868
bos@558 869 <listitem><para>Merge all patches using
bos@558 870 \hgcmdargs{qpush}{<option
bos@558 871 role="hg-ext-mq-cmd-qpush-opt">-m</option> <option
bos@558 872 role="hg-ext-mq-cmd-qpush-opt">-a</option>}. The <option
bos@558 873 role="hg-ext-mq-cmd-qpush-opt">-m</option> option to
bos@558 874 <command role="hg-ext-mq">hg qpush</command> tells MQ to
bos@558 875 perform a three-way merge if the patch fails to
bos@558 876 apply.</para></listitem>
bos@558 877 </orderedlist>
bos@558 878
bos@558 879 <para>During the <command role="hg-cmd">hg qpush <option
bos@558 880 role="hg-ext-mq-cmd-qpush-opt">-m</option></command>, each
bos@558 881 patch in the <filename role="special">series</filename> file is
bos@558 882 applied normally. If a patch applies with fuzz or rejects, MQ
bos@558 883 looks at the queue you <command
bos@558 884 role="hg-ext-mq">hg qsave</command>d, and performs a three-way
bos@558 885 merge with the corresponding changeset. This merge uses
bos@558 886 Mercurial's normal merge machinery, so it may pop up a GUI merge
bos@558 887 tool to help you to resolve problems.</para>
bos@558 888
bos@558 889 <para>When you finish resolving the effects of a patch, MQ
bos@558 890 refreshes your patch based on the result of the merge.</para>
bos@558 891
bos@558 892 <para>At the end of this process, your repository will have one
bos@558 893 extra head from the old patch queue, and a copy of the old patch
bos@558 894 queue will be in <filename role="special"
bos@558 895 class="directory">.hg/patches.N</filename>. You can remove the
bos@558 896 extra head using <command role="hg-cmd">hg qpop <option
bos@558 897 role="hg-ext-mq-cmd-qpop-opt">-a</option> <option
bos@558 898 role="hg-ext-mq-cmd-qpop-opt">-n</option>
bos@558 899 patches.N</command> or <command role="hg-cmd">hg
bos@558 900 strip</command>. You can delete <filename role="special"
bos@558 901 class="directory">.hg/patches.N</filename> once you are sure
bos@558 902 that you no longer need it as a backup.</para>
bos@558 903
bos@558 904 </sect1>
bos@558 905 <sect1>
bos@558 906 <title>Identifying patches</title>
bos@558 907
bos@558 908 <para>MQ commands that work with patches let you refer to a patch
bos@558 909 either by using its name or by a number. By name is obvious
bos@558 910 enough; pass the name <filename>foo.patch</filename> to <command
bos@558 911 role="hg-ext-mq">hg qpush</command>, for example, and it will
bos@558 912 push patches until <filename>foo.patch</filename> is
bos@558 913 applied.</para>
bos@558 914
bos@558 915 <para>As a shortcut, you can refer to a patch using both a name
bos@558 916 and a numeric offset; <literal>foo.patch-2</literal> means
bos@558 917 <quote>two patches before <literal>foo.patch</literal></quote>,
bos@558 918 while <literal>bar.patch+4</literal> means <quote>four patches
bos@558 919 after <literal>bar.patch</literal></quote>.</para>
bos@558 920
bos@558 921 <para>Referring to a patch by index isn't much different. The
bos@558 922 first patch printed in the output of <command
bos@558 923 role="hg-ext-mq">hg qseries</command> is patch zero (yes, it's
bos@558 924 one of those start-at-zero counting systems); the second is
bos@558 925 patch one; and so on.</para>
bos@558 926
bos@558 927 <para>MQ also makes it easy to work with patches when you are
bos@558 928 using normal Mercurial commands. Every command that accepts a
bos@558 929 changeset ID will also accept the name of an applied patch. MQ
bos@558 930 augments the tags normally in the repository with an eponymous
bos@558 931 one for each applied patch. In addition, the special tags
bos@558 932 \index{tags!special tag
bos@558 933 names!<literal>qbase</literal>}<literal>qbase</literal> and
bos@558 934 \index{tags!special tag
bos@558 935 names!<literal>qtip</literal>}<literal>qtip</literal> identify
bos@558 936 the <quote>bottom-most</quote> and topmost applied patches,
bos@558 937 respectively.</para>
bos@558 938
bos@558 939 <para>These additions to Mercurial's normal tagging capabilities
bos@558 940 make dealing with patches even more of a breeze.</para>
bos@558 941 <itemizedlist>
bos@558 942 <listitem><para>Want to patchbomb a mailing list with your
bos@558 943 latest series of changes?</para>
bos@558 944 <programlisting>
bos@558 945 hg email qbase:qtip
bos@558 946 </programlisting></listitem>
bos@558 947 <listitem><para> (Don't know what <quote>patchbombing</quote>
bos@558 948 is? See section <xref
bos@558 949 linkend="sec:hgext:patchbomb"/>.)</para></listitem>
bos@558 950
bos@558 951 <listitem><para>Need to see all of the patches since
bos@558 952 <literal>foo.patch</literal> that have touched files in a
bos@558 953 subdirectory of your tree?</para>
bos@558 954 <programlisting>
bos@558 955 hg log -r foo.patch:qtip <emphasis>subdir</emphasis>
bos@558 956 </programlisting></listitem>
bos@558 957 </itemizedlist>
bos@558 958
bos@558 959 <para>Because MQ makes the names of patches available to the rest
bos@558 960 of Mercurial through its normal internal tag machinery, you
bos@558 961 don't need to type in the entire name of a patch when you want
bos@558 962 to identify it by name.</para>
bos@558 963
bos@558 964 <!-- &interaction.mq.id.output; -->
bos@558 965
bos@558 966 <para>Another nice consequence of representing patch names as tags
bos@558 967 is that when you run the <command role="hg-cmd">hg log</command>
bos@558 968 command, it will display a patch's name as a tag, simply as part
bos@558 969 of its normal output. This makes it easy to visually
bos@558 970 distinguish applied patches from underlying
bos@558 971 <quote>normal</quote> revisions. Figure <xref
bos@558 972 linkend="ex:mq:id"/> shows a few normal Mercurial commands in
bos@558 973 use with applied patches.</para>
bos@558 974
bos@558 975 </sect1>
bos@558 976 <sect1>
bos@558 977 <title>Useful things to know about</title>
bos@558 978
bos@558 979 <para>There are a number of aspects of MQ usage that don't fit
bos@558 980 tidily into sections of their own, but that are good to know.
bos@558 981 Here they are, in one place.</para>
bos@558 982
bos@558 983 <itemizedlist>
bos@558 984 <listitem><para>Normally, when you <command
bos@558 985 role="hg-ext-mq">hg qpop</command> a patch and <command
bos@558 986 role="hg-ext-mq">hg qpush</command> it again, the changeset
bos@558 987 that represents the patch after the pop/push will have a
bos@558 988 <emphasis>different identity</emphasis> than the changeset
bos@558 989 that represented the hash beforehand. See section <xref
bos@558 990 linkend="sec:mqref:cmd:qpush"/> for information as to why
bos@558 991 this is.</para></listitem>
bos@558 992
bos@558 993 <listitem><para>It's not a good idea to <command
bos@558 994 role="hg-cmd">hg merge</command> changes from another
bos@558 995 branch with a patch changeset, at least if you want to
bos@558 996 maintain the <quote>patchiness</quote> of that changeset and
bos@558 997 changesets below it on the patch stack. If you try to do
bos@558 998 this, it will appear to succeed, but MQ will become
bos@558 999 confused.</para></listitem>
bos@558 1000 </itemizedlist>
bos@558 1001
bos@558 1002 </sect1>
bos@558 1003
bos@558 1004 <sect1 id="sec:mq:repo">
bos@558 1005 <title>Managing patches in a repository</title>
bos@558 1006
bos@558 1007 <para>Because MQ's <filename role="special"
bos@558 1008 class="directory">.hg/patches</filename> directory resides
bos@558 1009 outside a Mercurial repository's working directory, the
bos@558 1010 <quote>underlying</quote> Mercurial repository knows nothing
bos@558 1011 about the management or presence of patches.</para>
bos@558 1012
bos@558 1013 <para>This presents the interesting possibility of managing the
bos@558 1014 contents of the patch directory as a Mercurial repository in its
bos@558 1015 own right. This can be a useful way to work. For example, you
bos@558 1016 can work on a patch for a while, <command
bos@558 1017 role="hg-ext-mq">hg qrefresh</command> it, then <command
bos@558 1018 role="hg-cmd">hg commit</command> the current state of the
bos@558 1019 patch. This lets you <quote>roll back</quote> to that version
bos@558 1020 of the patch later on.</para>
bos@558 1021
bos@558 1022 <para>You can then share different versions of the same patch
bos@558 1023 stack among multiple underlying repositories. I use this when I
bos@558 1024 am developing a Linux kernel feature. I have a pristine copy of
bos@558 1025 my kernel sources for each of several CPU architectures, and a
bos@558 1026 cloned repository under each that contains the patches I am
bos@558 1027 working on. When I want to test a change on a different
bos@558 1028 architecture, I push my current patches to the patch repository
bos@558 1029 associated with that kernel tree, pop and push all of my
bos@558 1030 patches, and build and test that kernel.</para>
bos@558 1031
bos@558 1032 <para>Managing patches in a repository makes it possible for
bos@558 1033 multiple developers to work on the same patch series without
bos@558 1034 colliding with each other, all on top of an underlying source
bos@558 1035 base that they may or may not control.</para>
bos@558 1036
bos@558 1037 <sect2>
bos@558 1038 <title>MQ support for patch repositories</title>
bos@558 1039
bos@558 1040 <para>MQ helps you to work with the <filename role="special"
bos@558 1041 class="directory">.hg/patches</filename> directory as a
bos@558 1042 repository; when you prepare a repository for working with
bos@558 1043 patches using <command role="hg-ext-mq">hg qinit</command>, you
bos@558 1044 can pass the <option
bos@558 1045 role="hg-ext-mq-cmd-qinit-opt">-c</option> option to create
bos@558 1046 the <filename role="special"
bos@558 1047 class="directory">.hg/patches</filename> directory as a
bos@558 1048 Mercurial repository.</para>
bos@558 1049
bos@558 1050 <note>
bos@558 1051 <para> If you forget to use the <option
bos@558 1052 role="hg-ext-mq-cmd-qinit-opt">-c</option> option, you can
bos@558 1053 simply go into the <filename role="special"
bos@558 1054 class="directory">.hg/patches</filename> directory at any
bos@558 1055 time and run <command role="hg-cmd">hg init</command>. Don't
bos@558 1056 forget to add an entry for the <filename
bos@558 1057 role="special">status</filename> file to the <filename
bos@558 1058 role="special">.hgignore</filename> file, though</para>
bos@558 1059
bos@558 1060 <para> (<command role="hg-cmd">hg qinit <option
bos@558 1061 role="hg-ext-mq-cmd-qinit-opt">-c</option></command>
bos@558 1062 does this for you automatically); you
bos@558 1063 <emphasis>really</emphasis> don't want to manage the
bos@558 1064 <filename role="special">status</filename> file.</para>
bos@558 1065 </note>
bos@558 1066
bos@558 1067 <para>As a convenience, if MQ notices that the <filename
bos@558 1068 class="directory">.hg/patches</filename> directory is a
bos@558 1069 repository, it will automatically <command role="hg-cmd">hg
bos@558 1070 add</command> every patch that you create and import.</para>
bos@558 1071
bos@558 1072 <para>MQ provides a shortcut command, <command
bos@558 1073 role="hg-ext-mq">hg qcommit</command>, that runs <command
bos@558 1074 role="hg-cmd">hg commit</command> in the <filename
bos@558 1075 role="special" class="directory">.hg/patches</filename>
bos@558 1076 directory. This saves some bothersome typing.</para>
bos@558 1077
bos@558 1078 <para>Finally, as a convenience to manage the patch directory,
bos@558 1079 you can define the alias <command>mq</command> on Unix
bos@558 1080 systems. For example, on Linux systems using the
bos@558 1081 <command>bash</command> shell, you can include the following
bos@558 1082 snippet in your <filename role="home">
bos@558 1083 /.bashrc</filename>.</para>
bos@558 1084
bos@558 1085 <programlisting>
bos@558 1086 alias mq=`hg -R $(hg root)/.hg/patches'
bos@558 1087 </programlisting>
bos@558 1088
bos@558 1089 <para>You can then issue commands of the form <command>mq
bos@558 1090 pull</command> from the main repository.</para>
bos@558 1091
bos@558 1092 </sect2>
bos@558 1093 <sect2>
bos@558 1094 <title>A few things to watch out for</title>
bos@558 1095
bos@558 1096 <para>MQ's support for working with a repository full of patches
bos@558 1097 is limited in a few small respects.</para>
bos@558 1098
bos@558 1099 <para>MQ cannot automatically detect changes that you make to
bos@558 1100 the patch directory. If you <command role="hg-cmd">hg
bos@558 1101 pull</command>, manually edit, or <command role="hg-cmd">hg
bos@558 1102 update</command> changes to patches or the <filename
bos@558 1103 role="special">series</filename> file, you will have to
bos@558 1104 <command role="hg-cmd">hg qpop <option
bos@558 1105 role="hg-ext-mq-cmd-qpop-opt">-a</option></command> and
bos@558 1106 then <command role="hg-cmd">hg qpush <option
bos@558 1107 role="hg-ext-mq-cmd-qpush-opt">-a</option></command> in
bos@558 1108 the underlying repository to see those changes show up there.
bos@558 1109 If you forget to do this, you can confuse MQ's idea of which
bos@558 1110 patches are applied.</para>
bos@558 1111
bos@558 1112 </sect2>
bos@558 1113 </sect1>
bos@558 1114 <sect1 id="sec:mq:tools">
bos@558 1115 <title>Third party tools for working with patches</title>
bos@558 1116
bos@558 1117 <para>Once you've been working with patches for a while, you'll
bos@558 1118 find yourself hungry for tools that will help you to understand
bos@558 1119 and manipulate the patches you're dealing with.</para>
bos@558 1120
bos@558 1121 <para>The <command>diffstat</command> command
bos@558 1122 <citation>web:diffstat</citation> generates a histogram of the
bos@558 1123 modifications made to each file in a patch. It provides a good
bos@558 1124 way to <quote>get a sense of</quote> a patch&emdash;which files it
bos@558 1125 affects, and how much change it introduces to each file and as a
bos@558 1126 whole. (I find that it's a good idea to use
bos@558 1127 <command>diffstat</command>'s <option
bos@558 1128 role="cmd-opt-diffstat">-p</option> option as a matter of
bos@558 1129 course, as otherwise it will try to do clever things with
bos@558 1130 prefixes of file names that inevitably confuse at least
bos@558 1131 me.)</para>
bos@558 1132
bos@558 1133 <!-- &interaction.mq.tools.tools; -->
bos@558 1134
bos@558 1135 <para>The <literal role="package">patchutils</literal> package
bos@558 1136 <citation>web:patchutils</citation> is invaluable. It provides a
bos@558 1137 set of small utilities that follow the <quote>Unix
bos@558 1138 philosophy;</quote> each does one useful thing with a patch.
bos@558 1139 The <literal role="package">patchutils</literal> command I use
bos@558 1140 most is <command>filterdiff</command>, which extracts subsets
bos@558 1141 from a patch file. For example, given a patch that modifies
bos@558 1142 hundreds of files across dozens of directories, a single
bos@558 1143 invocation of <command>filterdiff</command> can generate a
bos@558 1144 smaller patch that only touches files whose names match a
bos@558 1145 particular glob pattern. See section <xref
bos@558 1146 linkend="mq-collab:tips:interdiff"/> for another
bos@558 1147 example.</para>
bos@558 1148
bos@558 1149 </sect1>
bos@558 1150 <sect1>
bos@558 1151 <title>Good ways to work with patches</title>
bos@558 1152
bos@558 1153 <para>Whether you are working on a patch series to submit to a
bos@558 1154 free software or open source project, or a series that you
bos@558 1155 intend to treat as a sequence of regular changesets when you're
bos@558 1156 done, you can use some simple techniques to keep your work well
bos@558 1157 organised.</para>
bos@558 1158
bos@558 1159 <para>Give your patches descriptive names. A good name for a
bos@558 1160 patch might be <filename>rework-device-alloc.patch</filename>,
bos@558 1161 because it will immediately give you a hint what the purpose of
bos@558 1162 the patch is. Long names shouldn't be a problem; you won't be
bos@558 1163 typing the names often, but you <emphasis>will</emphasis> be
bos@558 1164 running commands like <command
bos@558 1165 role="hg-ext-mq">hg qapplied</command> and <command
bos@558 1166 role="hg-ext-mq">hg qtop</command> over and over. Good naming
bos@558 1167 becomes especially important when you have a number of patches
bos@558 1168 to work with, or if you are juggling a number of different tasks
bos@558 1169 and your patches only get a fraction of your attention.</para>
bos@558 1170
bos@558 1171 <para>Be aware of what patch you're working on. Use the <command
bos@558 1172 role="hg-ext-mq">hg qtop</command> command and skim over the text
bos@558 1173 of your patches frequently&emdash;for example, using <command
bos@558 1174 role="hg-cmd">hg tip <option
bos@558 1175 role="hg-opt-tip">-p</option></command>)&emdash;to be sure of
bos@558 1176 where you stand. I have several times worked on and <command
bos@558 1177 role="hg-ext-mq">hg qrefresh</command>ed a patch other than the
bos@558 1178 one I intended, and it's often tricky to migrate changes into
bos@558 1179 the right patch after making them in the wrong one.</para>
bos@558 1180
bos@558 1181 <para>For this reason, it is very much worth investing a little
bos@558 1182 time to learn how to use some of the third-party tools I
bos@558 1183 described in section <xref linkend="sec:mq:tools"/>,
bos@558 1184 particularly <command>diffstat</command> and
bos@558 1185 <command>filterdiff</command>. The former will give you a quick
bos@558 1186 idea of what changes your patch is making, while the latter
bos@558 1187 makes it easy to splice hunks selectively out of one patch and
bos@558 1188 into another.</para>
bos@558 1189
bos@558 1190 </sect1>
bos@558 1191 <sect1>
bos@558 1192 <title>MQ cookbook</title>
bos@558 1193
bos@558 1194 <sect2>
bos@558 1195 <title>Manage <quote>trivial</quote> patches</title>
bos@558 1196
bos@558 1197 <para>Because the overhead of dropping files into a new
bos@558 1198 Mercurial repository is so low, it makes a lot of sense to
bos@558 1199 manage patches this way even if you simply want to make a few
bos@558 1200 changes to a source tarball that you downloaded.</para>
bos@558 1201
bos@558 1202 <para>Begin by downloading and unpacking the source tarball, and
bos@558 1203 turning it into a Mercurial repository. <!--
bos@558 1204 &interaction.mq.tarball.download; --></para>
bos@558 1205
bos@558 1206 <para>Continue by creating a patch stack and making your
bos@558 1207 changes. <!-- &interaction.mq.tarball.qinit; --></para>
bos@558 1208
bos@558 1209 <para>Let's say a few weeks or months pass, and your package
bos@558 1210 author releases a new version. First, bring their changes
bos@558 1211 into the repository. <!-- &interaction.mq.tarball.newsource;
bos@558 1212 --> The pipeline starting with <command role="hg-cmd">hg
bos@558 1213 locate</command> above deletes all files in the working
bos@558 1214 directory, so that <command role="hg-cmd">hg
bos@558 1215 commit</command>'s <option
bos@558 1216 role="hg-opt-commit">--addremove</option> option can
bos@558 1217 actually tell which files have really been removed in the
bos@558 1218 newer version of the source.</para>
bos@558 1219
bos@558 1220 <para>Finally, you can apply your patches on top of the new
bos@558 1221 tree. <!-- &interaction.mq.tarball.repush; --></para>
bos@558 1222
bos@558 1223 </sect2>
bos@558 1224 <sect2 id="sec:mq:combine">
bos@558 1225 <title>Combining entire patches</title>
bos@558 1226
bos@558 1227 <para>MQ provides a command, <command
bos@558 1228 role="hg-ext-mq">hg qfold</command> that lets you combine
bos@558 1229 entire patches. This <quote>folds</quote> the patches you
bos@558 1230 name, in the order you name them, into the topmost applied
bos@558 1231 patch, and concatenates their descriptions onto the end of its
bos@558 1232 description. The patches that you fold must be unapplied
bos@558 1233 before you fold them.</para>
bos@558 1234
bos@558 1235 <para>The order in which you fold patches matters. If your
bos@558 1236 topmost applied patch is <literal>foo</literal>, and you
bos@558 1237 <command role="hg-ext-mq">hg qfold</command>
bos@558 1238 <literal>bar</literal> and <literal>quux</literal> into it,
bos@558 1239 you will end up with a patch that has the same effect as if
bos@558 1240 you applied first <literal>foo</literal>, then
bos@558 1241 <literal>bar</literal>, followed by
bos@558 1242 <literal>quux</literal>.</para>
bos@558 1243
bos@558 1244 </sect2>
bos@558 1245 <sect2>
bos@558 1246 <title>Merging part of one patch into another</title>
bos@558 1247
bos@558 1248 <para>Merging <emphasis>part</emphasis> of one patch into
bos@558 1249 another is more difficult than combining entire
bos@558 1250 patches.</para>
bos@558 1251
bos@558 1252 <para>If you want to move changes to entire files, you can use
bos@558 1253 <command>filterdiff</command>'s <option
bos@558 1254 role="cmd-opt-filterdiff">-i</option> and <option
bos@558 1255 role="cmd-opt-filterdiff">-x</option> options to choose the
bos@558 1256 modifications to snip out of one patch, concatenating its
bos@558 1257 output onto the end of the patch you want to merge into. You
bos@558 1258 usually won't need to modify the patch you've merged the
bos@558 1259 changes from. Instead, MQ will report some rejected hunks
bos@558 1260 when you <command role="hg-ext-mq">hg qpush</command> it (from
bos@558 1261 the hunks you moved into the other patch), and you can simply
bos@558 1262 <command role="hg-ext-mq">hg qrefresh</command> the patch to drop
bos@558 1263 the duplicate hunks.</para>
bos@558 1264
bos@558 1265 <para>If you have a patch that has multiple hunks modifying a
bos@558 1266 file, and you only want to move a few of those hunks, the job
bos@558 1267 becomes more messy, but you can still partly automate it. Use
bos@558 1268 <command>lsdiff -nvv</command> to print some metadata about
bos@558 1269 the patch. <!-- &interaction.mq.tools.lsdiff; --></para>
bos@558 1270
bos@558 1271 <para>This command prints three different kinds of
bos@558 1272 number:</para>
bos@558 1273 <itemizedlist>
bos@558 1274 <listitem><para>(in the first column) a <emphasis>file
bos@558 1275 number</emphasis> to identify each file modified in the
bos@558 1276 patch;</para></listitem>
bos@558 1277
bos@558 1278 <listitem><para>(on the next line, indented) the line number
bos@558 1279 within a modified file where a hunk starts;
bos@558 1280 and</para></listitem>
bos@558 1281
bos@558 1282 <listitem><para>(on the same line) a <emphasis>hunk
bos@558 1283 number</emphasis> to identify that
bos@558 1284 hunk.</para></listitem>
bos@558 1285 </itemizedlist>
bos@558 1286
bos@558 1287 <para>You'll have to use some visual inspection, and reading of
bos@558 1288 the patch, to identify the file and hunk numbers you'll want,
bos@558 1289 but you can then pass them to to
bos@558 1290 <command>filterdiff</command>'s <option
bos@558 1291 role="cmd-opt-filterdiff">--files</option> and <option
bos@558 1292 role="cmd-opt-filterdiff">--hunks</option> options, to
bos@558 1293 select exactly the file and hunk you want to extract.</para>
bos@558 1294
bos@558 1295 <para>Once you have this hunk, you can concatenate it onto the
bos@558 1296 end of your destination patch and continue with the remainder
bos@558 1297 of section <xref linkend="sec:mq:combine"/>.</para>
bos@558 1298
bos@558 1299 </sect2>
bos@558 1300 </sect1>
bos@558 1301 <sect1>
bos@558 1302 <title>Differences between quilt and MQ</title>
bos@558 1303
bos@558 1304 <para>If you are already familiar with quilt, MQ provides a
bos@558 1305 similar command set. There are a few differences in the way
bos@558 1306 that it works.</para>
bos@558 1307
bos@558 1308 <para>You will already have noticed that most quilt commands have
bos@558 1309 MQ counterparts that simply begin with a
bos@558 1310 <quote><literal>q</literal></quote>. The exceptions are quilt's
bos@558 1311 <literal>add</literal> and <literal>remove</literal> commands,
bos@558 1312 the counterparts for which are the normal Mercurial <command
bos@558 1313 role="hg-cmd">hg add</command> and <command role="hg-cmd">hg
bos@558 1314 remove</command> commands. There is no MQ equivalent of the
bos@558 1315 quilt <literal>edit</literal> command.</para>
bos@558 1316
bos@558 1317 </sect1>
bos@558 1318 </chapter>
bos@558 1319
bos@558 1320 <!--
bos@558 1321 local variables:
bos@558 1322 sgml-parent-document: ("00book.xml" "book" "chapter")
bos@558 1323 end:
bos@558 1324 -->