hgbook
view en/mq.tex @ 25:9d5b6d303ef5
Make it 2/3 through documenting MQ commands.
Add a note on a few quirks of how patches represent data.
Add a note on a few quirks of how patches represent data.
| author | Bryan O'Sullivan <bos@serpentine.com> |
|---|---|
| date | Sun Jul 09 23:18:39 2006 -0700 (2006-07-09) |
| parents | 187702df428b |
| children | 1bc6c1f0192a |
line source
1 \chapter{Managing change with Mercurial Queues}
2 \label{chap:mq}
4 \section{The patch management problem}
5 \label{sec:mq:patch-mgmt}
7 Here is a common scenario: you need to install a software package from
8 source, but you find a bug that you must fix in the source before you
9 can start using the package. You make your changes, forget about the
10 package for a while, and a few months later you need to upgrade to a
11 newer version of the package. If the newer version of the package
12 still has the bug, you must extract your fix from the older source
13 tree and apply it against the newer version. This is a tedious task,
14 and it's easy to make mistakes.
16 This is a simple case of the ``patch management'' problem. You have
17 an ``upstream'' source tree that you can't change; you need to make
18 some local changes on top of the upstream tree; and you'd like to be
19 able to keep those changes separate, so that you can apply them to
20 newer versions of the upstream source.
22 The patch management problem arises in many situations. Probably the
23 most visible is that a user of an open source software project will
24 contribute a bug fix or new feature to the project's maintainers in the
25 form of a patch.
27 Distributors of operating systems that include open source software
28 often need to make changes to the packages they distribute so that
29 they will build properly in their environments.
31 When you have few changes to maintain, it is easy to manage a single
32 patch using the standard \texttt{diff} and \texttt{patch} programs
33 (see section~\ref{sec:mq:patch} for a discussion of these tools).
34 Once the number of changes grows, it starts to makes sense to maintain
35 patches as discrete ``chunks of work,'' so that for example a single
36 patch will contain only one bug fix (the patch might modify several
37 files, but it's doing ``only one thing''), and you may have a number
38 of such patches for different bugs you need fixed and local changes
39 you require. In this situation, if you submit a bug fix patch to the
40 upstream maintainers of a package and they include your fix in a
41 subsequent release, you can simply drop that single patch when you're
42 updating to the newer release.
44 Maintaining a single patch against an upstream tree is a little
45 tedious and error-prone, but not difficult. However, the complexity
46 of the problem grows rapidly as the number of patches you have to
47 maintain increases. With more than a tiny number of patches in hand,
48 understanding which ones you have applied and maintaining them moves
49 from messy to overwhelming.
51 Fortunately, Mercurial includes a powerful extension, Mercurial Queues
52 (or simply ``MQ''), that massively simplifies the patch management
53 problem.
55 \section{The prehistory of Mercurial Queues}
56 \label{sec:mq:history}
58 During the late 1990s, several Linux kernel developers started to
59 maintain ``patch series'' that modified the behaviour of the Linux
60 kernel. Some of these series were focused on stability, some on
61 feature coverage, and others were more speculative.
63 The sizes of these patch series grew rapidly. In 2002, Andrew Morton
64 published some shell scripts he had been using to automate the task of
65 managing his patch queues. Andrew was successfully using these
66 scripts to manage hundreds (sometimes thousands) of patches on top of
67 the Linux kernel.
69 \subsection{A patchwork quilt}
70 \label{sec:mq:quilt}
73 In early 2003, Andreas Gruenbacher and Martin Quinson borrowed the
74 approach of Andrew's scripts and published a tool called ``patchwork
75 quilt''~\cite{web:quilt}, or simply ``quilt''
76 (see~\cite{gruenbacher:2005} for a paper describing it). Because
77 quilt substantially automated patch management, it rapidly gained a
78 large following among open source software developers.
80 Quilt manages a \emph{stack of patches} on top of a directory tree.
81 To begin, you tell quilt to manage a directory tree; it stores away
82 the names and contents of all files in the tree. To fix a bug, you
83 create a new patch (using a single command), edit the files you need
84 to fix, then ``refresh'' the patch.
86 The refresh step causes quilt to scan the directory tree; it updates
87 the patch with all of the changes you have made. You can create
88 another patch on top of the first, which will track the changes
89 required to modify the tree from ``tree with one patch applied'' to
90 ``tree with two patches applied''.
92 You can \emph{change} which patches are applied to the tree. If you
93 ``pop'' a patch, the changes made by that patch will vanish from the
94 directory tree. Quilt remembers which patches you have popped,
95 though, so you can ``push'' a popped patch again, and the directory
96 tree will be restored to contain the modifications in the patch. Most
97 importantly, you can run the ``refresh'' command at any time, and the
98 topmost applied patch will be updated. This means that you can, at
99 any time, change both which patches are applied and what
100 modifications those patches make.
102 Quilt knows nothing about revision control tools, so it works equally
103 well on top of an unpacked tarball or a Subversion repository.
105 \subsection{From patchwork quilt to Mercurial Queues}
106 \label{sec:mq:quilt-mq}
108 In mid-2005, Chris Mason took the features of quilt and wrote an
109 extension that he called Mercurial Queues, which added quilt-like
110 behaviour to Mercurial.
112 The key difference between quilt and MQ is that quilt knows nothing
113 about revision control systems, while MQ is \emph{integrated} into
114 Mercurial. Each patch that you push is represented as a Mercurial
115 changeset. Pop a patch, and the changeset goes away.
117 This integration makes understanding patches and debugging their
118 effects \emph{enormously} easier. Since every applied patch has an
119 associated changeset, you can use \hgcmdargs{log}{\emph{filename}} to
120 see which changesets and patches affected a file. You can use the
121 \hgext{bisect} extension to binary-search through all changesets and
122 applied patches to see where a bug got introduced or fixed. You can
123 use the \hgcmd{annotate} command to see which changeset or patch
124 modified a particular line of a source file. And so on.
126 Because quilt does not care about revision control tools, it is still
127 a tremendously useful piece of software to know about for situations
128 where you cannot use Mercurial and MQ.
130 \section{Understanding patches}
132 Because MQ doesn't hide its patch-oriented nature, it is helpful to
133 understand what patches are, and a little about the tools that work
134 with them.
136 The traditional Unix \command{diff} command compares two files, and
137 prints a list of differences between them. The \command{patch} command
138 understands these differences as \emph{modifications} to make to a
139 file. Take a look at figure~\ref{ex:mq:diff} for a simple example of
140 these commands in action.
142 \begin{figure}[ht]
143 \interaction{mq.diff.diff}
144 \caption{Simple uses of the \command{diff} and \command{patch} commands}
145 \label{ex:mq:diff}
146 \end{figure}
148 The type of file that \command{diff} generates (and \command{patch}
149 takes as input) is called a ``patch'' or a ``diff''; there is no
150 difference between a patch and a diff. (We'll use the term ``patch'',
151 since it's more commonly used.)
153 A patch file can start with arbitrary text; the \command{patch}
154 command ignores this text, but MQ uses it as the commit message when
155 creating changesets. To find the beginning of the patch content,
156 \command{patch} searches for the first line that starts with the
157 string ``\texttt{diff~-}''.
159 MQ works with \emph{unified} diffs (\command{patch} can accept several
160 other diff formats, but MQ doesn't). A unified diff contains two
161 kinds of header. The \emph{file header} describes the file being
162 modified; it contains the name of the file to modify. When
163 \command{patch} sees a new file header, it looks for a file with that
164 name to start modifying.
166 After the file header comes a series of \emph{hunks}. Each hunk
167 starts with a header; this identifies the range of line numbers within
168 the file that the hunk should modify. Following the header, a hunk
169 starts and ends with a few (usually three) lines of text from the
170 unmodified file; these are called the \emph{context} for the hunk. If
171 there's only a small amount of context between successive hunks,
172 \command{diff} doesn't print a new hunk header; it just runs the hunks
173 together, with a few lines of context between modifications.
175 Each line of context begins with a space character. Within the hunk,
176 a line that begins with ``\texttt{-}'' means ``remove this line,''
177 while a line that begins with ``\texttt{+}'' means ``insert this
178 line.'' For example, a line that is modified is represented by one
179 deletion and one insertion.
181 We will return to ome of the more subtle aspects of patches later (in
182 section~\ref{ex:mq:adv-patch}), but you should have enough information
183 now to use MQ.
185 \section{Getting started with Mercurial Queues}
186 \label{sec:mq:start}
188 Because MQ is implemented as an extension, you must explicitly enable
189 before you can use it. (You don't need to download anything; MQ ships
190 with the standard Mercurial distribution.) To enable MQ, edit your
191 \tildefile{.hgrc} file, and add the lines in figure~\ref{ex:mq:config}.
193 \begin{figure}[ht]
194 \begin{codesample4}
195 [extensions]
196 hgext.mq =
197 \end{codesample4}
198 \label{ex:mq:config}
199 \caption{Contents to add to \tildefile{.hgrc} to enable the MQ extension}
200 \end{figure}
202 Once the extension is enabled, it will make a number of new commands
203 available. To verify that the extension is working, you can use
204 \hgcmd{help} to see if the \hgcmd{qinit} command is now available; see
205 the example in figure~\ref{ex:mq:enabled}.
207 \begin{figure}[ht]
208 \interaction{mq.qinit-help.help}
209 \caption{How to verify that MQ is enabled}
210 \label{ex:mq:enabled}
211 \end{figure}
213 You can use MQ with \emph{any} Mercurial repository, and its commands
214 only operate within that repository. To get started, simply prepare
215 the repository using the \hgcmd{qinit} command (see
216 figure~\ref{ex:mq:qinit}). This command creates an empty directory
217 called \sdirname{.hg/patches}, where MQ will keep its metadata. As
218 with many Mercurial commands, the \hgcmd{qinit} command prints nothing
219 if it succeeds.
221 \begin{figure}[ht]
222 \interaction{mq.tutorial.qinit}
223 \caption{Preparing a repository for use with MQ}
224 \label{ex:mq:qinit}
225 \end{figure}
227 \begin{figure}[ht]
228 \interaction{mq.tutorial.qnew}
229 \caption{Creating a new patch}
230 \label{ex:mq:qnew}
231 \end{figure}
233 \subsection{Creating a new patch}
235 To begin work on a new patch, use the \hgcmd{qnew} command. This
236 command takes one argument, the name of the patch to create. MQ will
237 use this as the name of an actual file in the \sdirname{.hg/patches}
238 directory, as you can see in figure~\ref{ex:mq:qnew}.
240 Also newly present in the \sdirname{.hg/patches} directory are two
241 other files, \sfilename{series} and \sfilename{status}. The
242 \sfilename{series} file lists all of the patches that MQ knows about
243 for this repository, with one patch per line. Mercurial uses the
244 \sfilename{status} file for internal book-keeping; it tracks all of the
245 patches that MQ has \emph{applied} in this repository.
247 \begin{note}
248 You may sometimes want to edit the \sfilename{series} file by hand;
249 for example, to change the sequence in which some patches are
250 applied. However, manually editing the \sfilename{status} file is
251 almost always a bad idea, as it's easy to corrupt MQ's idea of what
252 is happening.
253 \end{note}
255 Once you have created your new patch, you can edit files in the
256 working directory as you usually would. All of the normal Mercurial
257 commands, such as \hgcmd{diff} and \hgcmd{annotate}, work exactly as
258 they did before.
260 \subsection{Refreshing a patch}
262 When you reach a point where you want to save your work, use the
263 \hgcmd{qrefresh} command (figure~\ref{ex:mq:qnew}) to update the patch
264 you are working on. This command folds the changes you have made in
265 the working directory into your patch, and updates its corresponding
266 changeset to contain those changes.
268 \begin{figure}[ht]
269 \interaction{mq.tutorial.qrefresh}
270 \caption{Refreshing a patch}
271 \label{ex:mq:qrefresh}
272 \end{figure}
274 You can run \hgcmd{qrefresh} as often as you like, so it's a good way
275 to ``checkpoint'' your work. Refresh your patch at an opportune
276 time; try an experiment; and if the experiment doesn't work out,
277 \hgcmd{revert} your modifications back to the last time you refreshed.
279 \begin{figure}[ht]
280 \interaction{mq.tutorial.qrefresh2}
281 \caption{Refresh a patch many times to accumulate changes}
282 \label{ex:mq:qrefresh2}
283 \end{figure}
285 \subsection{Stacking and tracking patches}
287 Once you have finished working on a patch, or need to work on another,
288 you can use the \hgcmd{qnew} command again to create a new patch.
289 Mercurial will apply this patch on top of your existing patch. See
290 figure~\ref{ex:mq:qnew2} for an example. Notice that the patch
291 contains the changes in our prior patch as part of its context (you
292 can see this more clearly in the output of \hgcmd{annotate}).
294 \begin{figure}[ht]
295 \interaction{mq.tutorial.qnew2}
296 \caption{Stacking a second patch on top of the first}
297 \label{ex:mq:qnew2}
298 \end{figure}
300 So far, with the exception of \hgcmd{qnew} and \hgcmd{qrefresh}, we've
301 been careful to only use regular Mercurial commands. However, there
302 are more ``natural'' commands you can use when thinking about patches
303 with MQ, as illustrated in figure~\ref{ex:mq:qseries}:
305 \begin{itemize}
306 \item The \hgcmd{qseries} command lists every patch that MQ knows
307 about in this repository, from oldest to newest (most recently
308 \emph{created}).
309 \item The \hgcmd{qapplied} command lists every patch that MQ has
310 \emph{applied} in this repository, again from oldest to newest (most
311 recently applied).
312 \end{itemize}
314 \begin{figure}[ht]
315 \interaction{mq.tutorial.qseries}
316 \caption{Understanding the patch stack with \hgcmd{qseries} and
317 \hgcmd{qapplied}}
318 \label{ex:mq:qseries}
319 \end{figure}
321 \subsection{Manipulating the patch stack}
323 The previous discussion implied that there must be a difference
324 between ``known'' and ``applied'' patches, and there is. MQ can
325 manage a patch without it being applied in the repository.
327 An \emph{applied} patch has a corresponding changeset in the
328 repository, and the effects of the patch and changeset are visible in
329 the working directory. You can undo the application of a patch using
330 the \hgcmd{qpop} command. MQ still \emph{knows about}, or manages, a
331 popped patch, but the patch no longer has a corresponding changeset in
332 the repository, and the working directory does not contain the changes
333 made by the patch. Figure~\ref{fig:mq:stack} illustrates the
334 difference between applied and tracked patches.
336 \begin{figure}[ht]
337 \centering
338 \grafix{mq-stack}
339 \caption{Applied and unapplied patches in the MQ patch stack}
340 \label{fig:mq:stack}
341 \end{figure}
343 You can reapply an unapplied, or popped, patch using the \hgcmd{qpush}
344 command. This creates a new changeset to correspond to the patch, and
345 the patch's changes once again become present in the working
346 directory. See figure~\ref{ex:mq:qpop} for examples of \hgcmd{qpop}
347 and \hgcmd{qpush} in action. Notice that once we have popped a patch
348 or two patches, the output of \hgcmd{qseries} remains the same, while
349 that of \hgcmd{qapplied} has changed.
351 \begin{figure}[ht]
352 \interaction{mq.tutorial.qpop}
353 \caption{Modifying the stack of applied patches}
354 \label{ex:mq:qpop}
355 \end{figure}
357 MQ does not limit you to pushing or popping one patch. You can have
358 no patches, all of them, or any number in between applied at some
359 point in time.
361 \subsection{Working on several patches at once}
363 The \hgcmd{qrefresh} command always refreshes the \emph{topmost}
364 applied patch. This means that you can suspend work on one patch (by
365 refreshing it), pop or push to make a different patch the top, and
366 work on \emph{that} patch for a while.
368 Here's an example that illustrates how you can use this ability.
369 Let's say you're developing a new feature as two patches. The first
370 is a change to the core of your software, and the second---layered on
371 top of the first---changes the user interface to use the code you just
372 added to the core. If you notice a bug in the core while you're
373 working on the UI patch, it's easy to fix the core. Simply
374 \hgcmd{qrefresh} the UI patch to save your in-progress changes, and
375 \hgcmd{qpop} down to the core patch. Fix the core bug,
376 \hgcmd{qrefresh} the core patch, and \hgcmd{qpush} back to the UI
377 patch to continue where you left off.
379 \section{More about patches}
380 \label{sec:mq:adv-patch}
382 MQ uses the GNU \command{patch} command to apply patches, so it's
383 helpful to know about a few more detailed aspects of how
384 \command{patch} works.
386 When \command{patch} applies a hunk, it tries a handful of
387 successively less accurate strategies to try to make the hunk apply.
388 This falling-back technique often makes it possible to take a patch
389 that was generated against an old version of a file, and apply it
390 against a newer version of that file.
392 First, \command{patch} tries an exact match, where the line numbers,
393 the context, and the text to be modified must apply exactly. If it
394 cannot make an exact match, it tries to find an exact match for the
395 context, without honouring the line numbering information. If this
396 succeeds, it prints a line of output saying that the hunk was applied,
397 but at some \emph{offset} from the original line number.
399 If a context-only match fails, \command{patch} removes the first and
400 last lines of the context, and tries a \emph{reduced} context-only
401 match. If the hunk with reduced context succeeds, it prints a message
402 saying that it applied the hunk with a \emph{fuzz factor} (the number
403 after the fuzz factor indicates how many lines of context
404 \command{patch} had to trim before the patch applied).
406 When neither of these techniques works, \command{patch} prints a
407 message saying that the hunk in question was rejected. It saves
408 rejected hunks (also simply called ``rejects'') to a file with the
409 same name, and an added \sfilename{.rej} extension. It also saves an
410 unmodified copy of the file with a \sfilename{.orig} extension; the
411 copy of the file without any extensions will contain any changes made
412 by hunks that \emph{did} apply cleanly. If you have a patch that
413 modifies \filename{foo} with six hunks, and one of them fails to
414 apply, you will have: an unmodified \filename{foo.orig}, a
415 \filename{foo.rej} containing one hunk, and \filename{foo}, containing
416 the changes made by the five successful five hunks.
418 \subsection{Some quirks of patch representation}
420 There are a few useful things to know about how \command{patch} works
421 with files.
422 \begin{itemize}
423 \item This should already be obvious, but \command{patch} cannot
424 handle binary files.
425 \item Neither does it care about the executable bit; it creates new
426 files as readable, but not executable.
427 \item \command{patch} treats the removal of a file as a diff between
428 the file to be removed and the empty file. So your idea of ``I
429 deleted this file'' looks like ``every line of this file was
430 deleted'' in a patch.
431 \item It treats the addition of a file as a diff between the empty
432 file and the file to be added. So in a patch, your idea of ``I
433 added this file'' looks like ``every line of this file was added''.
434 \item It treats a renamed file as the removal of the old name, and the
435 addition of the new name. This means that renamed files have a big
436 footprint in patches. (Note also that Mercurial does not currently
437 try to infer when files have been renamed or copied in a patch.)
438 \item \command{patch} cannot represent empty files, so you cannot use
439 a patch to represent the notion ``I added this empty file to the
440 tree''.
441 \end{itemize}
442 \subsection{Beware the fuzz}
444 While applying a hunk at an offset, or with a fuzz factor, will often
445 be completely successful, these inexact techniques naturally leave
446 open the possibility of corrupting the patched file. The most common
447 cases typically involve applying a patch twice, or at an incorrect
448 location in the file. If \command{patch} or \hgcmd{qpush} ever
449 mentions an offset or fuzz factor, you should make sure that the
450 modified files are correct afterwards.
452 It's often a good idea to refresh a patch that has applied with an
453 offset or fuzz factor; refreshing the patch generates new context
454 information that will make it apply cleanly. I say ``often,'' not
455 ``always,'' because sometimes refreshing a patch will make it fail to
456 apply against a different revision of the underlying files. In some
457 cases, such as when you're maintaining a patch that must sit on top of
458 multiple versions of a source tree, it's acceptable to have a patch
459 apply with some fuzz, provided you've verified the results of the
460 patching process in such cases.
462 \subsection{Handling rejection}
464 If \hgcmd{qpush} fails to apply a patch, it will print an error
465 message and exit. If it has left \sfilename{.rej} files behind, it is
466 usually best to fix up the rejected hunks before you push more patches
467 or do any further work.
469 If your patch \emph{used to} apply cleanly, and no longer does because
470 you've changed the underlying code that your patches are based on,
471 Mercurial Queues can help; see section~\ref{sec:mq:merge} for details.
473 Unfortunately, there aren't any great techniques for dealing with
474 rejected hunks. Most often, you'll need to view the \sfilename{.rej}
475 file and edit the target file, applying the rejected hunks by hand.
477 If you're feeling adventurous, Neil Brown, a Linux kernel hacker,
478 wrote a tool called \command{wiggle}~\cite{web:wiggle}, which is more
479 vigorous than \command{patch} in its attempts to make a patch apply.
481 Another Linux kernel hacker, Chris Mason (the author of Mercurial
482 Queues), wrote a similar tool called \command{rej}~\cite{web:rej},
483 which takes a simple approach to automating the application of hunks
484 rejected by \command{patch}. \command{rej} can help with four common
485 reasons that a hunk may be rejected:
487 \begin{itemize}
488 \item The context in the middle of a hunk has changed.
489 \item A hunk is missing some context at the beginning or end.
490 \item A large hunk might apply better---either entirely or in
491 part---if it was broken up into smaller hunks.
492 \item A hunk removes lines with slightly different content than those
493 currently present in the file.
494 \end{itemize}
496 If you use \command{wiggle} or \command{rej}, you should be doubly
497 careful to check your results when you're done.
499 \section{Getting the best performance out of MQ}
501 MQ is very efficient at handling a large number of patches. I ran
502 some performance experiments in mid-2006 for a talk that I gave at the
503 2006 EuroPython conference~\cite{web:europython}. I used as my data
504 set the Linux 2.6.17-mm1 patch series, which consists of 1,738
505 patches. I applied thes on top of a Linux kernel repository
506 containing all 27,472 revisions between Linux 2.6.12-rc2 and Linux
507 2.6.17.
509 On my old, slow laptop, I was able to
510 \hgcmdargs{qpush}{\hgopt{qpush}{-a}} all 1,738 patches in 3.5 minutes,
511 and \hgcmdargs{qpop}{\hgopt{qpop}{-a}} them all in 30 seconds. I
512 could \hgcmd{qrefresh} one of the biggest patches (which made 22,779
513 lines of changes to 287 files) in 6.6 seconds.
515 Clearly, MQ is well suited to working in large trees, but there are a
516 few tricks you can use to get the best performance of it.
518 First of all, try to ``batch'' operations together. Every time you
519 run \hgcmd{qpush} or \hgcmd{qpop}, these commands scan the working
520 directory once to make sure you haven't made some changes and then
521 forgotten to run \hgcmd{qrefresh}. On a small tree, the time that
522 this scan takes is unnoticeable. However, on a medium-sized tree
523 (containing tens of thousands of files), it can take a second or more.
525 The \hgcmd{qpush} and \hgcmd{qpop} commands allow you to push and pop
526 multiple patches at a time. You can identify the ``destination
527 patch'' that you want to end up at. When you \hgcmd{qpush} with a
528 destination specified, it will push patches until that patch is at the
529 top of the applied stack. When you \hgcmd{qpop} to a destination, MQ
530 will pop patches until the destination patch \emph{is no longer}
531 applied.
533 You can identify a destination patch using either the name of the
534 patch, or by number. If you use numeric addressing, patches are
535 counted from zero; this means that the first patch is zero, the second
536 is one, and so on.
538 \section{Updating your patches when the underlying code changes}
539 \label{sec:mq:merge}
541 It's common to have a stack of patches on top of an underlying
542 repository that you don't modify directly. If you're working on
543 changes to third-party code, or on a feature that is taking longer to
544 develop than the rate of change of the code beneath, you will often
545 need to sync up with the underlying code, and fix up any hunks in your
546 patches that no longer apply. This is called \emph{rebasing} your
547 patch series.
549 The simplest way to do this is to \hgcmdargs{qpop}{\hgopt{qpop}{-a}}
550 your patches, then \hgcmd{pull} changes into the underlying
551 repository, and finally \hgcmdargs{qpush}{\hgopt{qpop}{-a}} your
552 patches again. MQ will stop pushing any time it runs across a patch
553 that fails to apply during conflicts, allowing you to fix your
554 conflicts, \hgcmd{qrefresh} the affected patch, and continue pushing
555 until you have fixed your entire stack.
557 This approach is easy to use and works well if you don't expect
558 changes to the underlying code to affect how well your patches apply.
559 If your patch stack touches code that is modified frequently or
560 invasively in the underlying repository, however, fixing up rejected
561 hunks by hand quickly becomes tiresome.
563 It's possible to partially automate the rebasing process. If your
564 patches apply cleanly against some revision of the underlying repo, MQ
565 can use this information to help you to resolve conflicts between your
566 patches and a different revision.
568 The process is a little involved.
569 \begin{enumerate}
570 \item To begin, \hgcmdargs{qpush}{-a} all of your patches on top of
571 the revision where you know that they apply cleanly.
572 \item Save a backup copy of your patch directory using
573 \hgcmdargs{qsave}{\hgopt{qsave}{-e} \hgopt{qsave}{-c}}. This prints
574 the name of the directory that it has saved the patches in. It will
575 save the patches to a directory called
576 \sdirname{.hg/patches.\emph{N}}, where \texttt{\emph{N}} is a small
577 integer. It also commits a ``save changeset'' on top of your
578 applied patches; this is for internal book-keeping, and records the
579 states of the \sfilename{series} and \sfilename{status} files.
580 \item Use \hgcmd{pull} to bring new changes into the underlying
581 repository. (Don't run \hgcmdargs{pull}{-u}; see below for why.)
582 \item Update to the new tip revision, using
583 \hgcmdargs{update}{\hgopt{update}{-C}} to override the patches you
584 have pushed.
585 \item Merge all patches using \hgcmdargs{qpush}{\hgopt{qpush}{-m}
586 \hgopt{qpush}{-a}}. The \hgopt{qpush}{-m} option to \hgcmd{qpush}
587 tells MQ to perform a three-way merge if the patch fails to apply.
588 \end{enumerate}
590 During the \hgcmdargs{qpush}{\hgopt{qpush}{-m}}, each patch in the
591 \sfilename{series} file is applied normally. If a patch applies with
592 fuzz or rejects, MQ looks at the queue you \hgcmd{qsave}d, and
593 performs a three-way merge with the corresponding changeset. This
594 merge uses Mercurial's normal merge machinery, so it may pop up a GUI
595 merge tool to help you to resolve problems.
597 When you finish resolving the effects of a patch, MQ refreshes your
598 patch based on the result of the merge.
600 At the end of this process, your repository will have one extra head
601 from the old patch queue, and a copy of the old patch queue will be in
602 \sdirname{.hg/patches.\emph{N}}. You can remove the extra head using
603 \hgcmdargs{qpop}{\hgopt{qpop}{-a} \hgopt{qpop}{-n} patches.\emph{N}}
604 or \hgcmd{strip}. You can delete \sdirname{.hg/patches.\emph{N}} once
605 you are sure that you no longer need it as a backup.
607 \section{Managing patches in a repository}
609 Because MQ's \sdirname{.hg/patches} directory resides outside a
610 Mercurial repository's working directory, the ``underlying'' Mercurial
611 repository knows nothing about the management or presence of patches.
613 This presents the interesting possibility of managing the contents of
614 the patch directory as a Mercurial repository in its own right. This
615 can be a useful way to work. For example, you can work on a patch for
616 a while, \hgcmd{qrefresh} it, then \hgcmd{commit} the current state of
617 the patch. This lets you ``roll back'' to that version of the patch
618 later on.
620 In addition, you can then share different versions of the same patch
621 stack among multiple underlying repositories. I use this when I am
622 developing a Linux kernel feature. I have a pristine copy of my
623 kernel sources for each of several CPU architectures, and a cloned
624 repository under each that contains the patches I am working on. When
625 I want to test a change on a different architecture, I push my current
626 patches to the patch repository associated with that kernel tree, pop
627 and push all of my patches, and build and test that kernel.
629 Managing patches in a repository makes it possible for multiple
630 developers to work on the same patch series without colliding with
631 each other, all on top of an underlying source base that they may or
632 may not control.
634 \subsection{MQ support for patch repositories}
636 MQ helps you to work with the \sdirname{.hg/patches} directory as a
637 repository; when you prepare a repository for working with patches
638 using \hgcmd{qinit}, you can pass the \hgopt{qinit}{-c} option to
639 create the \sdirname{.hg/patches} directory as a Mercurial repository.
641 \begin{note}
642 If you forget to use the \hgopt{qinit}{-c} option, you can simply go
643 into the \sdirname{.hg/patches} directory at any time and run
644 \hgcmd{init}. Don't forget to add an entry for the
645 \sfilename{status} file to the \sfilename{.hgignore} file, though
646 (\hgcmdargs{qinit}{\hgopt{qinit}{-c}} does this for you
647 automatically); you \emph{really} don't want to manage the
648 \sfilename{status} file.
649 \end{note}
651 As a convenience, if MQ notices that the \dirname{.hg/patches}
652 directory is a repository, it will automatically \hgcmd{add} every
653 patch that you create and import.
655 Finally, MQ provides a shortcut command, \hgcmd{qcommit}, that runs
656 \hgcmd{commit} in the \sdirname{.hg/patches} directory. This saves
657 some cumbersome typing.
659 \subsection{A few things to watch out for}
661 MQ's support for working with a repository full of patches is limited
662 in a few small respects.
664 MQ cannot automatically detect changes that you make to the patch
665 directory. If you \hgcmd{pull}, manually edit, or \hgcmd{update}
666 changes to patches or the \sfilename{series} file, you will have to
667 \hgcmdargs{qpop}{\hgopt{qpop}{-a}} and then
668 \hgcmdargs{qpush}{\hgopt{qpush}{-a}} in the underlying repository to
669 see those changes show up there. If you forget to do this, you can
670 confuse MQ's idea of which patches are applied.
672 \section{Commands for working with patches}
673 \label{sec:mq:tools}
675 Once you've been working with patches for a while, you'll find
676 yourself hungry for tools that will help you to understand and
677 manipulate the patches you're dealing with.
679 The \command{diffstat} command~\cite{web:diffstat} generates a
680 histogram of the modifications made to each file in a patch. It
681 provides a good way to ``get a sense of'' a patch---which files it
682 affects, and how much change it introduces to each file and as a
683 whole. (I find that it's a good idea to use \command{diffstat}'s
684 \texttt{-p} option as a matter of course, as otherwise it will try to
685 do clever things with prefixes of file names that inevitably confuse
686 at least me.)
688 \begin{figure}[ht]
689 \interaction{mq.tools.tools}
690 \caption{The \command{diffstat}, \command{filterdiff}, and \command{lsdiff} commands}
691 \label{ex:mq:tools}
692 \end{figure}
694 The \package{patchutils} package~\cite{web:patchutils} is invaluable.
695 It provides a set of small utilities that follow the ``Unix
696 philosophy;'' each does one useful thing with a patch. The
697 \package{patchutils} command I use most is \command{filterdiff}, which
698 extracts subsets from a patch file. For example, given a patch that
699 modifies hundreds of files across dozens of directories, a single
700 invocation of \command{filterdiff} can generate a smaller patch that
701 only touches files whose names match a particular glob pattern.
703 \section{Good ways to work with patches}
705 Whether you are working on a patch series to submit to a free software
706 or open source project, or a series that you intend to treat as a
707 sequence of regular changesets when you're done, you can use some
708 simple techniques to keep your work well organised.
710 Give your patches descriptive names. A good name for a patch might be
711 \filename{rework-device-alloc.patch}, because it will immediately give
712 you a hint what the purpose of the patch is. Long names shouldn't be
713 a problem; you won't be typing the names often, but you \emph{will} be
714 running commands like \hgcmd{qapplied} and \hgcmd{qtop} over and over.
715 Good naming becomes especially important when you have a number of
716 patches to work with, or if you are juggling a number of different
717 tasks and your patches only get a fraction of your attention.
719 Be aware of what patch you're working on. Use the \hgcmd{qtop}
720 command and skim over the text of your patches frequently---for
721 example, using \hgcmdargs{tip}{\hgopt{tip}{-p}})---to be sure of where
722 you stand. I have several times worked on and \hgcmd{qrefresh}ed a
723 patch other than the one I intended, and it's often tricky to migrate
724 changes into the right patch after making them in the wrong one.
726 For this reason, it is very much worth investing a little time to
727 learn how to use some of the third-party tools I described in
728 section~\ref{sec:mq:tools}, particularly \command{diffstat} and
729 \command{filterdiff}. The former will give you a quick idea of what
730 changes your patch is making, while the latter makes it easy to splice
731 hunks selectively out of one patch and into another.
733 \section{MQ cookbook}
735 \subsection{Manage ``trivial'' patches}
737 Because the overhead of dropping files into a new Mercurial repository
738 is so low, it makes a lot of sense to manage patches this way even if
739 you simply want to make a few changes to a source tarball that you
740 downloaded.
742 Begin by downloading and unpacking the source tarball,
743 and turning it into a Mercurial repository.
744 \interaction{mq.tarball.download}
746 Continue by creating a patch stack and making your changes.
747 \interaction{mq.tarball.qinit}
749 Let's say a few weeks or months pass, and your package author releases
750 a new version. First, bring their changes into the repository.
751 \interaction{mq.tarball.newsource}
752 The pipeline starting with \hgcmd{locate} above deletes all files in
753 the working directory, so that \hgcmd{commit}'s
754 \hgopt{commit}{--addremove} option can actually tell which files have
755 really been removed in the newer version of the source.
757 Finally, you can apply your patches on top of the new tree.
758 \interaction{mq.tarball.repush}
760 \subsection{Combining entire patches}
761 \label{sec:mq:combine}
763 It's easy to combine entire patches.
765 \begin{enumerate}
766 \item \hgcmd{qpop} your applied patches until neither patch is
767 applied.
768 \item Concatenate the patches that you want to combine together:
769 \begin{codesample4}
770 cat patch-to-drop.patch >> patch-to-augment.patch
771 \end{codesample4}
772 The description from the first patch (if you have one) will be used
773 as the commit comment when you \hgcmd{qpush} the combined patch.
774 Edit the patch description if you need to.
775 \item Use the \hgcmd{qdel} command to delete the patch you're dropping
776 from the \sfilename{series} file.
777 \item \hgcmd{qpush} the combined patch. Fix up any rejects.
778 \item \hgcmd{qrefresh} the combined patch to tidy it up.
779 \end{enumerate}
781 \subsection{Merging part of one patch into another}
783 Merging \emph{part} of one patch into another is more difficult than
784 combining entire patches.
786 If you want to move changes to entire files, you can use
787 \command{filterdiff}'s \cmdopt{filterdiff}{-i} and
788 \cmdopt{filterdiff}{-x} options to choose the modifications to snip
789 out of one patch, concatenating its output onto the end of the patch
790 you want to merge into. You usually won't need to modify the patch
791 you've merged the changes from. Instead, MQ will report some rejected
792 hunks when you \hgcmd{qpush} it (from the hunks you moved into the
793 other patch), and you can simply \hgcmd{qrefresh} the patch to drop
794 the duplicate hunks.
796 If you have a patch that has multiple hunks modifying a file, and you
797 only want to move a few of those hunks, the job becomes more messy,
798 but you can still partly automate it. Use \cmdargs{lsdiff}{-nvv} to
799 print some metadata about the patch.
800 \interaction{mq.tools.lsdiff}
802 This command prints three different kinds of number:
803 \begin{itemize}
804 \item a \emph{file number} to identify each file modified in the patch;
805 \item the line number within a modified file that a hunk starts at; and
806 \item a \emph{hunk number} to identify that hunk.
807 \end{itemize}
809 You'll have to use some visual inspection, and reading of the patch,
810 to identify the file and hunk numbers you'll want, but you can then
811 pass them to to \command{filterdiff}'s \cmdopt{filterdiff}{--files}
812 and \cmdopt{filterdiff}{--hunks} options, to select exactly the file
813 and hunk you want to extract.
815 Once you have this hunk, you can concatenate it onto the end of your
816 destination patch and continue with the remainder of
817 section~\ref{sec:mq:combine}.
818 \section{MQ command reference}
819 \label{sec:mq:cmdref}
821 For an overview of the commands provided by MQ, use the command
822 \hgcmdargs{help}{mq}.
824 \subsection{\hgcmd{qapplied}---print applied patches}
826 The \hgcmd{qapplied} command prints the current stack of applied
827 patches. Patches are printed in oldest-to-newest order, so the last
828 patch in the list is the ``top'' patch.
830 \subsection{\hgcmd{qcommit}---commit changes in the queue repository}
832 The \hgcmd{qcommit} command commits any outstanding changes in the
833 \sdirname{.hg/patches} repository. This command only works if the
834 \sdirname{.hg/patches} directory is a repository, i.e.~you created the
835 directory using \hgcmdargs{qinit}{\hgopt{qinit}{-c}} or ran
836 \hgcmd{init} in the directory after running \hgcmd{qinit}.
838 This command is shorthand for \hgcmdargs{commit}{--cwd .hg/patches}.
840 \subsection{\hgcmd{qdelete}---delete a patch from the
841 \sfilename{series} file}
843 The \hgcmd{qdelete} command removes the entry for a patch from the
844 \sfilename{series} file in the \sdirname{.hg/patches} directory. It
845 does not delete the patch file, nor does it pop the patch if the patch
846 is already applied.
848 \subsection{\hgcmd{qdiff}---print a diff of the topmost applied patch}
850 The \hgcmd{qdiff} command prints a diff of the topmost applied patch.
851 It is equivalent to \hgcmdargs{diff}{-r-2:-1}.
853 \subsection{\hgcmd{qimport}---import a third-party patch into the queue}
855 The \hgcmd{qimport} command adds an entry for an external patch to the
856 \sfilename{series} file, and copies the patch into the
857 \sdirname{.hg/patches} directory. It adds the entry immediately after
858 the topmost applied patch, but does not push the patch.
860 If the \sdirname{.hg/patches} directory is a repository,
861 \hgcmd{qimport} automatically does an \hgcmd{add} of the imported
862 patch.
864 \subsection{\hgcmd{qinit}---prepare a repository to work with MQ}
866 The \hgcmd{qinit} command prepares a repository to work with MQ. It
867 creates a directory called \sdirname{.hg/patches}.
869 Options:
870 \begin{itemize}
871 \item[\hgopt{qinit}{-c}] Create \sdirname{.hg/patches} as a repository
872 in its own right. Also creates a \sfilename{.hgignore} file that
873 will ignore the \sfilename{status} file.
874 \end{itemize}
876 When the \sdirname{.hg/patches} directory is a repository, the
877 \hgcmd{qimport} and \hgcmd{qnew} commands automatically \hgcmd{add}
878 new patches.
880 \subsection{\hgcmd{qnew}---create a new patch}
882 The \hgcmd{qnew} command creates a new patch. It takes one mandatory
883 argument, the name to use for the patch file. The newly created patch
884 is created empty by default. It is added to the \sfilename{series}
885 file after the current topmost applied patch, and is immediately
886 pushed on top of that patch.
888 If \hgcmd{qnew} finds modified files in the working directory, it will
889 refuse to create a new patch unless the \hgopt{qnew}{-f} option is
890 used (see below). This behaviour allows you to \hgcmd{qrefresh} your
891 topmost applied patch before you apply a new patch on top of it.
893 Options:
894 \begin{itemize}
895 \item[\hgopt{qnew}{-f}] Create a new patch if the contents of the
896 working directory are modified. Any outstanding modifications are
897 added to the newly created patch, so after this command completes,
898 the working directory will no longer be modified.
899 \item[\hgopt{qnew}{-m}] Use the given text as the commit message.
900 This text will be stored at the beginning of the patch file, before
901 the patch data.
902 \end{itemize}
904 \subsection{\hgcmd{qnext}---print the name of the next patch}
906 The \hgcmd{qnext} command prints the name name of the next patch in
907 the \sfilename{series} file after the topmost applied patch. This
908 patch will become the topmost applied patch if you run \hgcmd{qpush}.
910 \subsection{\hgcmd{qpop}---pop patches off the stack}
912 The \hgcmd{qpop} command removes applied patches from the top of the
913 stack of applied patches. By default, it removes only one patch.
915 This command removes the changesets that represent the popped patches
916 from the repository, and updates the working directory to undo the
917 effects of the patches.
919 This command takes an optional argument, which it uses as the name or
920 index of the patch to pop to. If given a name, it will pop patches
921 until the named patch is no longer applied. If given a number,
922 \hgcmd{qpop} treats the number as an index into the entries in the
923 series file, counting from zero (empty lines and lines containing only
924 comments do not count). It pops patches until the patch identified by
925 the given index is no longer applied.
927 The \hgcmd{qpop} command does not read or write patches or the
928 \sfilename{series} file. It is thus safe to \hgcmd{qpop} a patch that
929 you have removed from the \sfilename{series} file, or a patch that you
930 have renamed or deleted entirely. In the latter two cases, use the
931 name of the patch as it was when you applied it.
933 By default, the \hgcmd{qpop} command will not pop any patches if the
934 working directory has been modified. You can override this behaviour
935 using the \hgopt{qpop}{-f} option, which reverts all modifications in
936 the working directory.
938 Options:
939 \begin{itemize}
940 \item[\hgopt{qpop}{-a}] Pop all applied patches. This returns the
941 repository to its state before you applied any patches.
942 \item[\hgopt{qpop}{-f}] Forcibly revert any modifications to the
943 working directory when popping.
944 \item[\hgopt{qpop}{-n}] Pop a patch from the named queue.
945 \end{itemize}
947 The \hgcmd{qpop} command removes one line from the end of the
948 \sfilename{status} file for each patch that it pops.
949 \subsection{\hgcmd{qprev}---print the name of the previous patch}
951 The \hgcmd{qprev} command prints the name of the patch in the
952 \sfilename{series} file that comes before the topmost applied patch.
953 This will become the topmost applied patch if you run \hgcmd{qpop}.
955 \subsection{\hgcmd{qpush}---push patches onto the stack}
957 The \hgcmd{qpush} command adds patches onto the applied stack. By
958 default, it adds only one patch.
960 This command creates a new changeset to represent each applied patch,
961 and updates the working directory to apply the effects of the patches.
963 The data used when creating a changeset are as follows:
964 \begin{itemize}
965 \item The commit date and time zone are the current date and time
966 zone. Because these data are used to compute the identity of a
967 changeset, this means that if you \hgcmd{qpop} a patch and
968 \hgcmd{qpush} it again, the changeset that you push will have a
969 different identity than the changeset you popped.
970 \item The author is the same as the default used by the \hgcmd{commit}
971 command.
972 \item The commit message is any text from the patch file that comes
973 before the first diff header. If there is no such text, a default
974 commit message is used that identifies the name of the patch.
975 \end{itemize}
977 Options:
978 \begin{itemize}
979 \item[\hgopt{qpush}{-a}] Push all unapplied patches from the
980 \sfilename{series} file until there are none left to push.
981 \item[\hgopt{qpush}{-l}] Add the name of the patch to the end
982 of the commit message.
983 \item[\hgopt{qpush}{-m}] If a patch fails to apply cleanly, use the
984 entry for the patch in another saved queue to compute the parameters
985 for a three-way merge, and perform a three-way merge using the
986 normal Mercurial merge machinery. Use the resolution of the merge
987 as the new patch content.
988 \item[\hgopt{qpush}{-n}] Use the named queue if merging while pushing.
989 \end{itemize}
991 The \hgcmd{qpush} command reads, but does not modify, the
992 \sfilename{series} file. It appends one line to the \hgcmd{status}
993 file for each patch that it pushes.
995 \subsection{\hgcmd{qrefresh}---update the topmost applied patch}
997 The \hgcmd{qrefresh} command updates the topmost applied patch. It
998 modifies the patch, removes the old changeset that represented the
999 patch, and creates a new changeset to represent the modified patch.
1001 The \hgcmd{qrefresh} command looks for the following modifications:
1002 \begin{itemize}
1003 \item Changes to the commit message, i.e.~the text before the first
1004 diff header in the patch file, are reflected in the new changeset
1005 that represents the patch.
1006 \item Modifications to tracked files in the working directory are
1007 added to the patch.
1008 \item Changes to the files tracked using \hgcmd{add}, \hgcmd{copy},
1009 \hgcmd{remove}, or \hgcmd{rename}. Added files and copy and rename
1010 destinations are added to the patch, while removed files and rename
1011 sources are removed.
1012 \end{itemize}
1014 Even if \hgcmd{qrefresh} detects no changes, it still recreates the
1015 changeset that represents the patch. This causes the identity of the
1016 changeset to differ from the previous changeset that identified the
1017 patch.
1019 \section{MQ file reference}
1022 \subsection{The \sfilename{series} file}
1025 \subsection{The \sfilename{status} file}
1029 %%% Local Variables:
1030 %%% mode: latex
1031 %%% TeX-master: "00book"
1032 %%% End: