bos@1: \chapter{Managing change with Mercurial Queues} bos@1: \label{chap:mq} bos@1: bos@1: \section{The patch management problem} bos@1: \label{sec:mq:patch-mgmt} bos@1: bos@1: Here is a common scenario: you need to install a software package from bos@1: source, but you find a bug that you must fix in the source before you bos@1: can start using the package. You make your changes, forget about the bos@1: package for a while, and a few months later you need to upgrade to a bos@1: newer version of the package. If the newer version of the package bos@1: still has the bug, you must extract your fix from the older source bos@1: tree and apply it against the newer version. This is a tedious task, bos@1: and it's easy to make mistakes. bos@1: bos@1: This is a simple case of the ``patch management'' problem. You have bos@1: an ``upstream'' source tree that you can't change; you need to make bos@1: some local changes on top of the upstream tree; and you'd like to be bos@1: able to keep those changes separate, so that you can apply them to bos@1: newer versions of the upstream source. bos@1: bos@1: The patch management problem arises in many situations. Probably the bos@1: most visible is that a user of an open source software project will bos@3: contribute a bug fix or new feature to the project's maintainers in the bos@1: form of a patch. bos@1: bos@1: Distributors of operating systems that include open source software bos@1: often need to make changes to the packages they distribute so that bos@1: they will build properly in their environments. bos@1: bos@1: When you have few changes to maintain, it is easy to manage a single bos@15: patch using the standard \texttt{diff} and \texttt{patch} programs bos@15: (see section~\ref{sec:mq:patch} for a discussion of these tools). bos@1: Once the number of changes grows, it starts to makes sense to maintain bos@1: patches as discrete ``chunks of work,'' so that for example a single bos@1: patch will contain only one bug fix (the patch might modify several bos@1: files, but it's doing ``only one thing''), and you may have a number bos@1: of such patches for different bugs you need fixed and local changes bos@3: you require. In this situation, if you submit a bug fix patch to the bos@1: upstream maintainers of a package and they include your fix in a bos@1: subsequent release, you can simply drop that single patch when you're bos@1: updating to the newer release. bos@1: bos@1: Maintaining a single patch against an upstream tree is a little bos@1: tedious and error-prone, but not difficult. However, the complexity bos@1: of the problem grows rapidly as the number of patches you have to bos@1: maintain increases. With more than a tiny number of patches in hand, bos@1: understanding which ones you have applied and maintaining them moves bos@1: from messy to overwhelming. bos@1: bos@1: Fortunately, Mercurial includes a powerful extension, Mercurial Queues bos@1: (or simply ``MQ''), that massively simplifies the patch management bos@1: problem. bos@1: bos@1: \section{The prehistory of Mercurial Queues} bos@1: \label{sec:mq:history} bos@1: bos@1: During the late 1990s, several Linux kernel developers started to bos@1: maintain ``patch series'' that modified the behaviour of the Linux bos@1: kernel. Some of these series were focused on stability, some on bos@1: feature coverage, and others were more speculative. bos@1: bos@1: The sizes of these patch series grew rapidly. In 2002, Andrew Morton bos@1: published some shell scripts he had been using to automate the task of bos@1: managing his patch queues. Andrew was successfully using these bos@1: scripts to manage hundreds (sometimes thousands) of patches on top of bos@1: the Linux kernel. bos@1: bos@1: \subsection{A patchwork quilt} bos@1: \label{sec:mq:quilt} bos@1: bos@1: bos@1: In early 2003, Andreas Gruenbacher and Martin Quinson borrowed the bos@2: approach of Andrew's scripts and published a tool called ``patchwork bos@2: quilt''~\cite{web:quilt}, or simply ``quilt'' bos@2: (see~\cite{gruenbacher:2005} for a paper describing it). Because bos@2: quilt substantially automated patch management, it rapidly gained a bos@2: large following among open source software developers. bos@1: bos@1: Quilt manages a \emph{stack of patches} on top of a directory tree. bos@1: To begin, you tell quilt to manage a directory tree; it stores away bos@1: the names and contents of all files in the tree. To fix a bug, you bos@1: create a new patch (using a single command), edit the files you need bos@1: to fix, then ``refresh'' the patch. bos@1: bos@1: The refresh step causes quilt to scan the directory tree; it updates bos@1: the patch with all of the changes you have made. You can create bos@1: another patch on top of the first, which will track the changes bos@1: required to modify the tree from ``tree with one patch applied'' to bos@1: ``tree with two patches applied''. bos@1: bos@1: You can \emph{change} which patches are applied to the tree. If you bos@1: ``pop'' a patch, the changes made by that patch will vanish from the bos@1: directory tree. Quilt remembers which patches you have popped, bos@1: though, so you can ``push'' a popped patch again, and the directory bos@1: tree will be restored to contain the modifications in the patch. Most bos@1: importantly, you can run the ``refresh'' command at any time, and the bos@1: topmost applied patch will be updated. This means that you can, at bos@1: any time, change both which patches are applied and what bos@1: modifications those patches make. bos@1: bos@1: Quilt knows nothing about revision control tools, so it works equally bos@3: well on top of an unpacked tarball or a Subversion repository. bos@1: bos@1: \subsection{From patchwork quilt to Mercurial Queues} bos@1: \label{sec:mq:quilt-mq} bos@1: bos@1: In mid-2005, Chris Mason took the features of quilt and wrote an bos@1: extension that he called Mercurial Queues, which added quilt-like bos@1: behaviour to Mercurial. bos@1: bos@1: The key difference between quilt and MQ is that quilt knows nothing bos@1: about revision control systems, while MQ is \emph{integrated} into bos@1: Mercurial. Each patch that you push is represented as a Mercurial bos@1: changeset. Pop a patch, and the changeset goes away. bos@1: bos@1: This integration makes understanding patches and debugging their bos@1: effects \emph{enormously} easier. Since every applied patch has an bos@1: associated changeset, you can use \hgcmdargs{log}{\emph{filename}} to bos@1: see which changesets and patches affected a file. You can use the bos@1: \hgext{bisect} extension to binary-search through all changesets and bos@1: applied patches to see where a bug got introduced or fixed. You can bos@1: use the \hgcmd{annotate} command to see which changeset or patch bos@1: modified a particular line of a source file. And so on. bos@1: bos@1: Because quilt does not care about revision control tools, it is still bos@1: a tremendously useful piece of software to know about for situations bos@1: where you cannot use Mercurial and MQ. bos@2: \section{Getting started with Mercurial Queues} bos@2: \label{sec:mq:start} bos@1: bos@3: Because MQ is implemented as an extension, you must explicitly enable bos@3: before you can use it. (You don't need to download anything; MQ ships bos@3: with the standard Mercurial distribution.) To enable MQ, edit your bos@4: \tildefile{.hgrc} file, and add the lines in figure~\ref{ex:mq:config}. bos@2: bos@12: \begin{figure}[ht] bos@4: \begin{codesample4} bos@4: [extensions] bos@4: hgext.mq = bos@4: \end{codesample4} bos@4: \label{ex:mq:config} bos@4: \caption{Contents to add to \tildefile{.hgrc} to enable the MQ extension} bos@4: \end{figure} bos@3: bos@3: Once the extension is enabled, it will make a number of new commands bos@7: available. To verify that the extension is working, you can use bos@7: \hgcmd{help} to see if the \hgcmd{qinit} command is now available; see bos@7: the example in figure~\ref{ex:mq:enabled}. bos@3: bos@12: \begin{figure}[ht] bos@4: \interaction{mq.qinit-help.help} bos@4: \caption{How to verify that MQ is enabled} bos@4: \label{ex:mq:enabled} bos@4: \end{figure} bos@1: bos@8: You can use MQ with \emph{any} Mercurial repository, and its commands bos@8: only operate within that repository. To get started, simply prepare bos@8: the repository using the \hgcmd{qinit} command (see bos@7: figure~\ref{ex:mq:qinit}). This command creates an empty directory bos@16: called \sdirname{.hg/patches}, where MQ will keep its metadata. As bos@7: with many Mercurial commands, the \hgcmd{qinit} command prints nothing bos@7: if it succeeds. bos@7: bos@12: \begin{figure}[ht] bos@7: \interaction{mq.tutorial.qinit} bos@7: \caption{Preparing a repository for use with MQ} bos@7: \label{ex:mq:qinit} bos@7: \end{figure} bos@7: bos@12: \begin{figure}[ht] bos@7: \interaction{mq.tutorial.qnew} bos@7: \caption{Creating a new patch} bos@7: \label{ex:mq:qnew} bos@7: \end{figure} bos@7: bos@8: \subsection{Creating a new patch} bos@8: bos@8: To begin work on a new patch, use the \hgcmd{qnew} command. This bos@7: command takes one argument, the name of the patch to create. MQ will bos@16: use this as the name of an actual file in the \sdirname{.hg/patches} bos@7: directory, as you can see in figure~\ref{ex:mq:qnew}. bos@7: bos@16: Also newly present in the \sdirname{.hg/patches} directory are two bos@16: other files, \sfilename{series} and \sfilename{status}. The bos@16: \sfilename{series} file lists all of the patches that MQ knows about bos@8: for this repository, with one patch per line. Mercurial uses the bos@16: \sfilename{status} file for internal book-keeping; it tracks all of the bos@7: patches that MQ has \emph{applied} in this repository. bos@7: bos@7: \begin{note} bos@16: You may sometimes want to edit the \sfilename{series} file by hand; bos@7: for example, to change the sequence in which some patches are bos@16: applied. However, manually editing the \sfilename{status} file is bos@7: almost always a bad idea, as it's easy to corrupt MQ's idea of what bos@7: is happening. bos@7: \end{note} bos@7: bos@8: Once you have created your new patch, you can edit files in the bos@8: working directory as you usually would. All of the normal Mercurial bos@8: commands, such as \hgcmd{diff} and \hgcmd{annotate}, work exactly as bos@8: they did before. bos@8: \subsection{Refreshing a patch} bos@8: bos@8: When you reach a point where you want to save your work, use the bos@8: \hgcmd{qrefresh} command (figure~\ref{ex:mq:qnew}) to update the patch bos@8: you are working on. This command folds the changes you have made in bos@8: the working directory into your patch, and updates its corresponding bos@8: changeset to contain those changes. bos@8: bos@12: \begin{figure}[ht] bos@8: \interaction{mq.tutorial.qrefresh} bos@8: \caption{Refreshing a patch} bos@8: \label{ex:mq:qrefresh} bos@8: \end{figure} bos@8: bos@8: You can run \hgcmd{qrefresh} as often as you like, so it's a good way bos@13: to ``checkpoint'' your work. Refresh your patch at an opportune bos@8: time; try an experiment; and if the experiment doesn't work out, bos@8: \hgcmd{revert} your modifications back to the last time you refreshed. bos@8: bos@12: \begin{figure}[ht] bos@8: \interaction{mq.tutorial.qrefresh2} bos@8: \caption{Refresh a patch many times to accumulate changes} bos@8: \label{ex:mq:qrefresh2} bos@8: \end{figure} bos@8: bos@8: \subsection{Stacking and tracking patches} bos@8: bos@8: Once you have finished working on a patch, or need to work on another, bos@8: you can use the \hgcmd{qnew} command again to create a new patch. bos@8: Mercurial will apply this patch on top of your existing patch. See bos@8: figure~\ref{ex:mq:qnew2} for an example. Notice that the patch bos@8: contains the changes in our prior patch as part of its context (you bos@8: can see this more clearly in the output of \hgcmd{annotate}). bos@8: bos@12: \begin{figure}[ht] bos@8: \interaction{mq.tutorial.qnew2} bos@8: \caption{Stacking a second patch on top of the first} bos@8: \label{ex:mq:qnew2} bos@8: \end{figure} bos@8: bos@8: So far, with the exception of \hgcmd{qnew} and \hgcmd{qrefresh}, we've bos@8: been careful to only use regular Mercurial commands. However, there bos@8: are more ``natural'' commands you can use when thinking about patches bos@8: with MQ, as illustrated in figure~\ref{ex:mq:qseries}: bos@8: bos@8: \begin{itemize} bos@8: \item The \hgcmd{qseries} command lists every patch that MQ knows bos@8: about in this repository, from oldest to newest (most recently bos@8: \emph{created}). bos@8: \item The \hgcmd{qapplied} command lists every patch that MQ has bos@8: \emph{applied} in this repository, again from oldest to newest (most bos@8: recently applied). bos@8: \end{itemize} bos@8: bos@12: \begin{figure}[ht] bos@8: \interaction{mq.tutorial.qseries} bos@8: \caption{Understanding the patch stack with \hgcmd{qseries} and bos@8: \hgcmd{qapplied}} bos@8: \label{ex:mq:qseries} bos@8: \end{figure} bos@8: bos@8: \subsection{Manipulating the patch stack} bos@8: bos@8: The previous discussion implied that there must be a difference bos@11: between ``known'' and ``applied'' patches, and there is. MQ can bos@11: manage a patch without it being applied in the repository. bos@8: bos@8: An \emph{applied} patch has a corresponding changeset in the bos@8: repository, and the effects of the patch and changeset are visible in bos@8: the working directory. You can undo the application of a patch using bos@12: the \hgcmd{qpop} command. MQ still \emph{knows about}, or manages, a bos@12: popped patch, but the patch no longer has a corresponding changeset in bos@12: the repository, and the working directory does not contain the changes bos@12: made by the patch. Figure~\ref{fig:mq:stack} illustrates the bos@12: difference between applied and tracked patches. bos@12: bos@12: \begin{figure}[ht] bos@12: \centering bos@12: \grafix{mq-stack} bos@12: \caption{Applied and unapplied patches in the MQ patch stack} bos@12: \label{fig:mq:stack} bos@8: \end{figure} bos@8: bos@8: You can reapply an unapplied, or popped, patch using the \hgcmd{qpush} bos@8: command. This creates a new changeset to correspond to the patch, and bos@8: the patch's changes once again become present in the working bos@8: directory. See figure~\ref{ex:mq:qpop} for examples of \hgcmd{qpop} bos@8: and \hgcmd{qpush} in action. Notice that once we have popped a patch bos@8: or two patches, the output of \hgcmd{qseries} remains the same, while bos@8: that of \hgcmd{qapplied} has changed. bos@8: bos@12: \begin{figure}[ht] bos@12: \interaction{mq.tutorial.qpop} bos@12: \caption{Modifying the stack of applied patches} bos@12: \label{ex:mq:qpop} bos@11: \end{figure} bos@11: bos@8: MQ does not limit you to pushing or popping one patch. You can have bos@8: no patches, all of them, or any number in between applied at some bos@8: point in time. bos@8: bos@13: \subsection{Working on several patches at once} bos@13: bos@13: The \hgcmd{qrefresh} command always refreshes the \emph{topmost} bos@13: applied patch. This means that you can suspend work on one patch (by bos@13: refreshing it), pop or push to make a different patch the top, and bos@13: work on \emph{that} patch for a while. bos@13: bos@13: Here's an example that illustrates how you can use this ability. bos@13: Let's say you're developing a new feature as two patches. The first bos@13: is a change to the core of your software, and the second--layered on bos@13: top of the first--changes the user interface to use the code you just bos@13: added to the core. If you notice a bug in the core while you're bos@13: working on the UI patch, it's easy to fix the core. Simply bos@13: \hgcmd{qrefresh} the UI patch to save your in-progress changes, and bos@13: \hgcmd{qpop} down to the core patch. Fix the core bug, bos@13: \hgcmd{qrefresh} the core patch, and \hgcmd{qpush} back to the UI bos@13: patch to continue where you left off. bos@13: bos@14: \section{Mercurial Queues and GNU patch} bos@15: \label{sec:mq:patch} bos@14: bos@16: MQ uses the GNU \command{patch} command to apply patches. Because MQ bos@16: doesn't hide its patch-oriented nature, it is helpful to understand bos@16: the data that MQ and \command{patch} work with, and a few aspects of bos@16: how \command{patch} operates. bos@14: bos@15: The \command{diff} command generates a list of modifications by bos@15: comparing two files. The \command{patch} command applies a list of bos@15: modifications to a file. The kinds of files that \command{diff} and bos@15: \command{patch} work with are referred to as both ``diffs'' and bos@15: ``patches;'' there is no difference between a diff and a patch. bos@15: bos@14: A patch file can start with arbitrary text; MQ uses this text as the bos@14: commit message when creating changesets. It treats the first line bos@14: that starts with the string ``\texttt{diff~-}'' as the separator bos@14: between header and content. bos@14: bos@15: MQ works with \emph{unified} diffs (\command{patch} can accept several bos@15: other diff formats, but MQ doesn't). A unified diff contains two bos@14: kinds of header. The \emph{file header} describes the file being bos@14: modified; it contains the name of the file to modify. When bos@15: \command{patch} sees a new file header, it looks for a file with that bos@14: name to start modifying. bos@14: bos@15: After the file header comes a series of \emph{hunks}. Each hunk bos@15: starts with a header; this identifies the range of line numbers within bos@15: the file that the hunk should modify. Following the header, a hunk bos@15: starts and ends with a few (usually three) lines of text from the bos@15: unmodified file; these are called the \emph{context} for the hunk. bos@15: Each unmodified line begins with a space characters. Within the hunk, bos@15: a line that begins with ``\texttt{-}'' means ``remove this line,'' bos@15: while a line that begins with ``\texttt{+}'' means ``insert this bos@15: line.'' For example, a line that is modified is represented by one bos@15: deletion and one insertion. bos@15: bos@15: The \command{diff} command runs hunks together when there's not enough bos@15: context between modifications to justify bos@14: bos@14: When \command{patch} applies a hunk, it tries a handful of bos@14: successively less accurate strategies to try to make the hunk apply. bos@14: This falling-back technique often makes it possible to take a patch bos@14: that was generated against an old version of a file, and apply it bos@14: against a newer version of that file. bos@14: bos@14: First, \command{patch} tries an exact match, where the line numbers, bos@14: the context, and the text to be modified must apply exactly. If it bos@14: cannot make an exact match, it tries to find an exact match for the bos@14: context, without honouring the line numbering information. If this bos@14: succeeds, it prints a line of output saying that the hunk was applied, bos@14: but at some \emph{offset} from the original line number. bos@14: bos@14: If a context-only match fails, \command{patch} removes the first and bos@14: last lines of the context, and tries a \emph{reduced} context-only bos@14: match. If the hunk with reduced context succeeds, it prints a message bos@14: saying that it applied the hunk with a \emph{fuzz factor} (the number bos@14: after the fuzz factor indicates how many lines of context bos@14: \command{patch} had to trim before the patch applied). bos@14: bos@14: When neither of these techniques works, \command{patch} prints a bos@14: message saying that the hunk in question was rejected. It saves bos@17: rejected hunks (also simply called ``rejects'') to a file with the bos@17: same name, and an added \sfilename{.rej} extension. It also saves an bos@17: unmodified copy of the file with a \sfilename{.orig} extension; the bos@17: copy of the file without any extensions will contain any changes made bos@17: by hunks that \emph{did} apply cleanly. If you have a patch that bos@17: modifies \filename{foo} with six hunks, and one of them fails to bos@17: apply, you will have: an unmodified \filename{foo.orig}, a bos@17: \filename{foo.rej} containing one hunk, and \filename{foo}, containing bos@17: the changes made by the five successful five hunks. bos@14: bos@14: \subsection{Beware the fuzz} bos@14: bos@14: While applying a hunk at an offset, or with a fuzz factor, will often bos@14: be completely successful, these inexact techniques naturally leave bos@14: open the possibility of corrupting the patched file. The most common bos@14: cases typically involve applying a patch twice, or at an incorrect bos@14: location in the file. If \command{patch} or \hgcmd{qpush} ever bos@14: mentions an offset or fuzz factor, you should make sure that the bos@14: modified files are correct afterwards. bos@14: bos@14: It's often a good idea to refresh a patch that has applied with an bos@14: offset or fuzz factor; refreshing the patch generates new context bos@14: information that will make it apply cleanly. I say ``often,'' not bos@14: ``always,'' because sometimes refreshing a patch will make it fail to bos@14: apply against a different revision of the underlying files. In some bos@14: cases, such as when you're maintaining a patch that must sit on top of bos@14: multiple versions of a source tree, it's acceptable to have a patch bos@14: apply with some fuzz, provided you've verified the results of the bos@14: patching process in such cases. bos@14: bos@15: \subsection{Handling rejection} bos@15: bos@15: If \hgcmd{qpush} fails to apply a patch, it will print an error bos@16: message and exit. If it has left \sfilename{.rej} files behind, it is bos@15: usually best to fix up the rejected hunks before you push more patches bos@15: or do any further work. bos@15: bos@15: If your patch \emph{used to} apply cleanly, and no longer does because bos@15: you've changed the underlying code that your patches are based on, bos@17: Mercurial Queues can help; see section~\ref{sec:mq:merge} for details. bos@15: bos@15: Unfortunately, there aren't any great techniques for dealing with bos@16: rejected hunks. Most often, you'll need to view the \sfilename{.rej} bos@15: file and edit the target file, applying the rejected hunks by hand. bos@15: bos@16: If you're feeling adventurous, Neil Brown, a Linux kernel hacker, bos@16: wrote a tool called \command{wiggle}~\cite{web:wiggle}, which is more bos@16: vigorous than \command{patch} in its attempts to make a patch apply. bos@15: bos@15: Another Linux kernel hacker, Chris Mason (the author of Mercurial bos@15: Queues), wrote a similar tool called \command{rej}~\cite{web:rej}, bos@15: which takes a simple approach to automating the application of hunks bos@15: rejected by \command{patch}. \command{rej} can help with four common bos@15: reasons that a hunk may be rejected: bos@15: bos@15: \begin{itemize} bos@15: \item The context in the middle of a hunk has changed. bos@15: \item A hunk is missing some context at the beginning or end. bos@15: \item A large hunk might apply better--either entirely or in part--if bos@15: it was broken up into smaller hunks. bos@15: \item A hunk removes lines with slightly different content than those bos@15: currently present in the file. bos@15: \end{itemize} bos@15: bos@15: If you use \command{wiggle} or \command{rej}, you should be doubly bos@15: careful to check your results when you're done. bos@15: bos@17: \section{Getting the best performance out of MQ} bos@17: bos@17: MQ is very efficient at handling a large number of patches. I ran bos@17: some performance experiments in mid-2006 for a talk that I gave at the bos@17: 2006 EuroPython conference~\cite{web:europython}. I used as my data bos@17: set the Linux 2.6.17-mm1 patch series, which consists of 1,738 bos@17: patches. I applied thes on top of a Linux kernel repository bos@17: containing all 27,472 revisions between Linux 2.6.12-rc2 and Linux bos@17: 2.6.17. bos@17: bos@17: On my old, slow laptop, I was able to bos@17: \hgcmdargs{qpush}{\hgopt{qpush}{-a}} all 1,738 patches in 3.5 minutes, bos@17: and \hgcmdargs{qpop}{\hgopt{qpop}{-a}} them all in 30 seconds. I bos@17: could \hgcmd{qrefresh} one of the biggest patches (which made 22,779 bos@17: lines of changes to 287 files) in 6.6 seconds. bos@17: bos@17: Clearly, MQ is well suited to working in large trees, but there are a bos@17: few tricks you can use to get the best performance of it. bos@17: bos@17: First of all, try to ``batch'' operations together. Every time you bos@17: run \hgcmd{qpush} or \hgcmd{qpop}, these commands scan the working bos@17: directory once to make sure you haven't made some changes and then bos@17: forgotten to run \hgcmd{qrefresh}. On a small tree, the time that bos@17: this scan takes is unnoticeable. However, on a medium-sized tree bos@17: (containing tens of thousands of files), it can take a second or more. bos@17: bos@17: The \hgcmd{qpush} and \hgcmd{qpop} commands allow you to push and pop bos@17: multiple patches at a time. You can identify the ``destination bos@17: patch'' that you want to end up at. When you \hgcmd{qpush} with a bos@17: destination specified, it will push patches until that patch is at the bos@17: top of the applied stack. When you \hgcmd{qpop} to a destination, MQ bos@17: will pop patches until the destination patch \emph{is no longer} bos@17: applied. bos@17: bos@17: You can identify a destination patch using either the name of the bos@17: patch, or by number. If you use numeric addressing, patches are bos@17: counted from zero; this means that the first patch is zero, the second bos@17: is one, and so on. bos@17: bos@15: \section{Updating your patches when the underlying code changes} bos@15: \label{sec:mq:merge} bos@15: bos@17: It's common to have a stack of patches on top of an underlying bos@17: repository that you don't modify directly. If you're working on bos@17: changes to third-party code, or on a feature that is taking longer to bos@17: develop than the rate of change of the code beneath, you will often bos@17: need to sync up with the underlying code, and fix up any hunks in your bos@17: patches that no longer apply. This is called \emph{rebasing} your bos@17: patch series. bos@17: bos@17: The simplest way to do this is to \hgcmdargs{qpop}{\hgopt{qpop}{-a}} bos@17: your patches, then \hgcmd{pull} changes into the underlying bos@17: repository, and finally \hgcmdargs{qpush}{\hgopt{qpop}{-a}} your bos@17: patches again. MQ will stop pushing any time it runs across a patch bos@17: that fails to apply during conflicts, allowing you to fix your bos@17: conflicts, \hgcmd{qrefresh} the affected patch, and continue pushing bos@17: until you have fixed your entire stack. bos@17: bos@17: This approach is easy to use and works well if you don't expect bos@17: changes to the underlying code to affect how well your patches apply. bos@17: If your patch stack touches code that is modified frequently or bos@17: invasively in the underlying repository, however, fixing up rejected bos@17: hunks by hand quickly becomes tiresome. bos@17: bos@17: It's possible to partially automate the rebasing process. If your bos@17: patches apply cleanly against some revision of the underlying repo, MQ bos@17: can use this information to help you to resolve conflicts between your bos@17: patches and a different revision. bos@17: bos@17: The process is a little involved. bos@17: \begin{enumerate} bos@17: \item To begin, \hgcmdargs{qpush}{-a} all of your patches on top of bos@17: the revision where you know that they apply cleanly. bos@17: \item Save a backup copy of your patch directory using bos@17: \hgcmdargs{qsave}{\hgopt{qsave}{-e} \hgopt{qsave}{-c}}. This prints bos@17: the name of the directory that it has saved the patches in. It will bos@17: save the patches to a directory called bos@17: \sdirname{.hg/patches.\emph{N}}, where \texttt{\emph{N}} is a small bos@17: integer. It also commits a ``save changeset'' on top of your bos@17: applied patches; this is for internal book-keeping, and records the bos@17: states of the \sfilename{series} and \sfilename{status} files. bos@17: \item Use \hgcmd{pull} to bring new changes into the underlying bos@17: repository. (Don't run \hgcmdargs{pull}{-u}; see below for why.) bos@17: \item Update to the new tip revision, using bos@17: \hgcmdargs{update}{\hgopt{update}{-C}} to override the patches you bos@17: have pushed. bos@17: \item Merge all patches using \hgcmdargs{qpush}{\hgopt{qpush}{-m} bos@17: \hgopt{qpush}{-a}}. The \hgopt{qpush}{-m} option to \hgcmd{qpush} bos@17: tells MQ to perform a three-way merge if the patch fails to apply. bos@17: \end{enumerate} bos@17: bos@17: During the \hgcmdargs{qpush}{\hgopt{qpush}{-m}}, each patch in the bos@17: \sfilename{series} file is applied normally. If a patch applies with bos@17: fuzz or rejects, MQ looks at the queue you \hgcmd{qsave}d, and bos@17: performs a three-way merge with the corresponding changeset. This bos@17: merge uses Mercurial's normal merge machinery, so it may pop up a GUI bos@17: merge tool to help you to resolve problems. bos@17: bos@17: When you finish resolving the effects of a patch, MQ refreshes your bos@17: patch based on the result of the merge. bos@17: bos@17: At the end of this process, your repository will have one extra head bos@17: from the old patch queue, and a copy of the old patch queue will be in bos@17: \sdirname{.hg/patches.\emph{N}}. You can remove the extra head using bos@17: \hgcmdargs{qpop}{\hgopt{qpop}{-a} \hgopt{qpop}{-n} patches.\emph{N}} bos@17: or \hgcmd{strip}. You can delete \sdirname{.hg/patches.\emph{N}} once bos@17: you are sure that you no longer need it as a backup. bos@13: bos@16: \section{Managing patches in a repository} bos@16: bos@16: Because MQ's \sdirname{.hg/patches} directory resides outside a bos@16: Mercurial repository's working directory, the ``underlying'' Mercurial bos@16: repository knows nothing about the management or presence of patches. bos@16: bos@16: This presents the interesting possibility of managing the contents of bos@16: the patch directory as a Mercurial repository in its own right. This bos@16: can be a useful way to work. For example, you can work on a patch for bos@16: a while, \hgcmd{qrefresh} it, then \hgcmd{commit} the current state of bos@16: the patch. This lets you ``roll back'' to that version of the patch bos@16: later on. bos@16: bos@16: In addition, you can then share different versions of the same patch bos@16: stack among multiple underlying repositories. I use this when I am bos@16: developing a Linux kernel feature. I have a pristine copy of my bos@16: kernel sources for each of several CPU architectures, and a cloned bos@16: repository under each that contains the patches I am working on. When bos@16: I want to test a change on a different architecture, I push my current bos@16: patches to the patch repository associated with that kernel tree, pop bos@16: and push all of my patches, and build and test that kernel. bos@16: bos@16: Managing patches in a repository makes it possible for multiple bos@16: developers to work on the same patch series without colliding with bos@16: each other, all on top of an underlying source base that they may or bos@16: may not control. bos@16: bos@17: \subsection{MQ support for patch repositories} bos@16: bos@16: MQ helps you to work with the \sdirname{.hg/patches} directory as a bos@16: repository; when you prepare a repository for working with patches bos@17: using \hgcmd{qinit}, you can pass the \hgopt{qinit}{-c} option to bos@16: create the \sdirname{.hg/patches} directory as a Mercurial repository. bos@16: bos@16: \begin{note} bos@16: If you forget to use the \hgopt{qinit}{-c} option, you can simply go bos@16: into the \sdirname{.hg/patches} directory at any time and run bos@16: \hgcmd{init}. Don't forget to add an entry for the bos@17: \sfilename{status} file to the \sfilename{.hgignore} file, though bos@17: (\hgcmdargs{qinit}{\hgopt{qinit}{-c}} does this for you bos@17: automatically); you \emph{really} don't want to manage the bos@17: \sfilename{status} file. bos@16: \end{note} bos@16: bos@16: As a convenience, if MQ notices that the \dirname{.hg/patches} bos@16: directory is a repository, it will automatically \hgcmd{add} every bos@16: patch that you create and import. bos@16: bos@16: Finally, MQ provides a shortcut command, \hgcmd{qcommit}, that runs bos@16: \hgcmd{commit} in the \sdirname{.hg/patches} directory. This saves bos@16: some cumbersome typing. bos@16: bos@16: \subsection{A few things to watch out for} bos@16: bos@16: MQ's support for working with a repository full of patches is limited bos@16: in a few small respects. bos@16: bos@16: MQ cannot automatically detect changes that you make to the patch bos@16: directory. If you \hgcmd{pull}, manually edit, or \hgcmd{update} bos@16: changes to patches or the \sfilename{series} file, you will have to bos@17: \hgcmdargs{qpop}{\hgopt{qpop}{-a}} and then bos@17: \hgcmdargs{qpush}{\hgopt{qpush}{-a}} in the underlying repository to bos@17: see those changes show up there. If you forget to do this, you can bos@17: confuse MQ's idea of which patches are applied. bos@16: bos@16: \section{Commands for working with patches} bos@16: bos@16: Once you've been working with patches for a while, you'll find bos@16: yourself hungry for tools that will help you to understand and bos@16: manipulate the patches you're dealing with. bos@16: bos@16: The \command{diffstat} command~\cite{web:diffstat} generates a bos@16: histogram of the modifications made to each file in a patch. It bos@16: provides a good way to ``get a sense of'' a patch--which files it bos@16: affects, and how much change it introduces to each file and as a bos@16: whole. (I find that it's a good idea to use \command{diffstat}'s bos@16: \texttt{-p} option as a matter of course, as otherwise it will try to bos@16: do clever things with prefixes of file names that inevitably confuse bos@16: at least me.) bos@16: bos@16: The \package{patchutils} package~\cite{web:patchutils} is invaluable. bos@16: It provides a set of small utilities that follow the ``Unix bos@16: philosophy;'' each does one useful thing with a patch. The bos@16: \package{patchutils} command I use most is \command{filterdiff}, which bos@16: extracts subsets from a patch file. For example, given a patch that bos@16: modifies hundreds of files across dozens of directories, a single bos@16: invocation of \command{filterdiff} can generate a smaller patch that bos@16: only touches files whose names match a particular glob pattern. bos@16: bos@1: %%% Local Variables: bos@1: %%% mode: latex bos@1: %%% TeX-master: "00book" bos@1: %%% End: