jerojasro@336: \chapter{Managing change with Mercurial Queues} jerojasro@336: \label{chap:mq} jerojasro@336: jerojasro@336: \section{The patch management problem} jerojasro@336: \label{sec:mq:patch-mgmt} jerojasro@336: jerojasro@336: Here is a common scenario: you need to install a software package from jerojasro@336: source, but you find a bug that you must fix in the source before you jerojasro@336: can start using the package. You make your changes, forget about the jerojasro@336: package for a while, and a few months later you need to upgrade to a jerojasro@336: newer version of the package. If the newer version of the package jerojasro@336: still has the bug, you must extract your fix from the older source jerojasro@336: tree and apply it against the newer version. This is a tedious task, jerojasro@336: and it's easy to make mistakes. jerojasro@336: jerojasro@336: This is a simple case of the ``patch management'' problem. You have jerojasro@336: an ``upstream'' source tree that you can't change; you need to make jerojasro@336: some local changes on top of the upstream tree; and you'd like to be jerojasro@336: able to keep those changes separate, so that you can apply them to jerojasro@336: newer versions of the upstream source. jerojasro@336: jerojasro@336: The patch management problem arises in many situations. Probably the jerojasro@336: most visible is that a user of an open source software project will jerojasro@336: contribute a bug fix or new feature to the project's maintainers in the jerojasro@336: form of a patch. jerojasro@336: jerojasro@336: Distributors of operating systems that include open source software jerojasro@336: often need to make changes to the packages they distribute so that jerojasro@336: they will build properly in their environments. jerojasro@336: jerojasro@336: When you have few changes to maintain, it is easy to manage a single jerojasro@336: patch using the standard \command{diff} and \command{patch} programs jerojasro@336: (see section~\ref{sec:mq:patch} for a discussion of these tools). jerojasro@336: Once the number of changes grows, it starts to make sense to maintain jerojasro@336: patches as discrete ``chunks of work,'' so that for example a single jerojasro@336: patch will contain only one bug fix (the patch might modify several jerojasro@336: files, but it's doing ``only one thing''), and you may have a number jerojasro@336: of such patches for different bugs you need fixed and local changes jerojasro@336: you require. In this situation, if you submit a bug fix patch to the jerojasro@336: upstream maintainers of a package and they include your fix in a jerojasro@336: subsequent release, you can simply drop that single patch when you're jerojasro@336: updating to the newer release. jerojasro@336: jerojasro@336: Maintaining a single patch against an upstream tree is a little jerojasro@336: tedious and error-prone, but not difficult. However, the complexity jerojasro@336: of the problem grows rapidly as the number of patches you have to jerojasro@336: maintain increases. With more than a tiny number of patches in hand, jerojasro@336: understanding which ones you have applied and maintaining them moves jerojasro@336: from messy to overwhelming. jerojasro@336: jerojasro@336: Fortunately, Mercurial includes a powerful extension, Mercurial Queues jerojasro@336: (or simply ``MQ''), that massively simplifies the patch management jerojasro@336: problem. jerojasro@336: jerojasro@336: \section{The prehistory of Mercurial Queues} jerojasro@336: \label{sec:mq:history} jerojasro@336: jerojasro@336: During the late 1990s, several Linux kernel developers started to jerojasro@336: maintain ``patch series'' that modified the behaviour of the Linux jerojasro@336: kernel. Some of these series were focused on stability, some on jerojasro@336: feature coverage, and others were more speculative. jerojasro@336: jerojasro@336: The sizes of these patch series grew rapidly. In 2002, Andrew Morton jerojasro@336: published some shell scripts he had been using to automate the task of jerojasro@336: managing his patch queues. Andrew was successfully using these jerojasro@336: scripts to manage hundreds (sometimes thousands) of patches on top of jerojasro@336: the Linux kernel. jerojasro@336: jerojasro@336: \subsection{A patchwork quilt} jerojasro@336: \label{sec:mq:quilt} jerojasro@336: jerojasro@336: In early 2003, Andreas Gruenbacher and Martin Quinson borrowed the jerojasro@336: approach of Andrew's scripts and published a tool called ``patchwork jerojasro@336: quilt''~\cite{web:quilt}, or simply ``quilt'' jerojasro@336: (see~\cite{gruenbacher:2005} for a paper describing it). Because jerojasro@336: quilt substantially automated patch management, it rapidly gained a jerojasro@336: large following among open source software developers. jerojasro@336: jerojasro@336: Quilt manages a \emph{stack of patches} on top of a directory tree. jerojasro@336: To begin, you tell quilt to manage a directory tree, and tell it which jerojasro@336: files you want to manage; it stores away the names and contents of jerojasro@336: those files. To fix a bug, you create a new patch (using a single jerojasro@336: command), edit the files you need to fix, then ``refresh'' the patch. jerojasro@336: jerojasro@336: The refresh step causes quilt to scan the directory tree; it updates jerojasro@336: the patch with all of the changes you have made. You can create jerojasro@336: another patch on top of the first, which will track the changes jerojasro@336: required to modify the tree from ``tree with one patch applied'' to jerojasro@336: ``tree with two patches applied''. jerojasro@336: jerojasro@336: You can \emph{change} which patches are applied to the tree. If you jerojasro@336: ``pop'' a patch, the changes made by that patch will vanish from the jerojasro@336: directory tree. Quilt remembers which patches you have popped, jerojasro@336: though, so you can ``push'' a popped patch again, and the directory jerojasro@336: tree will be restored to contain the modifications in the patch. Most jerojasro@336: importantly, you can run the ``refresh'' command at any time, and the jerojasro@336: topmost applied patch will be updated. This means that you can, at jerojasro@336: any time, change both which patches are applied and what jerojasro@336: modifications those patches make. jerojasro@336: jerojasro@336: Quilt knows nothing about revision control tools, so it works equally jerojasro@336: well on top of an unpacked tarball or a Subversion working copy. jerojasro@336: jerojasro@336: \subsection{From patchwork quilt to Mercurial Queues} jerojasro@336: \label{sec:mq:quilt-mq} jerojasro@336: jerojasro@336: In mid-2005, Chris Mason took the features of quilt and wrote an jerojasro@336: extension that he called Mercurial Queues, which added quilt-like jerojasro@336: behaviour to Mercurial. jerojasro@336: jerojasro@336: The key difference between quilt and MQ is that quilt knows nothing jerojasro@336: about revision control systems, while MQ is \emph{integrated} into jerojasro@336: Mercurial. Each patch that you push is represented as a Mercurial jerojasro@336: changeset. Pop a patch, and the changeset goes away. jerojasro@336: jerojasro@336: Because quilt does not care about revision control tools, it is still jerojasro@336: a tremendously useful piece of software to know about for situations jerojasro@336: where you cannot use Mercurial and MQ. jerojasro@336: jerojasro@336: \section{The huge advantage of MQ} jerojasro@336: jerojasro@336: I cannot overstate the value that MQ offers through the unification of jerojasro@336: patches and revision control. jerojasro@336: jerojasro@336: A major reason that patches have persisted in the free software and jerojasro@336: open source world---in spite of the availability of increasingly jerojasro@336: capable revision control tools over the years---is the \emph{agility} jerojasro@336: they offer. jerojasro@336: jerojasro@336: Traditional revision control tools make a permanent, irreversible jerojasro@336: record of everything that you do. While this has great value, it's jerojasro@336: also somewhat stifling. If you want to perform a wild-eyed jerojasro@336: experiment, you have to be careful in how you go about it, or you risk jerojasro@336: leaving unneeded---or worse, misleading or destabilising---traces of jerojasro@336: your missteps and errors in the permanent revision record. jerojasro@336: jerojasro@336: By contrast, MQ's marriage of distributed revision control with jerojasro@336: patches makes it much easier to isolate your work. Your patches live jerojasro@336: on top of normal revision history, and you can make them disappear or jerojasro@336: reappear at will. If you don't like a patch, you can drop it. If a jerojasro@336: patch isn't quite as you want it to be, simply fix it---as many times jerojasro@336: as you need to, until you have refined it into the form you desire. jerojasro@336: jerojasro@336: As an example, the integration of patches with revision control makes jerojasro@336: understanding patches and debugging their effects---and their jerojasro@336: interplay with the code they're based on---\emph{enormously} easier. jerojasro@336: Since every applied patch has an associated changeset, you can use jerojasro@336: \hgcmdargs{log}{\emph{filename}} to see which changesets and patches jerojasro@336: affected a file. You can use the \hgext{bisect} command to jerojasro@336: binary-search through all changesets and applied patches to see where jerojasro@336: a bug got introduced or fixed. You can use the \hgcmd{annotate} jerojasro@336: command to see which changeset or patch modified a particular line of jerojasro@336: a source file. And so on. jerojasro@336: jerojasro@336: \section{Understanding patches} jerojasro@336: \label{sec:mq:patch} jerojasro@336: jerojasro@336: Because MQ doesn't hide its patch-oriented nature, it is helpful to jerojasro@336: understand what patches are, and a little about the tools that work jerojasro@336: with them. jerojasro@336: jerojasro@336: The traditional Unix \command{diff} command compares two files, and jerojasro@336: prints a list of differences between them. The \command{patch} command jerojasro@336: understands these differences as \emph{modifications} to make to a jerojasro@336: file. Take a look at figure~\ref{ex:mq:diff} for a simple example of jerojasro@336: these commands in action. jerojasro@336: jerojasro@336: \begin{figure}[ht] jerojasro@336: \interaction{mq.dodiff.diff} jerojasro@336: \caption{Simple uses of the \command{diff} and \command{patch} commands} jerojasro@336: \label{ex:mq:diff} jerojasro@336: \end{figure} jerojasro@336: jerojasro@336: The type of file that \command{diff} generates (and \command{patch} jerojasro@336: takes as input) is called a ``patch'' or a ``diff''; there is no jerojasro@336: difference between a patch and a diff. (We'll use the term ``patch'', jerojasro@336: since it's more commonly used.) jerojasro@336: jerojasro@336: A patch file can start with arbitrary text; the \command{patch} jerojasro@336: command ignores this text, but MQ uses it as the commit message when jerojasro@336: creating changesets. To find the beginning of the patch content, jerojasro@336: \command{patch} searches for the first line that starts with the jerojasro@336: string ``\texttt{diff~-}''. jerojasro@336: jerojasro@336: MQ works with \emph{unified} diffs (\command{patch} can accept several jerojasro@336: other diff formats, but MQ doesn't). A unified diff contains two jerojasro@336: kinds of header. The \emph{file header} describes the file being jerojasro@336: modified; it contains the name of the file to modify. When jerojasro@336: \command{patch} sees a new file header, it looks for a file with that jerojasro@336: name to start modifying. jerojasro@336: jerojasro@336: After the file header comes a series of \emph{hunks}. Each hunk jerojasro@336: starts with a header; this identifies the range of line numbers within jerojasro@336: the file that the hunk should modify. Following the header, a hunk jerojasro@336: starts and ends with a few (usually three) lines of text from the jerojasro@336: unmodified file; these are called the \emph{context} for the hunk. If jerojasro@336: there's only a small amount of context between successive hunks, jerojasro@336: \command{diff} doesn't print a new hunk header; it just runs the hunks jerojasro@336: together, with a few lines of context between modifications. jerojasro@336: jerojasro@336: Each line of context begins with a space character. Within the hunk, jerojasro@336: a line that begins with ``\texttt{-}'' means ``remove this line,'' jerojasro@336: while a line that begins with ``\texttt{+}'' means ``insert this jerojasro@336: line.'' For example, a line that is modified is represented by one jerojasro@336: deletion and one insertion. jerojasro@336: jerojasro@336: We will return to some of the more subtle aspects of patches later (in jerojasro@336: section~\ref{sec:mq:adv-patch}), but you should have enough information jerojasro@336: now to use MQ. jerojasro@336: jerojasro@336: \section{Getting started with Mercurial Queues} jerojasro@336: \label{sec:mq:start} jerojasro@336: jerojasro@336: Because MQ is implemented as an extension, you must explicitly enable jerojasro@336: before you can use it. (You don't need to download anything; MQ ships jerojasro@336: with the standard Mercurial distribution.) To enable MQ, edit your jerojasro@336: \tildefile{.hgrc} file, and add the lines in figure~\ref{ex:mq:config}. jerojasro@336: jerojasro@336: \begin{figure}[ht] jerojasro@336: \begin{codesample4} jerojasro@336: [extensions] jerojasro@336: hgext.mq = jerojasro@336: \end{codesample4} jerojasro@336: \label{ex:mq:config} jerojasro@336: \caption{Contents to add to \tildefile{.hgrc} to enable the MQ extension} jerojasro@336: \end{figure} jerojasro@336: jerojasro@336: Once the extension is enabled, it will make a number of new commands jerojasro@336: available. To verify that the extension is working, you can use jerojasro@336: \hgcmd{help} to see if the \hgxcmd{mq}{qinit} command is now available; see jerojasro@336: the example in figure~\ref{ex:mq:enabled}. jerojasro@336: jerojasro@336: \begin{figure}[ht] jerojasro@336: \interaction{mq.qinit-help.help} jerojasro@336: \caption{How to verify that MQ is enabled} jerojasro@336: \label{ex:mq:enabled} jerojasro@336: \end{figure} jerojasro@336: jerojasro@336: You can use MQ with \emph{any} Mercurial repository, and its commands jerojasro@336: only operate within that repository. To get started, simply prepare jerojasro@336: the repository using the \hgxcmd{mq}{qinit} command (see jerojasro@336: figure~\ref{ex:mq:qinit}). This command creates an empty directory jerojasro@336: called \sdirname{.hg/patches}, where MQ will keep its metadata. As jerojasro@336: with many Mercurial commands, the \hgxcmd{mq}{qinit} command prints nothing jerojasro@336: if it succeeds. jerojasro@336: jerojasro@336: \begin{figure}[ht] jerojasro@336: \interaction{mq.tutorial.qinit} jerojasro@336: \caption{Preparing a repository for use with MQ} jerojasro@336: \label{ex:mq:qinit} jerojasro@336: \end{figure} jerojasro@336: jerojasro@336: \begin{figure}[ht] jerojasro@336: \interaction{mq.tutorial.qnew} jerojasro@336: \caption{Creating a new patch} jerojasro@336: \label{ex:mq:qnew} jerojasro@336: \end{figure} jerojasro@336: jerojasro@336: \subsection{Creating a new patch} jerojasro@336: jerojasro@336: To begin work on a new patch, use the \hgxcmd{mq}{qnew} command. This jerojasro@336: command takes one argument, the name of the patch to create. MQ will jerojasro@336: use this as the name of an actual file in the \sdirname{.hg/patches} jerojasro@336: directory, as you can see in figure~\ref{ex:mq:qnew}. jerojasro@336: jerojasro@336: Also newly present in the \sdirname{.hg/patches} directory are two jerojasro@336: other files, \sfilename{series} and \sfilename{status}. The jerojasro@336: \sfilename{series} file lists all of the patches that MQ knows about jerojasro@336: for this repository, with one patch per line. Mercurial uses the jerojasro@336: \sfilename{status} file for internal book-keeping; it tracks all of the jerojasro@336: patches that MQ has \emph{applied} in this repository. jerojasro@336: jerojasro@336: \begin{note} jerojasro@336: You may sometimes want to edit the \sfilename{series} file by hand; jerojasro@336: for example, to change the sequence in which some patches are jerojasro@336: applied. However, manually editing the \sfilename{status} file is jerojasro@336: almost always a bad idea, as it's easy to corrupt MQ's idea of what jerojasro@336: is happening. jerojasro@336: \end{note} jerojasro@336: jerojasro@336: Once you have created your new patch, you can edit files in the jerojasro@336: working directory as you usually would. All of the normal Mercurial jerojasro@336: commands, such as \hgcmd{diff} and \hgcmd{annotate}, work exactly as jerojasro@336: they did before. jerojasro@336: jerojasro@336: \subsection{Refreshing a patch} jerojasro@336: jerojasro@336: When you reach a point where you want to save your work, use the jerojasro@336: \hgxcmd{mq}{qrefresh} command (figure~\ref{ex:mq:qnew}) to update the patch jerojasro@336: you are working on. This command folds the changes you have made in jerojasro@336: the working directory into your patch, and updates its corresponding jerojasro@336: changeset to contain those changes. jerojasro@336: jerojasro@336: \begin{figure}[ht] jerojasro@336: \interaction{mq.tutorial.qrefresh} jerojasro@336: \caption{Refreshing a patch} jerojasro@336: \label{ex:mq:qrefresh} jerojasro@336: \end{figure} jerojasro@336: jerojasro@336: You can run \hgxcmd{mq}{qrefresh} as often as you like, so it's a good way jerojasro@336: to ``checkpoint'' your work. Refresh your patch at an opportune jerojasro@336: time; try an experiment; and if the experiment doesn't work out, jerojasro@336: \hgcmd{revert} your modifications back to the last time you refreshed. jerojasro@336: jerojasro@336: \begin{figure}[ht] jerojasro@336: \interaction{mq.tutorial.qrefresh2} jerojasro@336: \caption{Refresh a patch many times to accumulate changes} jerojasro@336: \label{ex:mq:qrefresh2} jerojasro@336: \end{figure} jerojasro@336: jerojasro@336: \subsection{Stacking and tracking patches} jerojasro@336: jerojasro@336: Once you have finished working on a patch, or need to work on another, jerojasro@336: you can use the \hgxcmd{mq}{qnew} command again to create a new patch. jerojasro@336: Mercurial will apply this patch on top of your existing patch. See jerojasro@336: figure~\ref{ex:mq:qnew2} for an example. Notice that the patch jerojasro@336: contains the changes in our prior patch as part of its context (you jerojasro@336: can see this more clearly in the output of \hgcmd{annotate}). jerojasro@336: jerojasro@336: \begin{figure}[ht] jerojasro@336: \interaction{mq.tutorial.qnew2} jerojasro@336: \caption{Stacking a second patch on top of the first} jerojasro@336: \label{ex:mq:qnew2} jerojasro@336: \end{figure} jerojasro@336: jerojasro@336: So far, with the exception of \hgxcmd{mq}{qnew} and \hgxcmd{mq}{qrefresh}, we've jerojasro@336: been careful to only use regular Mercurial commands. However, MQ jerojasro@336: provides many commands that are easier to use when you are thinking jerojasro@336: about patches, as illustrated in figure~\ref{ex:mq:qseries}: jerojasro@336: jerojasro@336: \begin{itemize} jerojasro@336: \item The \hgxcmd{mq}{qseries} command lists every patch that MQ knows jerojasro@336: about in this repository, from oldest to newest (most recently jerojasro@336: \emph{created}). jerojasro@336: \item The \hgxcmd{mq}{qapplied} command lists every patch that MQ has jerojasro@336: \emph{applied} in this repository, again from oldest to newest (most jerojasro@336: recently applied). jerojasro@336: \end{itemize} jerojasro@336: jerojasro@336: \begin{figure}[ht] jerojasro@336: \interaction{mq.tutorial.qseries} jerojasro@336: \caption{Understanding the patch stack with \hgxcmd{mq}{qseries} and jerojasro@336: \hgxcmd{mq}{qapplied}} jerojasro@336: \label{ex:mq:qseries} jerojasro@336: \end{figure} jerojasro@336: jerojasro@336: \subsection{Manipulating the patch stack} jerojasro@336: jerojasro@336: The previous discussion implied that there must be a difference jerojasro@336: between ``known'' and ``applied'' patches, and there is. MQ can jerojasro@336: manage a patch without it being applied in the repository. jerojasro@336: jerojasro@336: An \emph{applied} patch has a corresponding changeset in the jerojasro@336: repository, and the effects of the patch and changeset are visible in jerojasro@336: the working directory. You can undo the application of a patch using jerojasro@336: the \hgxcmd{mq}{qpop} command. MQ still \emph{knows about}, or manages, a jerojasro@336: popped patch, but the patch no longer has a corresponding changeset in jerojasro@336: the repository, and the working directory does not contain the changes jerojasro@336: made by the patch. Figure~\ref{fig:mq:stack} illustrates the jerojasro@336: difference between applied and tracked patches. jerojasro@336: jerojasro@336: \begin{figure}[ht] jerojasro@336: \centering jerojasro@336: \grafix{mq-stack} jerojasro@336: \caption{Applied and unapplied patches in the MQ patch stack} jerojasro@336: \label{fig:mq:stack} jerojasro@336: \end{figure} jerojasro@336: jerojasro@336: You can reapply an unapplied, or popped, patch using the \hgxcmd{mq}{qpush} jerojasro@336: command. This creates a new changeset to correspond to the patch, and jerojasro@336: the patch's changes once again become present in the working jerojasro@336: directory. See figure~\ref{ex:mq:qpop} for examples of \hgxcmd{mq}{qpop} jerojasro@336: and \hgxcmd{mq}{qpush} in action. Notice that once we have popped a patch jerojasro@336: or two patches, the output of \hgxcmd{mq}{qseries} remains the same, while jerojasro@336: that of \hgxcmd{mq}{qapplied} has changed. jerojasro@336: jerojasro@336: \begin{figure}[ht] jerojasro@336: \interaction{mq.tutorial.qpop} jerojasro@336: \caption{Modifying the stack of applied patches} jerojasro@336: \label{ex:mq:qpop} jerojasro@336: \end{figure} jerojasro@336: jerojasro@336: \subsection{Pushing and popping many patches} jerojasro@336: jerojasro@336: While \hgxcmd{mq}{qpush} and \hgxcmd{mq}{qpop} each operate on a single patch at jerojasro@336: a time by default, you can push and pop many patches in one go. The jerojasro@336: \hgxopt{mq}{qpush}{-a} option to \hgxcmd{mq}{qpush} causes it to push all jerojasro@336: unapplied patches, while the \hgxopt{mq}{qpop}{-a} option to \hgxcmd{mq}{qpop} jerojasro@336: causes it to pop all applied patches. (For some more ways to push and jerojasro@336: pop many patches, see section~\ref{sec:mq:perf} below.) jerojasro@336: jerojasro@336: \begin{figure}[ht] jerojasro@336: \interaction{mq.tutorial.qpush-a} jerojasro@336: \caption{Pushing all unapplied patches} jerojasro@336: \label{ex:mq:qpush-a} jerojasro@336: \end{figure} jerojasro@336: jerojasro@336: \subsection{Safety checks, and overriding them} jerojasro@336: jerojasro@336: Several MQ commands check the working directory before they do jerojasro@336: anything, and fail if they find any modifications. They do this to jerojasro@336: ensure that you won't lose any changes that you have made, but not yet jerojasro@336: incorporated into a patch. Figure~\ref{ex:mq:add} illustrates this; jerojasro@336: the \hgxcmd{mq}{qnew} command will not create a new patch if there are jerojasro@336: outstanding changes, caused in this case by the \hgcmd{add} of jerojasro@336: \filename{file3}. jerojasro@336: jerojasro@336: \begin{figure}[ht] jerojasro@336: \interaction{mq.tutorial.add} jerojasro@336: \caption{Forcibly creating a patch} jerojasro@336: \label{ex:mq:add} jerojasro@336: \end{figure} jerojasro@336: jerojasro@336: Commands that check the working directory all take an ``I know what jerojasro@336: I'm doing'' option, which is always named \option{-f}. The exact jerojasro@336: meaning of \option{-f} depends on the command. For example, jerojasro@336: \hgcmdargs{qnew}{\hgxopt{mq}{qnew}{-f}} will incorporate any outstanding jerojasro@336: changes into the new patch it creates, but jerojasro@336: \hgcmdargs{qpop}{\hgxopt{mq}{qpop}{-f}} will revert modifications to any jerojasro@336: files affected by the patch that it is popping. Be sure to read the jerojasro@336: documentation for a command's \option{-f} option before you use it! jerojasro@336: jerojasro@336: \subsection{Working on several patches at once} jerojasro@336: jerojasro@336: The \hgxcmd{mq}{qrefresh} command always refreshes the \emph{topmost} jerojasro@336: applied patch. This means that you can suspend work on one patch (by jerojasro@336: refreshing it), pop or push to make a different patch the top, and jerojasro@336: work on \emph{that} patch for a while. jerojasro@336: jerojasro@336: Here's an example that illustrates how you can use this ability. jerojasro@336: Let's say you're developing a new feature as two patches. The first jerojasro@336: is a change to the core of your software, and the second---layered on jerojasro@336: top of the first---changes the user interface to use the code you just jerojasro@336: added to the core. If you notice a bug in the core while you're jerojasro@336: working on the UI patch, it's easy to fix the core. Simply jerojasro@336: \hgxcmd{mq}{qrefresh} the UI patch to save your in-progress changes, and jerojasro@336: \hgxcmd{mq}{qpop} down to the core patch. Fix the core bug, jerojasro@336: \hgxcmd{mq}{qrefresh} the core patch, and \hgxcmd{mq}{qpush} back to the UI jerojasro@336: patch to continue where you left off. jerojasro@336: jerojasro@336: \section{More about patches} jerojasro@336: \label{sec:mq:adv-patch} jerojasro@336: jerojasro@336: MQ uses the GNU \command{patch} command to apply patches, so it's jerojasro@336: helpful to know a few more detailed aspects of how \command{patch} jerojasro@336: works, and about patches themselves. jerojasro@336: jerojasro@336: \subsection{The strip count} jerojasro@336: jerojasro@336: If you look at the file headers in a patch, you will notice that the jerojasro@336: pathnames usually have an extra component on the front that isn't jerojasro@336: present in the actual path name. This is a holdover from the way that jerojasro@336: people used to generate patches (people still do this, but it's jerojasro@336: somewhat rare with modern revision control tools). jerojasro@336: jerojasro@336: Alice would unpack a tarball, edit her files, then decide that she jerojasro@336: wanted to create a patch. So she'd rename her working directory, jerojasro@336: unpack the tarball again (hence the need for the rename), and use the jerojasro@336: \cmdopt{diff}{-r} and \cmdopt{diff}{-N} options to \command{diff} to jerojasro@336: recursively generate a patch between the unmodified directory and the jerojasro@336: modified one. The result would be that the name of the unmodified jerojasro@336: directory would be at the front of the left-hand path in every file jerojasro@336: header, and the name of the modified directory would be at the front jerojasro@336: of the right-hand path. jerojasro@336: jerojasro@336: Since someone receiving a patch from the Alices of the net would be jerojasro@336: unlikely to have unmodified and modified directories with exactly the jerojasro@336: same names, the \command{patch} command has a \cmdopt{patch}{-p} jerojasro@336: option that indicates the number of leading path name components to jerojasro@336: strip when trying to apply a patch. This number is called the jerojasro@336: \emph{strip count}. jerojasro@336: jerojasro@336: An option of ``\texttt{-p1}'' means ``use a strip count of one''. If jerojasro@336: \command{patch} sees a file name \filename{foo/bar/baz} in a file jerojasro@336: header, it will strip \filename{foo} and try to patch a file named jerojasro@336: \filename{bar/baz}. (Strictly speaking, the strip count refers to the jerojasro@336: number of \emph{path separators} (and the components that go with them jerojasro@336: ) to strip. A strip count of one will turn \filename{foo/bar} into jerojasro@336: \filename{bar}, but \filename{/foo/bar} (notice the extra leading jerojasro@336: slash) into \filename{foo/bar}.) jerojasro@336: jerojasro@336: The ``standard'' strip count for patches is one; almost all patches jerojasro@336: contain one leading path name component that needs to be stripped. jerojasro@336: Mercurial's \hgcmd{diff} command generates path names in this form, jerojasro@336: and the \hgcmd{import} command and MQ expect patches to have a strip jerojasro@336: count of one. jerojasro@336: jerojasro@336: If you receive a patch from someone that you want to add to your patch jerojasro@336: queue, and the patch needs a strip count other than one, you cannot jerojasro@336: just \hgxcmd{mq}{qimport} the patch, because \hgxcmd{mq}{qimport} does not yet jerojasro@336: have a \texttt{-p} option (see~\bug{311}). Your best bet is to jerojasro@336: \hgxcmd{mq}{qnew} a patch of your own, then use \cmdargs{patch}{-p\emph{N}} jerojasro@336: to apply their patch, followed by \hgcmd{addremove} to pick up any jerojasro@336: files added or removed by the patch, followed by \hgxcmd{mq}{qrefresh}. jerojasro@336: This complexity may become unnecessary; see~\bug{311} for details. jerojasro@336: \subsection{Strategies for applying a patch} jerojasro@336: jerojasro@336: When \command{patch} applies a hunk, it tries a handful of jerojasro@336: successively less accurate strategies to try to make the hunk apply. jerojasro@336: This falling-back technique often makes it possible to take a patch jerojasro@336: that was generated against an old version of a file, and apply it jerojasro@336: against a newer version of that file. jerojasro@336: jerojasro@336: First, \command{patch} tries an exact match, where the line numbers, jerojasro@336: the context, and the text to be modified must apply exactly. If it jerojasro@336: cannot make an exact match, it tries to find an exact match for the jerojasro@336: context, without honouring the line numbering information. If this jerojasro@336: succeeds, it prints a line of output saying that the hunk was applied, jerojasro@336: but at some \emph{offset} from the original line number. jerojasro@336: jerojasro@336: If a context-only match fails, \command{patch} removes the first and jerojasro@336: last lines of the context, and tries a \emph{reduced} context-only jerojasro@336: match. If the hunk with reduced context succeeds, it prints a message jerojasro@336: saying that it applied the hunk with a \emph{fuzz factor} (the number jerojasro@336: after the fuzz factor indicates how many lines of context jerojasro@336: \command{patch} had to trim before the patch applied). jerojasro@336: jerojasro@336: When neither of these techniques works, \command{patch} prints a jerojasro@336: message saying that the hunk in question was rejected. It saves jerojasro@336: rejected hunks (also simply called ``rejects'') to a file with the jerojasro@336: same name, and an added \sfilename{.rej} extension. It also saves an jerojasro@336: unmodified copy of the file with a \sfilename{.orig} extension; the jerojasro@336: copy of the file without any extensions will contain any changes made jerojasro@336: by hunks that \emph{did} apply cleanly. If you have a patch that jerojasro@336: modifies \filename{foo} with six hunks, and one of them fails to jerojasro@336: apply, you will have: an unmodified \filename{foo.orig}, a jerojasro@336: \filename{foo.rej} containing one hunk, and \filename{foo}, containing jerojasro@336: the changes made by the five successful five hunks. jerojasro@336: jerojasro@336: \subsection{Some quirks of patch representation} jerojasro@336: jerojasro@336: There are a few useful things to know about how \command{patch} works jerojasro@336: with files. jerojasro@336: \begin{itemize} jerojasro@336: \item This should already be obvious, but \command{patch} cannot jerojasro@336: handle binary files. jerojasro@336: \item Neither does it care about the executable bit; it creates new jerojasro@336: files as readable, but not executable. jerojasro@336: \item \command{patch} treats the removal of a file as a diff between jerojasro@336: the file to be removed and the empty file. So your idea of ``I jerojasro@336: deleted this file'' looks like ``every line of this file was jerojasro@336: deleted'' in a patch. jerojasro@336: \item It treats the addition of a file as a diff between the empty jerojasro@336: file and the file to be added. So in a patch, your idea of ``I jerojasro@336: added this file'' looks like ``every line of this file was added''. jerojasro@336: \item It treats a renamed file as the removal of the old name, and the jerojasro@336: addition of the new name. This means that renamed files have a big jerojasro@336: footprint in patches. (Note also that Mercurial does not currently jerojasro@336: try to infer when files have been renamed or copied in a patch.) jerojasro@336: \item \command{patch} cannot represent empty files, so you cannot use jerojasro@336: a patch to represent the notion ``I added this empty file to the jerojasro@336: tree''. jerojasro@336: \end{itemize} jerojasro@336: \subsection{Beware the fuzz} jerojasro@336: jerojasro@336: While applying a hunk at an offset, or with a fuzz factor, will often jerojasro@336: be completely successful, these inexact techniques naturally leave jerojasro@336: open the possibility of corrupting the patched file. The most common jerojasro@336: cases typically involve applying a patch twice, or at an incorrect jerojasro@336: location in the file. If \command{patch} or \hgxcmd{mq}{qpush} ever jerojasro@336: mentions an offset or fuzz factor, you should make sure that the jerojasro@336: modified files are correct afterwards. jerojasro@336: jerojasro@336: It's often a good idea to refresh a patch that has applied with an jerojasro@336: offset or fuzz factor; refreshing the patch generates new context jerojasro@336: information that will make it apply cleanly. I say ``often,'' not jerojasro@336: ``always,'' because sometimes refreshing a patch will make it fail to jerojasro@336: apply against a different revision of the underlying files. In some jerojasro@336: cases, such as when you're maintaining a patch that must sit on top of jerojasro@336: multiple versions of a source tree, it's acceptable to have a patch jerojasro@336: apply with some fuzz, provided you've verified the results of the jerojasro@336: patching process in such cases. jerojasro@336: jerojasro@336: \subsection{Handling rejection} jerojasro@336: jerojasro@336: If \hgxcmd{mq}{qpush} fails to apply a patch, it will print an error jerojasro@336: message and exit. If it has left \sfilename{.rej} files behind, it is jerojasro@336: usually best to fix up the rejected hunks before you push more patches jerojasro@336: or do any further work. jerojasro@336: jerojasro@336: If your patch \emph{used to} apply cleanly, and no longer does because jerojasro@336: you've changed the underlying code that your patches are based on, jerojasro@336: Mercurial Queues can help; see section~\ref{sec:mq:merge} for details. jerojasro@336: jerojasro@336: Unfortunately, there aren't any great techniques for dealing with jerojasro@336: rejected hunks. Most often, you'll need to view the \sfilename{.rej} jerojasro@336: file and edit the target file, applying the rejected hunks by hand. jerojasro@336: jerojasro@336: If you're feeling adventurous, Neil Brown, a Linux kernel hacker, jerojasro@336: wrote a tool called \command{wiggle}~\cite{web:wiggle}, which is more jerojasro@336: vigorous than \command{patch} in its attempts to make a patch apply. jerojasro@336: jerojasro@336: Another Linux kernel hacker, Chris Mason (the author of Mercurial jerojasro@336: Queues), wrote a similar tool called jerojasro@336: \command{mpatch}~\cite{web:mpatch}, which takes a simple approach to jerojasro@336: automating the application of hunks rejected by \command{patch}. The jerojasro@336: \command{mpatch} command can help with four common reasons that a hunk jerojasro@336: may be rejected: jerojasro@336: jerojasro@336: \begin{itemize} jerojasro@336: \item The context in the middle of a hunk has changed. jerojasro@336: \item A hunk is missing some context at the beginning or end. jerojasro@336: \item A large hunk might apply better---either entirely or in jerojasro@336: part---if it was broken up into smaller hunks. jerojasro@336: \item A hunk removes lines with slightly different content than those jerojasro@336: currently present in the file. jerojasro@336: \end{itemize} jerojasro@336: jerojasro@336: If you use \command{wiggle} or \command{mpatch}, you should be doubly jerojasro@336: careful to check your results when you're done. In fact, jerojasro@336: \command{mpatch} enforces this method of double-checking the tool's jerojasro@336: output, by automatically dropping you into a merge program when it has jerojasro@336: done its job, so that you can verify its work and finish off any jerojasro@336: remaining merges. jerojasro@336: jerojasro@336: \section{Getting the best performance out of MQ} jerojasro@336: \label{sec:mq:perf} jerojasro@336: jerojasro@336: MQ is very efficient at handling a large number of patches. I ran jerojasro@336: some performance experiments in mid-2006 for a talk that I gave at the jerojasro@336: 2006 EuroPython conference~\cite{web:europython}. I used as my data jerojasro@336: set the Linux 2.6.17-mm1 patch series, which consists of 1,738 jerojasro@336: patches. I applied these on top of a Linux kernel repository jerojasro@336: containing all 27,472 revisions between Linux 2.6.12-rc2 and Linux jerojasro@336: 2.6.17. jerojasro@336: jerojasro@336: On my old, slow laptop, I was able to jerojasro@336: \hgcmdargs{qpush}{\hgxopt{mq}{qpush}{-a}} all 1,738 patches in 3.5 minutes, jerojasro@336: and \hgcmdargs{qpop}{\hgxopt{mq}{qpop}{-a}} them all in 30 seconds. (On a jerojasro@336: newer laptop, the time to push all patches dropped to two minutes.) I jerojasro@336: could \hgxcmd{mq}{qrefresh} one of the biggest patches (which made 22,779 jerojasro@336: lines of changes to 287 files) in 6.6 seconds. jerojasro@336: jerojasro@336: Clearly, MQ is well suited to working in large trees, but there are a jerojasro@336: few tricks you can use to get the best performance of it. jerojasro@336: jerojasro@336: First of all, try to ``batch'' operations together. Every time you jerojasro@336: run \hgxcmd{mq}{qpush} or \hgxcmd{mq}{qpop}, these commands scan the working jerojasro@336: directory once to make sure you haven't made some changes and then jerojasro@336: forgotten to run \hgxcmd{mq}{qrefresh}. On a small tree, the time that jerojasro@336: this scan takes is unnoticeable. However, on a medium-sized tree jerojasro@336: (containing tens of thousands of files), it can take a second or more. jerojasro@336: jerojasro@336: The \hgxcmd{mq}{qpush} and \hgxcmd{mq}{qpop} commands allow you to push and pop jerojasro@336: multiple patches at a time. You can identify the ``destination jerojasro@336: patch'' that you want to end up at. When you \hgxcmd{mq}{qpush} with a jerojasro@336: destination specified, it will push patches until that patch is at the jerojasro@336: top of the applied stack. When you \hgxcmd{mq}{qpop} to a destination, MQ jerojasro@336: will pop patches until the destination patch is at the top. jerojasro@336: jerojasro@336: You can identify a destination patch using either the name of the jerojasro@336: patch, or by number. If you use numeric addressing, patches are jerojasro@336: counted from zero; this means that the first patch is zero, the second jerojasro@336: is one, and so on. jerojasro@336: jerojasro@336: \section{Updating your patches when the underlying code changes} jerojasro@336: \label{sec:mq:merge} jerojasro@336: jerojasro@336: It's common to have a stack of patches on top of an underlying jerojasro@336: repository that you don't modify directly. If you're working on jerojasro@336: changes to third-party code, or on a feature that is taking longer to jerojasro@336: develop than the rate of change of the code beneath, you will often jerojasro@336: need to sync up with the underlying code, and fix up any hunks in your jerojasro@336: patches that no longer apply. This is called \emph{rebasing} your jerojasro@336: patch series. jerojasro@336: jerojasro@336: The simplest way to do this is to \hgcmdargs{qpop}{\hgxopt{mq}{qpop}{-a}} jerojasro@336: your patches, then \hgcmd{pull} changes into the underlying jerojasro@336: repository, and finally \hgcmdargs{qpush}{\hgxopt{mq}{qpop}{-a}} your jerojasro@336: patches again. MQ will stop pushing any time it runs across a patch jerojasro@336: that fails to apply during conflicts, allowing you to fix your jerojasro@336: conflicts, \hgxcmd{mq}{qrefresh} the affected patch, and continue pushing jerojasro@336: until you have fixed your entire stack. jerojasro@336: jerojasro@336: This approach is easy to use and works well if you don't expect jerojasro@336: changes to the underlying code to affect how well your patches apply. jerojasro@336: If your patch stack touches code that is modified frequently or jerojasro@336: invasively in the underlying repository, however, fixing up rejected jerojasro@336: hunks by hand quickly becomes tiresome. jerojasro@336: jerojasro@336: It's possible to partially automate the rebasing process. If your jerojasro@336: patches apply cleanly against some revision of the underlying repo, MQ jerojasro@336: can use this information to help you to resolve conflicts between your jerojasro@336: patches and a different revision. jerojasro@336: jerojasro@336: The process is a little involved. jerojasro@336: \begin{enumerate} jerojasro@336: \item To begin, \hgcmdargs{qpush}{-a} all of your patches on top of jerojasro@336: the revision where you know that they apply cleanly. jerojasro@336: \item Save a backup copy of your patch directory using jerojasro@336: \hgcmdargs{qsave}{\hgxopt{mq}{qsave}{-e} \hgxopt{mq}{qsave}{-c}}. This prints jerojasro@336: the name of the directory that it has saved the patches in. It will jerojasro@336: save the patches to a directory called jerojasro@336: \sdirname{.hg/patches.\emph{N}}, where \texttt{\emph{N}} is a small jerojasro@336: integer. It also commits a ``save changeset'' on top of your jerojasro@336: applied patches; this is for internal book-keeping, and records the jerojasro@336: states of the \sfilename{series} and \sfilename{status} files. jerojasro@336: \item Use \hgcmd{pull} to bring new changes into the underlying jerojasro@336: repository. (Don't run \hgcmdargs{pull}{-u}; see below for why.) jerojasro@336: \item Update to the new tip revision, using jerojasro@336: \hgcmdargs{update}{\hgopt{update}{-C}} to override the patches you jerojasro@336: have pushed. jerojasro@336: \item Merge all patches using \hgcmdargs{qpush}{\hgxopt{mq}{qpush}{-m} jerojasro@336: \hgxopt{mq}{qpush}{-a}}. The \hgxopt{mq}{qpush}{-m} option to \hgxcmd{mq}{qpush} jerojasro@336: tells MQ to perform a three-way merge if the patch fails to apply. jerojasro@336: \end{enumerate} jerojasro@336: jerojasro@336: During the \hgcmdargs{qpush}{\hgxopt{mq}{qpush}{-m}}, each patch in the jerojasro@336: \sfilename{series} file is applied normally. If a patch applies with jerojasro@336: fuzz or rejects, MQ looks at the queue you \hgxcmd{mq}{qsave}d, and jerojasro@336: performs a three-way merge with the corresponding changeset. This jerojasro@336: merge uses Mercurial's normal merge machinery, so it may pop up a GUI jerojasro@336: merge tool to help you to resolve problems. jerojasro@336: jerojasro@336: When you finish resolving the effects of a patch, MQ refreshes your jerojasro@336: patch based on the result of the merge. jerojasro@336: jerojasro@336: At the end of this process, your repository will have one extra head jerojasro@336: from the old patch queue, and a copy of the old patch queue will be in jerojasro@336: \sdirname{.hg/patches.\emph{N}}. You can remove the extra head using jerojasro@336: \hgcmdargs{qpop}{\hgxopt{mq}{qpop}{-a} \hgxopt{mq}{qpop}{-n} patches.\emph{N}} jerojasro@336: or \hgcmd{strip}. You can delete \sdirname{.hg/patches.\emph{N}} once jerojasro@336: you are sure that you no longer need it as a backup. jerojasro@336: jerojasro@336: \section{Identifying patches} jerojasro@336: jerojasro@336: MQ commands that work with patches let you refer to a patch either by jerojasro@336: using its name or by a number. By name is obvious enough; pass the jerojasro@336: name \filename{foo.patch} to \hgxcmd{mq}{qpush}, for example, and it will jerojasro@336: push patches until \filename{foo.patch} is applied. jerojasro@336: jerojasro@336: As a shortcut, you can refer to a patch using both a name and a jerojasro@336: numeric offset; \texttt{foo.patch-2} means ``two patches before jerojasro@336: \texttt{foo.patch}'', while \texttt{bar.patch+4} means ``four patches jerojasro@336: after \texttt{bar.patch}''. jerojasro@336: jerojasro@336: Referring to a patch by index isn't much different. The first patch jerojasro@336: printed in the output of \hgxcmd{mq}{qseries} is patch zero (yes, it's one jerojasro@336: of those start-at-zero counting systems); the second is patch one; and jerojasro@336: so on. jerojasro@336: jerojasro@336: MQ also makes it easy to work with patches when you are using normal jerojasro@336: Mercurial commands. Every command that accepts a changeset ID will jerojasro@336: also accept the name of an applied patch. MQ augments the tags jerojasro@336: normally in the repository with an eponymous one for each applied jerojasro@336: patch. In addition, the special tags \index{tags!special tag jerojasro@336: names!\texttt{qbase}}\texttt{qbase} and \index{tags!special tag jerojasro@336: names!\texttt{qtip}}\texttt{qtip} identify the ``bottom-most'' and jerojasro@336: topmost applied patches, respectively. jerojasro@336: jerojasro@336: These additions to Mercurial's normal tagging capabilities make jerojasro@336: dealing with patches even more of a breeze. jerojasro@336: \begin{itemize} jerojasro@336: \item Want to patchbomb a mailing list with your latest series of jerojasro@336: changes? jerojasro@336: \begin{codesample4} jerojasro@336: hg email qbase:qtip jerojasro@336: \end{codesample4} jerojasro@336: (Don't know what ``patchbombing'' is? See jerojasro@336: section~\ref{sec:hgext:patchbomb}.) jerojasro@336: \item Need to see all of the patches since \texttt{foo.patch} that jerojasro@336: have touched files in a subdirectory of your tree? jerojasro@336: \begin{codesample4} jerojasro@336: hg log -r foo.patch:qtip \emph{subdir} jerojasro@336: \end{codesample4} jerojasro@336: \end{itemize} jerojasro@336: jerojasro@336: Because MQ makes the names of patches available to the rest of jerojasro@336: Mercurial through its normal internal tag machinery, you don't need to jerojasro@336: type in the entire name of a patch when you want to identify it by jerojasro@336: name. jerojasro@336: jerojasro@336: \begin{figure}[ht] jerojasro@336: \interaction{mq.id.output} jerojasro@336: \caption{Using MQ's tag features to work with patches} jerojasro@336: \label{ex:mq:id} jerojasro@336: \end{figure} jerojasro@336: jerojasro@336: Another nice consequence of representing patch names as tags is that jerojasro@336: when you run the \hgcmd{log} command, it will display a patch's name jerojasro@336: as a tag, simply as part of its normal output. This makes it easy to jerojasro@336: visually distinguish applied patches from underlying ``normal'' jerojasro@336: revisions. Figure~\ref{ex:mq:id} shows a few normal Mercurial jerojasro@336: commands in use with applied patches. jerojasro@336: jerojasro@336: \section{Useful things to know about} jerojasro@336: jerojasro@336: There are a number of aspects of MQ usage that don't fit tidily into jerojasro@336: sections of their own, but that are good to know. Here they are, in jerojasro@336: one place. jerojasro@336: jerojasro@336: \begin{itemize} jerojasro@336: \item Normally, when you \hgxcmd{mq}{qpop} a patch and \hgxcmd{mq}{qpush} it jerojasro@336: again, the changeset that represents the patch after the pop/push jerojasro@336: will have a \emph{different identity} than the changeset that jerojasro@336: represented the hash beforehand. See jerojasro@336: section~\ref{sec:mqref:cmd:qpush} for information as to why this is. jerojasro@336: \item It's not a good idea to \hgcmd{merge} changes from another jerojasro@336: branch with a patch changeset, at least if you want to maintain the jerojasro@336: ``patchiness'' of that changeset and changesets below it on the jerojasro@336: patch stack. If you try to do this, it will appear to succeed, but jerojasro@336: MQ will become confused. jerojasro@336: \end{itemize} jerojasro@336: jerojasro@336: \section{Managing patches in a repository} jerojasro@336: \label{sec:mq:repo} jerojasro@336: jerojasro@336: Because MQ's \sdirname{.hg/patches} directory resides outside a jerojasro@336: Mercurial repository's working directory, the ``underlying'' Mercurial jerojasro@336: repository knows nothing about the management or presence of patches. jerojasro@336: jerojasro@336: This presents the interesting possibility of managing the contents of jerojasro@336: the patch directory as a Mercurial repository in its own right. This jerojasro@336: can be a useful way to work. For example, you can work on a patch for jerojasro@336: a while, \hgxcmd{mq}{qrefresh} it, then \hgcmd{commit} the current state of jerojasro@336: the patch. This lets you ``roll back'' to that version of the patch jerojasro@336: later on. jerojasro@336: jerojasro@336: You can then share different versions of the same patch stack among jerojasro@336: multiple underlying repositories. I use this when I am developing a jerojasro@336: Linux kernel feature. I have a pristine copy of my kernel sources for jerojasro@336: each of several CPU architectures, and a cloned repository under each jerojasro@336: that contains the patches I am working on. When I want to test a jerojasro@336: change on a different architecture, I push my current patches to the jerojasro@336: patch repository associated with that kernel tree, pop and push all of jerojasro@336: my patches, and build and test that kernel. jerojasro@336: jerojasro@336: Managing patches in a repository makes it possible for multiple jerojasro@336: developers to work on the same patch series without colliding with jerojasro@336: each other, all on top of an underlying source base that they may or jerojasro@336: may not control. jerojasro@336: jerojasro@336: \subsection{MQ support for patch repositories} jerojasro@336: jerojasro@336: MQ helps you to work with the \sdirname{.hg/patches} directory as a jerojasro@336: repository; when you prepare a repository for working with patches jerojasro@336: using \hgxcmd{mq}{qinit}, you can pass the \hgxopt{mq}{qinit}{-c} option to jerojasro@336: create the \sdirname{.hg/patches} directory as a Mercurial repository. jerojasro@336: jerojasro@336: \begin{note} jerojasro@336: If you forget to use the \hgxopt{mq}{qinit}{-c} option, you can simply go jerojasro@336: into the \sdirname{.hg/patches} directory at any time and run jerojasro@336: \hgcmd{init}. Don't forget to add an entry for the jerojasro@336: \sfilename{status} file to the \sfilename{.hgignore} file, though jerojasro@336: jerojasro@336: (\hgcmdargs{qinit}{\hgxopt{mq}{qinit}{-c}} does this for you jerojasro@336: automatically); you \emph{really} don't want to manage the jerojasro@336: \sfilename{status} file. jerojasro@336: \end{note} jerojasro@336: jerojasro@336: As a convenience, if MQ notices that the \dirname{.hg/patches} jerojasro@336: directory is a repository, it will automatically \hgcmd{add} every jerojasro@336: patch that you create and import. jerojasro@336: jerojasro@336: MQ provides a shortcut command, \hgxcmd{mq}{qcommit}, that runs jerojasro@336: \hgcmd{commit} in the \sdirname{.hg/patches} directory. This saves jerojasro@336: some bothersome typing. jerojasro@336: jerojasro@336: Finally, as a convenience to manage the patch directory, you can jerojasro@336: define the alias \command{mq} on Unix systems. For example, on Linux jerojasro@336: systems using the \command{bash} shell, you can include the following jerojasro@336: snippet in your \tildefile{.bashrc}. jerojasro@336: jerojasro@336: \begin{codesample2} jerojasro@336: alias mq=`hg -R \$(hg root)/.hg/patches' jerojasro@336: \end{codesample2} jerojasro@336: jerojasro@336: You can then issue commands of the form \cmdargs{mq}{pull} from jerojasro@336: the main repository. jerojasro@336: jerojasro@336: \subsection{A few things to watch out for} jerojasro@336: jerojasro@336: MQ's support for working with a repository full of patches is limited jerojasro@336: in a few small respects. jerojasro@336: jerojasro@336: MQ cannot automatically detect changes that you make to the patch jerojasro@336: directory. If you \hgcmd{pull}, manually edit, or \hgcmd{update} jerojasro@336: changes to patches or the \sfilename{series} file, you will have to jerojasro@336: \hgcmdargs{qpop}{\hgxopt{mq}{qpop}{-a}} and then jerojasro@336: \hgcmdargs{qpush}{\hgxopt{mq}{qpush}{-a}} in the underlying repository to jerojasro@336: see those changes show up there. If you forget to do this, you can jerojasro@336: confuse MQ's idea of which patches are applied. jerojasro@336: jerojasro@336: \section{Third party tools for working with patches} jerojasro@336: \label{sec:mq:tools} jerojasro@336: jerojasro@336: Once you've been working with patches for a while, you'll find jerojasro@336: yourself hungry for tools that will help you to understand and jerojasro@336: manipulate the patches you're dealing with. jerojasro@336: jerojasro@336: The \command{diffstat} command~\cite{web:diffstat} generates a jerojasro@336: histogram of the modifications made to each file in a patch. It jerojasro@336: provides a good way to ``get a sense of'' a patch---which files it jerojasro@336: affects, and how much change it introduces to each file and as a jerojasro@336: whole. (I find that it's a good idea to use \command{diffstat}'s jerojasro@336: \cmdopt{diffstat}{-p} option as a matter of course, as otherwise it jerojasro@336: will try to do clever things with prefixes of file names that jerojasro@336: inevitably confuse at least me.) jerojasro@336: jerojasro@336: \begin{figure}[ht] jerojasro@336: \interaction{mq.tools.tools} jerojasro@336: \caption{The \command{diffstat}, \command{filterdiff}, and \command{lsdiff} commands} jerojasro@336: \label{ex:mq:tools} jerojasro@336: \end{figure} jerojasro@336: jerojasro@336: The \package{patchutils} package~\cite{web:patchutils} is invaluable. jerojasro@336: It provides a set of small utilities that follow the ``Unix jerojasro@336: philosophy;'' each does one useful thing with a patch. The jerojasro@336: \package{patchutils} command I use most is \command{filterdiff}, which jerojasro@336: extracts subsets from a patch file. For example, given a patch that jerojasro@336: modifies hundreds of files across dozens of directories, a single jerojasro@336: invocation of \command{filterdiff} can generate a smaller patch that jerojasro@336: only touches files whose names match a particular glob pattern. See jerojasro@336: section~\ref{mq-collab:tips:interdiff} for another example. jerojasro@336: jerojasro@336: \section{Good ways to work with patches} jerojasro@336: jerojasro@336: Whether you are working on a patch series to submit to a free software jerojasro@336: or open source project, or a series that you intend to treat as a jerojasro@336: sequence of regular changesets when you're done, you can use some jerojasro@336: simple techniques to keep your work well organised. jerojasro@336: jerojasro@336: Give your patches descriptive names. A good name for a patch might be jerojasro@336: \filename{rework-device-alloc.patch}, because it will immediately give jerojasro@336: you a hint what the purpose of the patch is. Long names shouldn't be jerojasro@336: a problem; you won't be typing the names often, but you \emph{will} be jerojasro@336: running commands like \hgxcmd{mq}{qapplied} and \hgxcmd{mq}{qtop} over and over. jerojasro@336: Good naming becomes especially important when you have a number of jerojasro@336: patches to work with, or if you are juggling a number of different jerojasro@336: tasks and your patches only get a fraction of your attention. jerojasro@336: jerojasro@336: Be aware of what patch you're working on. Use the \hgxcmd{mq}{qtop} jerojasro@336: command and skim over the text of your patches frequently---for jerojasro@336: example, using \hgcmdargs{tip}{\hgopt{tip}{-p}})---to be sure of where jerojasro@336: you stand. I have several times worked on and \hgxcmd{mq}{qrefresh}ed a jerojasro@336: patch other than the one I intended, and it's often tricky to migrate jerojasro@336: changes into the right patch after making them in the wrong one. jerojasro@336: jerojasro@336: For this reason, it is very much worth investing a little time to jerojasro@336: learn how to use some of the third-party tools I described in jerojasro@336: section~\ref{sec:mq:tools}, particularly \command{diffstat} and jerojasro@336: \command{filterdiff}. The former will give you a quick idea of what jerojasro@336: changes your patch is making, while the latter makes it easy to splice jerojasro@336: hunks selectively out of one patch and into another. jerojasro@336: jerojasro@336: \section{MQ cookbook} jerojasro@336: jerojasro@336: \subsection{Manage ``trivial'' patches} jerojasro@336: jerojasro@336: Because the overhead of dropping files into a new Mercurial repository jerojasro@336: is so low, it makes a lot of sense to manage patches this way even if jerojasro@336: you simply want to make a few changes to a source tarball that you jerojasro@336: downloaded. jerojasro@336: jerojasro@336: Begin by downloading and unpacking the source tarball, jerojasro@336: and turning it into a Mercurial repository. jerojasro@336: \interaction{mq.tarball.download} jerojasro@336: jerojasro@336: Continue by creating a patch stack and making your changes. jerojasro@336: \interaction{mq.tarball.qinit} jerojasro@336: jerojasro@336: Let's say a few weeks or months pass, and your package author releases jerojasro@336: a new version. First, bring their changes into the repository. jerojasro@336: \interaction{mq.tarball.newsource} jerojasro@336: The pipeline starting with \hgcmd{locate} above deletes all files in jerojasro@336: the working directory, so that \hgcmd{commit}'s jerojasro@336: \hgopt{commit}{--addremove} option can actually tell which files have jerojasro@336: really been removed in the newer version of the source. jerojasro@336: jerojasro@336: Finally, you can apply your patches on top of the new tree. jerojasro@336: \interaction{mq.tarball.repush} jerojasro@336: jerojasro@336: \subsection{Combining entire patches} jerojasro@336: \label{sec:mq:combine} jerojasro@336: jerojasro@336: MQ provides a command, \hgxcmd{mq}{qfold} that lets you combine entire jerojasro@336: patches. This ``folds'' the patches you name, in the order you name jerojasro@336: them, into the topmost applied patch, and concatenates their jerojasro@336: descriptions onto the end of its description. The patches that you jerojasro@336: fold must be unapplied before you fold them. jerojasro@336: jerojasro@336: The order in which you fold patches matters. If your topmost applied jerojasro@336: patch is \texttt{foo}, and you \hgxcmd{mq}{qfold} \texttt{bar} and jerojasro@336: \texttt{quux} into it, you will end up with a patch that has the same jerojasro@336: effect as if you applied first \texttt{foo}, then \texttt{bar}, jerojasro@336: followed by \texttt{quux}. jerojasro@336: jerojasro@336: \subsection{Merging part of one patch into another} jerojasro@336: jerojasro@336: Merging \emph{part} of one patch into another is more difficult than jerojasro@336: combining entire patches. jerojasro@336: jerojasro@336: If you want to move changes to entire files, you can use jerojasro@336: \command{filterdiff}'s \cmdopt{filterdiff}{-i} and jerojasro@336: \cmdopt{filterdiff}{-x} options to choose the modifications to snip jerojasro@336: out of one patch, concatenating its output onto the end of the patch jerojasro@336: you want to merge into. You usually won't need to modify the patch jerojasro@336: you've merged the changes from. Instead, MQ will report some rejected jerojasro@336: hunks when you \hgxcmd{mq}{qpush} it (from the hunks you moved into the jerojasro@336: other patch), and you can simply \hgxcmd{mq}{qrefresh} the patch to drop jerojasro@336: the duplicate hunks. jerojasro@336: jerojasro@336: If you have a patch that has multiple hunks modifying a file, and you jerojasro@336: only want to move a few of those hunks, the job becomes more messy, jerojasro@336: but you can still partly automate it. Use \cmdargs{lsdiff}{-nvv} to jerojasro@336: print some metadata about the patch. jerojasro@336: \interaction{mq.tools.lsdiff} jerojasro@336: jerojasro@336: This command prints three different kinds of number: jerojasro@336: \begin{itemize} jerojasro@336: \item (in the first column) a \emph{file number} to identify each file jerojasro@336: modified in the patch; jerojasro@336: \item (on the next line, indented) the line number within a modified jerojasro@336: file where a hunk starts; and jerojasro@336: \item (on the same line) a \emph{hunk number} to identify that hunk. jerojasro@336: \end{itemize} jerojasro@336: jerojasro@336: You'll have to use some visual inspection, and reading of the patch, jerojasro@336: to identify the file and hunk numbers you'll want, but you can then jerojasro@336: pass them to to \command{filterdiff}'s \cmdopt{filterdiff}{--files} jerojasro@336: and \cmdopt{filterdiff}{--hunks} options, to select exactly the file jerojasro@336: and hunk you want to extract. jerojasro@336: jerojasro@336: Once you have this hunk, you can concatenate it onto the end of your jerojasro@336: destination patch and continue with the remainder of jerojasro@336: section~\ref{sec:mq:combine}. jerojasro@336: jerojasro@336: \section{Differences between quilt and MQ} jerojasro@336: jerojasro@336: If you are already familiar with quilt, MQ provides a similar command jerojasro@336: set. There are a few differences in the way that it works. jerojasro@336: jerojasro@336: You will already have noticed that most quilt commands have MQ jerojasro@336: counterparts that simply begin with a ``\texttt{q}''. The exceptions jerojasro@336: are quilt's \texttt{add} and \texttt{remove} commands, the jerojasro@336: counterparts for which are the normal Mercurial \hgcmd{add} and jerojasro@336: \hgcmd{remove} commands. There is no MQ equivalent of the quilt jerojasro@336: \texttt{edit} command. jerojasro@336: jerojasro@336: %%% Local Variables: jerojasro@336: %%% mode: latex jerojasro@336: %%% TeX-master: "00book" jerojasro@336: %%% End: