| rev |
line source |
|
bos@1
|
1 \chapter{Managing change with Mercurial Queues}
|
|
bos@1
|
2 \label{chap:mq}
|
|
bos@1
|
3
|
|
bos@1
|
4 \section{The patch management problem}
|
|
bos@1
|
5 \label{sec:mq:patch-mgmt}
|
|
bos@1
|
6
|
|
bos@1
|
7 Here is a common scenario: you need to install a software package from
|
|
bos@1
|
8 source, but you find a bug that you must fix in the source before you
|
|
bos@1
|
9 can start using the package. You make your changes, forget about the
|
|
bos@1
|
10 package for a while, and a few months later you need to upgrade to a
|
|
bos@1
|
11 newer version of the package. If the newer version of the package
|
|
bos@1
|
12 still has the bug, you must extract your fix from the older source
|
|
bos@1
|
13 tree and apply it against the newer version. This is a tedious task,
|
|
bos@1
|
14 and it's easy to make mistakes.
|
|
bos@1
|
15
|
|
bos@1
|
16 This is a simple case of the ``patch management'' problem. You have
|
|
bos@1
|
17 an ``upstream'' source tree that you can't change; you need to make
|
|
bos@1
|
18 some local changes on top of the upstream tree; and you'd like to be
|
|
bos@1
|
19 able to keep those changes separate, so that you can apply them to
|
|
bos@1
|
20 newer versions of the upstream source.
|
|
bos@1
|
21
|
|
bos@1
|
22 The patch management problem arises in many situations. Probably the
|
|
bos@1
|
23 most visible is that a user of an open source software project will
|
|
bos@3
|
24 contribute a bug fix or new feature to the project's maintainers in the
|
|
bos@1
|
25 form of a patch.
|
|
bos@1
|
26
|
|
bos@1
|
27 Distributors of operating systems that include open source software
|
|
bos@1
|
28 often need to make changes to the packages they distribute so that
|
|
bos@1
|
29 they will build properly in their environments.
|
|
bos@1
|
30
|
|
bos@1
|
31 When you have few changes to maintain, it is easy to manage a single
|
|
bos@1
|
32 patch using the standard \texttt{diff} and \texttt{patch} programs.
|
|
bos@1
|
33 Once the number of changes grows, it starts to makes sense to maintain
|
|
bos@1
|
34 patches as discrete ``chunks of work,'' so that for example a single
|
|
bos@1
|
35 patch will contain only one bug fix (the patch might modify several
|
|
bos@1
|
36 files, but it's doing ``only one thing''), and you may have a number
|
|
bos@1
|
37 of such patches for different bugs you need fixed and local changes
|
|
bos@3
|
38 you require. In this situation, if you submit a bug fix patch to the
|
|
bos@1
|
39 upstream maintainers of a package and they include your fix in a
|
|
bos@1
|
40 subsequent release, you can simply drop that single patch when you're
|
|
bos@1
|
41 updating to the newer release.
|
|
bos@1
|
42
|
|
bos@1
|
43 Maintaining a single patch against an upstream tree is a little
|
|
bos@1
|
44 tedious and error-prone, but not difficult. However, the complexity
|
|
bos@1
|
45 of the problem grows rapidly as the number of patches you have to
|
|
bos@1
|
46 maintain increases. With more than a tiny number of patches in hand,
|
|
bos@1
|
47 understanding which ones you have applied and maintaining them moves
|
|
bos@1
|
48 from messy to overwhelming.
|
|
bos@1
|
49
|
|
bos@1
|
50 Fortunately, Mercurial includes a powerful extension, Mercurial Queues
|
|
bos@1
|
51 (or simply ``MQ''), that massively simplifies the patch management
|
|
bos@1
|
52 problem.
|
|
bos@1
|
53
|
|
bos@1
|
54 \section{The prehistory of Mercurial Queues}
|
|
bos@1
|
55 \label{sec:mq:history}
|
|
bos@1
|
56
|
|
bos@1
|
57 During the late 1990s, several Linux kernel developers started to
|
|
bos@1
|
58 maintain ``patch series'' that modified the behaviour of the Linux
|
|
bos@1
|
59 kernel. Some of these series were focused on stability, some on
|
|
bos@1
|
60 feature coverage, and others were more speculative.
|
|
bos@1
|
61
|
|
bos@1
|
62 The sizes of these patch series grew rapidly. In 2002, Andrew Morton
|
|
bos@1
|
63 published some shell scripts he had been using to automate the task of
|
|
bos@1
|
64 managing his patch queues. Andrew was successfully using these
|
|
bos@1
|
65 scripts to manage hundreds (sometimes thousands) of patches on top of
|
|
bos@1
|
66 the Linux kernel.
|
|
bos@1
|
67
|
|
bos@1
|
68 \subsection{A patchwork quilt}
|
|
bos@1
|
69 \label{sec:mq:quilt}
|
|
bos@1
|
70
|
|
bos@1
|
71
|
|
bos@1
|
72 In early 2003, Andreas Gruenbacher and Martin Quinson borrowed the
|
|
bos@2
|
73 approach of Andrew's scripts and published a tool called ``patchwork
|
|
bos@2
|
74 quilt''~\cite{web:quilt}, or simply ``quilt''
|
|
bos@2
|
75 (see~\cite{gruenbacher:2005} for a paper describing it). Because
|
|
bos@2
|
76 quilt substantially automated patch management, it rapidly gained a
|
|
bos@2
|
77 large following among open source software developers.
|
|
bos@1
|
78
|
|
bos@1
|
79 Quilt manages a \emph{stack of patches} on top of a directory tree.
|
|
bos@1
|
80 To begin, you tell quilt to manage a directory tree; it stores away
|
|
bos@1
|
81 the names and contents of all files in the tree. To fix a bug, you
|
|
bos@1
|
82 create a new patch (using a single command), edit the files you need
|
|
bos@1
|
83 to fix, then ``refresh'' the patch.
|
|
bos@1
|
84
|
|
bos@1
|
85 The refresh step causes quilt to scan the directory tree; it updates
|
|
bos@1
|
86 the patch with all of the changes you have made. You can create
|
|
bos@1
|
87 another patch on top of the first, which will track the changes
|
|
bos@1
|
88 required to modify the tree from ``tree with one patch applied'' to
|
|
bos@1
|
89 ``tree with two patches applied''.
|
|
bos@1
|
90
|
|
bos@1
|
91 You can \emph{change} which patches are applied to the tree. If you
|
|
bos@1
|
92 ``pop'' a patch, the changes made by that patch will vanish from the
|
|
bos@1
|
93 directory tree. Quilt remembers which patches you have popped,
|
|
bos@1
|
94 though, so you can ``push'' a popped patch again, and the directory
|
|
bos@1
|
95 tree will be restored to contain the modifications in the patch. Most
|
|
bos@1
|
96 importantly, you can run the ``refresh'' command at any time, and the
|
|
bos@1
|
97 topmost applied patch will be updated. This means that you can, at
|
|
bos@1
|
98 any time, change both which patches are applied and what
|
|
bos@1
|
99 modifications those patches make.
|
|
bos@1
|
100
|
|
bos@1
|
101 Quilt knows nothing about revision control tools, so it works equally
|
|
bos@3
|
102 well on top of an unpacked tarball or a Subversion repository.
|
|
bos@1
|
103
|
|
bos@1
|
104 \subsection{From patchwork quilt to Mercurial Queues}
|
|
bos@1
|
105 \label{sec:mq:quilt-mq}
|
|
bos@1
|
106
|
|
bos@1
|
107 In mid-2005, Chris Mason took the features of quilt and wrote an
|
|
bos@1
|
108 extension that he called Mercurial Queues, which added quilt-like
|
|
bos@1
|
109 behaviour to Mercurial.
|
|
bos@1
|
110
|
|
bos@1
|
111 The key difference between quilt and MQ is that quilt knows nothing
|
|
bos@1
|
112 about revision control systems, while MQ is \emph{integrated} into
|
|
bos@1
|
113 Mercurial. Each patch that you push is represented as a Mercurial
|
|
bos@1
|
114 changeset. Pop a patch, and the changeset goes away.
|
|
bos@1
|
115
|
|
bos@1
|
116 This integration makes understanding patches and debugging their
|
|
bos@1
|
117 effects \emph{enormously} easier. Since every applied patch has an
|
|
bos@1
|
118 associated changeset, you can use \hgcmdargs{log}{\emph{filename}} to
|
|
bos@1
|
119 see which changesets and patches affected a file. You can use the
|
|
bos@1
|
120 \hgext{bisect} extension to binary-search through all changesets and
|
|
bos@1
|
121 applied patches to see where a bug got introduced or fixed. You can
|
|
bos@1
|
122 use the \hgcmd{annotate} command to see which changeset or patch
|
|
bos@1
|
123 modified a particular line of a source file. And so on.
|
|
bos@1
|
124
|
|
bos@1
|
125 Because quilt does not care about revision control tools, it is still
|
|
bos@1
|
126 a tremendously useful piece of software to know about for situations
|
|
bos@1
|
127 where you cannot use Mercurial and MQ.
|
|
bos@2
|
128 \section{Getting started with Mercurial Queues}
|
|
bos@2
|
129 \label{sec:mq:start}
|
|
bos@1
|
130
|
|
bos@3
|
131 Because MQ is implemented as an extension, you must explicitly enable
|
|
bos@3
|
132 before you can use it. (You don't need to download anything; MQ ships
|
|
bos@3
|
133 with the standard Mercurial distribution.) To enable MQ, edit your
|
|
bos@4
|
134 \tildefile{.hgrc} file, and add the lines in figure~\ref{ex:mq:config}.
|
|
bos@2
|
135
|
|
bos@12
|
136 \begin{figure}[ht]
|
|
bos@4
|
137 \begin{codesample4}
|
|
bos@4
|
138 [extensions]
|
|
bos@4
|
139 hgext.mq =
|
|
bos@4
|
140 \end{codesample4}
|
|
bos@4
|
141 \label{ex:mq:config}
|
|
bos@4
|
142 \caption{Contents to add to \tildefile{.hgrc} to enable the MQ extension}
|
|
bos@4
|
143 \end{figure}
|
|
bos@3
|
144
|
|
bos@3
|
145 Once the extension is enabled, it will make a number of new commands
|
|
bos@7
|
146 available. To verify that the extension is working, you can use
|
|
bos@7
|
147 \hgcmd{help} to see if the \hgcmd{qinit} command is now available; see
|
|
bos@7
|
148 the example in figure~\ref{ex:mq:enabled}.
|
|
bos@3
|
149
|
|
bos@12
|
150 \begin{figure}[ht]
|
|
bos@4
|
151 \interaction{mq.qinit-help.help}
|
|
bos@4
|
152 \caption{How to verify that MQ is enabled}
|
|
bos@4
|
153 \label{ex:mq:enabled}
|
|
bos@4
|
154 \end{figure}
|
|
bos@1
|
155
|
|
bos@8
|
156 You can use MQ with \emph{any} Mercurial repository, and its commands
|
|
bos@8
|
157 only operate within that repository. To get started, simply prepare
|
|
bos@8
|
158 the repository using the \hgcmd{qinit} command (see
|
|
bos@7
|
159 figure~\ref{ex:mq:qinit}). This command creates an empty directory
|
|
bos@7
|
160 called \filename{.hg/patches}, where MQ will keep its metadata. As
|
|
bos@7
|
161 with many Mercurial commands, the \hgcmd{qinit} command prints nothing
|
|
bos@7
|
162 if it succeeds.
|
|
bos@7
|
163
|
|
bos@12
|
164 \begin{figure}[ht]
|
|
bos@7
|
165 \interaction{mq.tutorial.qinit}
|
|
bos@7
|
166 \caption{Preparing a repository for use with MQ}
|
|
bos@7
|
167 \label{ex:mq:qinit}
|
|
bos@7
|
168 \end{figure}
|
|
bos@7
|
169
|
|
bos@12
|
170 \begin{figure}[ht]
|
|
bos@7
|
171 \interaction{mq.tutorial.qnew}
|
|
bos@7
|
172 \caption{Creating a new patch}
|
|
bos@7
|
173 \label{ex:mq:qnew}
|
|
bos@7
|
174 \end{figure}
|
|
bos@7
|
175
|
|
bos@8
|
176 \subsection{Creating a new patch}
|
|
bos@8
|
177
|
|
bos@8
|
178 To begin work on a new patch, use the \hgcmd{qnew} command. This
|
|
bos@7
|
179 command takes one argument, the name of the patch to create. MQ will
|
|
bos@7
|
180 use this as the name of an actual file in the \filename{.hg/patches}
|
|
bos@7
|
181 directory, as you can see in figure~\ref{ex:mq:qnew}.
|
|
bos@7
|
182
|
|
bos@8
|
183 Also newly present in the \filename{.hg/patches} directory are two
|
|
bos@8
|
184 other files, \filename{series} and \filename{status}. The
|
|
bos@8
|
185 \filename{series} file lists all of the patches that MQ knows about
|
|
bos@8
|
186 for this repository, with one patch per line. Mercurial uses the
|
|
bos@8
|
187 \filename{status} file for internal book-keeping; it tracks all of the
|
|
bos@7
|
188 patches that MQ has \emph{applied} in this repository.
|
|
bos@7
|
189
|
|
bos@7
|
190 \begin{note}
|
|
bos@7
|
191 You may sometimes want to edit the \filename{series} file by hand;
|
|
bos@7
|
192 for example, to change the sequence in which some patches are
|
|
bos@7
|
193 applied. However, manually editing the \filename{status} file is
|
|
bos@7
|
194 almost always a bad idea, as it's easy to corrupt MQ's idea of what
|
|
bos@7
|
195 is happening.
|
|
bos@7
|
196 \end{note}
|
|
bos@7
|
197
|
|
bos@8
|
198 Once you have created your new patch, you can edit files in the
|
|
bos@8
|
199 working directory as you usually would. All of the normal Mercurial
|
|
bos@8
|
200 commands, such as \hgcmd{diff} and \hgcmd{annotate}, work exactly as
|
|
bos@8
|
201 they did before.
|
|
bos@8
|
202 \subsection{Refreshing a patch}
|
|
bos@8
|
203
|
|
bos@8
|
204 When you reach a point where you want to save your work, use the
|
|
bos@8
|
205 \hgcmd{qrefresh} command (figure~\ref{ex:mq:qnew}) to update the patch
|
|
bos@8
|
206 you are working on. This command folds the changes you have made in
|
|
bos@8
|
207 the working directory into your patch, and updates its corresponding
|
|
bos@8
|
208 changeset to contain those changes.
|
|
bos@8
|
209
|
|
bos@12
|
210 \begin{figure}[ht]
|
|
bos@8
|
211 \interaction{mq.tutorial.qrefresh}
|
|
bos@8
|
212 \caption{Refreshing a patch}
|
|
bos@8
|
213 \label{ex:mq:qrefresh}
|
|
bos@8
|
214 \end{figure}
|
|
bos@8
|
215
|
|
bos@8
|
216 You can run \hgcmd{qrefresh} as often as you like, so it's a good way
|
|
bos@13
|
217 to ``checkpoint'' your work. Refresh your patch at an opportune
|
|
bos@8
|
218 time; try an experiment; and if the experiment doesn't work out,
|
|
bos@8
|
219 \hgcmd{revert} your modifications back to the last time you refreshed.
|
|
bos@8
|
220
|
|
bos@12
|
221 \begin{figure}[ht]
|
|
bos@8
|
222 \interaction{mq.tutorial.qrefresh2}
|
|
bos@8
|
223 \caption{Refresh a patch many times to accumulate changes}
|
|
bos@8
|
224 \label{ex:mq:qrefresh2}
|
|
bos@8
|
225 \end{figure}
|
|
bos@8
|
226
|
|
bos@8
|
227 \subsection{Stacking and tracking patches}
|
|
bos@8
|
228
|
|
bos@8
|
229 Once you have finished working on a patch, or need to work on another,
|
|
bos@8
|
230 you can use the \hgcmd{qnew} command again to create a new patch.
|
|
bos@8
|
231 Mercurial will apply this patch on top of your existing patch. See
|
|
bos@8
|
232 figure~\ref{ex:mq:qnew2} for an example. Notice that the patch
|
|
bos@8
|
233 contains the changes in our prior patch as part of its context (you
|
|
bos@8
|
234 can see this more clearly in the output of \hgcmd{annotate}).
|
|
bos@8
|
235
|
|
bos@12
|
236 \begin{figure}[ht]
|
|
bos@8
|
237 \interaction{mq.tutorial.qnew2}
|
|
bos@8
|
238 \caption{Stacking a second patch on top of the first}
|
|
bos@8
|
239 \label{ex:mq:qnew2}
|
|
bos@8
|
240 \end{figure}
|
|
bos@8
|
241
|
|
bos@8
|
242 So far, with the exception of \hgcmd{qnew} and \hgcmd{qrefresh}, we've
|
|
bos@8
|
243 been careful to only use regular Mercurial commands. However, there
|
|
bos@8
|
244 are more ``natural'' commands you can use when thinking about patches
|
|
bos@8
|
245 with MQ, as illustrated in figure~\ref{ex:mq:qseries}:
|
|
bos@8
|
246
|
|
bos@8
|
247 \begin{itemize}
|
|
bos@8
|
248 \item The \hgcmd{qseries} command lists every patch that MQ knows
|
|
bos@8
|
249 about in this repository, from oldest to newest (most recently
|
|
bos@8
|
250 \emph{created}).
|
|
bos@8
|
251 \item The \hgcmd{qapplied} command lists every patch that MQ has
|
|
bos@8
|
252 \emph{applied} in this repository, again from oldest to newest (most
|
|
bos@8
|
253 recently applied).
|
|
bos@8
|
254 \end{itemize}
|
|
bos@8
|
255
|
|
bos@12
|
256 \begin{figure}[ht]
|
|
bos@8
|
257 \interaction{mq.tutorial.qseries}
|
|
bos@8
|
258 \caption{Understanding the patch stack with \hgcmd{qseries} and
|
|
bos@8
|
259 \hgcmd{qapplied}}
|
|
bos@8
|
260 \label{ex:mq:qseries}
|
|
bos@8
|
261 \end{figure}
|
|
bos@8
|
262
|
|
bos@8
|
263 \subsection{Manipulating the patch stack}
|
|
bos@8
|
264
|
|
bos@8
|
265 The previous discussion implied that there must be a difference
|
|
bos@11
|
266 between ``known'' and ``applied'' patches, and there is. MQ can
|
|
bos@11
|
267 manage a patch without it being applied in the repository.
|
|
bos@8
|
268
|
|
bos@8
|
269 An \emph{applied} patch has a corresponding changeset in the
|
|
bos@8
|
270 repository, and the effects of the patch and changeset are visible in
|
|
bos@8
|
271 the working directory. You can undo the application of a patch using
|
|
bos@12
|
272 the \hgcmd{qpop} command. MQ still \emph{knows about}, or manages, a
|
|
bos@12
|
273 popped patch, but the patch no longer has a corresponding changeset in
|
|
bos@12
|
274 the repository, and the working directory does not contain the changes
|
|
bos@12
|
275 made by the patch. Figure~\ref{fig:mq:stack} illustrates the
|
|
bos@12
|
276 difference between applied and tracked patches.
|
|
bos@12
|
277
|
|
bos@12
|
278 \begin{figure}[ht]
|
|
bos@12
|
279 \centering
|
|
bos@12
|
280 \grafix{mq-stack}
|
|
bos@12
|
281 \caption{Applied and unapplied patches in the MQ patch stack}
|
|
bos@12
|
282 \label{fig:mq:stack}
|
|
bos@8
|
283 \end{figure}
|
|
bos@8
|
284
|
|
bos@8
|
285 You can reapply an unapplied, or popped, patch using the \hgcmd{qpush}
|
|
bos@8
|
286 command. This creates a new changeset to correspond to the patch, and
|
|
bos@8
|
287 the patch's changes once again become present in the working
|
|
bos@8
|
288 directory. See figure~\ref{ex:mq:qpop} for examples of \hgcmd{qpop}
|
|
bos@8
|
289 and \hgcmd{qpush} in action. Notice that once we have popped a patch
|
|
bos@8
|
290 or two patches, the output of \hgcmd{qseries} remains the same, while
|
|
bos@8
|
291 that of \hgcmd{qapplied} has changed.
|
|
bos@8
|
292
|
|
bos@12
|
293 \begin{figure}[ht]
|
|
bos@12
|
294 \interaction{mq.tutorial.qpop}
|
|
bos@12
|
295 \caption{Modifying the stack of applied patches}
|
|
bos@12
|
296 \label{ex:mq:qpop}
|
|
bos@11
|
297 \end{figure}
|
|
bos@11
|
298
|
|
bos@8
|
299 MQ does not limit you to pushing or popping one patch. You can have
|
|
bos@8
|
300 no patches, all of them, or any number in between applied at some
|
|
bos@8
|
301 point in time.
|
|
bos@8
|
302
|
|
bos@13
|
303 \subsection{Working on several patches at once}
|
|
bos@13
|
304
|
|
bos@13
|
305 The \hgcmd{qrefresh} command always refreshes the \emph{topmost}
|
|
bos@13
|
306 applied patch. This means that you can suspend work on one patch (by
|
|
bos@13
|
307 refreshing it), pop or push to make a different patch the top, and
|
|
bos@13
|
308 work on \emph{that} patch for a while.
|
|
bos@13
|
309
|
|
bos@13
|
310 Here's an example that illustrates how you can use this ability.
|
|
bos@13
|
311 Let's say you're developing a new feature as two patches. The first
|
|
bos@13
|
312 is a change to the core of your software, and the second--layered on
|
|
bos@13
|
313 top of the first--changes the user interface to use the code you just
|
|
bos@13
|
314 added to the core. If you notice a bug in the core while you're
|
|
bos@13
|
315 working on the UI patch, it's easy to fix the core. Simply
|
|
bos@13
|
316 \hgcmd{qrefresh} the UI patch to save your in-progress changes, and
|
|
bos@13
|
317 \hgcmd{qpop} down to the core patch. Fix the core bug,
|
|
bos@13
|
318 \hgcmd{qrefresh} the core patch, and \hgcmd{qpush} back to the UI
|
|
bos@13
|
319 patch to continue where you left off.
|
|
bos@13
|
320
|
|
bos@13
|
321
|
|
bos@1
|
322 %%% Local Variables:
|
|
bos@1
|
323 %%% mode: latex
|
|
bos@1
|
324 %%% TeX-master: "00book"
|
|
bos@1
|
325 %%% End:
|
