| rev |
line source |
|
bos@1
|
1 \chapter{Managing change with Mercurial Queues}
|
|
bos@1
|
2 \label{chap:mq}
|
|
bos@1
|
3
|
|
bos@1
|
4 \section{The patch management problem}
|
|
bos@1
|
5 \label{sec:mq:patch-mgmt}
|
|
bos@1
|
6
|
|
bos@1
|
7 Here is a common scenario: you need to install a software package from
|
|
bos@1
|
8 source, but you find a bug that you must fix in the source before you
|
|
bos@1
|
9 can start using the package. You make your changes, forget about the
|
|
bos@1
|
10 package for a while, and a few months later you need to upgrade to a
|
|
bos@1
|
11 newer version of the package. If the newer version of the package
|
|
bos@1
|
12 still has the bug, you must extract your fix from the older source
|
|
bos@1
|
13 tree and apply it against the newer version. This is a tedious task,
|
|
bos@1
|
14 and it's easy to make mistakes.
|
|
bos@1
|
15
|
|
bos@1
|
16 This is a simple case of the ``patch management'' problem. You have
|
|
bos@1
|
17 an ``upstream'' source tree that you can't change; you need to make
|
|
bos@1
|
18 some local changes on top of the upstream tree; and you'd like to be
|
|
bos@1
|
19 able to keep those changes separate, so that you can apply them to
|
|
bos@1
|
20 newer versions of the upstream source.
|
|
bos@1
|
21
|
|
bos@1
|
22 The patch management problem arises in many situations. Probably the
|
|
bos@1
|
23 most visible is that a user of an open source software project will
|
|
bos@3
|
24 contribute a bug fix or new feature to the project's maintainers in the
|
|
bos@1
|
25 form of a patch.
|
|
bos@1
|
26
|
|
bos@1
|
27 Distributors of operating systems that include open source software
|
|
bos@1
|
28 often need to make changes to the packages they distribute so that
|
|
bos@1
|
29 they will build properly in their environments.
|
|
bos@1
|
30
|
|
bos@1
|
31 When you have few changes to maintain, it is easy to manage a single
|
|
bos@15
|
32 patch using the standard \texttt{diff} and \texttt{patch} programs
|
|
bos@15
|
33 (see section~\ref{sec:mq:patch} for a discussion of these tools).
|
|
bos@1
|
34 Once the number of changes grows, it starts to makes sense to maintain
|
|
bos@1
|
35 patches as discrete ``chunks of work,'' so that for example a single
|
|
bos@1
|
36 patch will contain only one bug fix (the patch might modify several
|
|
bos@1
|
37 files, but it's doing ``only one thing''), and you may have a number
|
|
bos@1
|
38 of such patches for different bugs you need fixed and local changes
|
|
bos@3
|
39 you require. In this situation, if you submit a bug fix patch to the
|
|
bos@1
|
40 upstream maintainers of a package and they include your fix in a
|
|
bos@1
|
41 subsequent release, you can simply drop that single patch when you're
|
|
bos@1
|
42 updating to the newer release.
|
|
bos@1
|
43
|
|
bos@1
|
44 Maintaining a single patch against an upstream tree is a little
|
|
bos@1
|
45 tedious and error-prone, but not difficult. However, the complexity
|
|
bos@1
|
46 of the problem grows rapidly as the number of patches you have to
|
|
bos@1
|
47 maintain increases. With more than a tiny number of patches in hand,
|
|
bos@1
|
48 understanding which ones you have applied and maintaining them moves
|
|
bos@1
|
49 from messy to overwhelming.
|
|
bos@1
|
50
|
|
bos@1
|
51 Fortunately, Mercurial includes a powerful extension, Mercurial Queues
|
|
bos@1
|
52 (or simply ``MQ''), that massively simplifies the patch management
|
|
bos@1
|
53 problem.
|
|
bos@1
|
54
|
|
bos@1
|
55 \section{The prehistory of Mercurial Queues}
|
|
bos@1
|
56 \label{sec:mq:history}
|
|
bos@1
|
57
|
|
bos@1
|
58 During the late 1990s, several Linux kernel developers started to
|
|
bos@1
|
59 maintain ``patch series'' that modified the behaviour of the Linux
|
|
bos@1
|
60 kernel. Some of these series were focused on stability, some on
|
|
bos@1
|
61 feature coverage, and others were more speculative.
|
|
bos@1
|
62
|
|
bos@1
|
63 The sizes of these patch series grew rapidly. In 2002, Andrew Morton
|
|
bos@1
|
64 published some shell scripts he had been using to automate the task of
|
|
bos@1
|
65 managing his patch queues. Andrew was successfully using these
|
|
bos@1
|
66 scripts to manage hundreds (sometimes thousands) of patches on top of
|
|
bos@1
|
67 the Linux kernel.
|
|
bos@1
|
68
|
|
bos@1
|
69 \subsection{A patchwork quilt}
|
|
bos@1
|
70 \label{sec:mq:quilt}
|
|
bos@1
|
71
|
|
bos@1
|
72
|
|
bos@1
|
73 In early 2003, Andreas Gruenbacher and Martin Quinson borrowed the
|
|
bos@2
|
74 approach of Andrew's scripts and published a tool called ``patchwork
|
|
bos@2
|
75 quilt''~\cite{web:quilt}, or simply ``quilt''
|
|
bos@2
|
76 (see~\cite{gruenbacher:2005} for a paper describing it). Because
|
|
bos@2
|
77 quilt substantially automated patch management, it rapidly gained a
|
|
bos@2
|
78 large following among open source software developers.
|
|
bos@1
|
79
|
|
bos@1
|
80 Quilt manages a \emph{stack of patches} on top of a directory tree.
|
|
bos@1
|
81 To begin, you tell quilt to manage a directory tree; it stores away
|
|
bos@1
|
82 the names and contents of all files in the tree. To fix a bug, you
|
|
bos@1
|
83 create a new patch (using a single command), edit the files you need
|
|
bos@1
|
84 to fix, then ``refresh'' the patch.
|
|
bos@1
|
85
|
|
bos@1
|
86 The refresh step causes quilt to scan the directory tree; it updates
|
|
bos@1
|
87 the patch with all of the changes you have made. You can create
|
|
bos@1
|
88 another patch on top of the first, which will track the changes
|
|
bos@1
|
89 required to modify the tree from ``tree with one patch applied'' to
|
|
bos@1
|
90 ``tree with two patches applied''.
|
|
bos@1
|
91
|
|
bos@1
|
92 You can \emph{change} which patches are applied to the tree. If you
|
|
bos@1
|
93 ``pop'' a patch, the changes made by that patch will vanish from the
|
|
bos@1
|
94 directory tree. Quilt remembers which patches you have popped,
|
|
bos@1
|
95 though, so you can ``push'' a popped patch again, and the directory
|
|
bos@1
|
96 tree will be restored to contain the modifications in the patch. Most
|
|
bos@1
|
97 importantly, you can run the ``refresh'' command at any time, and the
|
|
bos@1
|
98 topmost applied patch will be updated. This means that you can, at
|
|
bos@1
|
99 any time, change both which patches are applied and what
|
|
bos@1
|
100 modifications those patches make.
|
|
bos@1
|
101
|
|
bos@1
|
102 Quilt knows nothing about revision control tools, so it works equally
|
|
bos@3
|
103 well on top of an unpacked tarball or a Subversion repository.
|
|
bos@1
|
104
|
|
bos@1
|
105 \subsection{From patchwork quilt to Mercurial Queues}
|
|
bos@1
|
106 \label{sec:mq:quilt-mq}
|
|
bos@1
|
107
|
|
bos@1
|
108 In mid-2005, Chris Mason took the features of quilt and wrote an
|
|
bos@1
|
109 extension that he called Mercurial Queues, which added quilt-like
|
|
bos@1
|
110 behaviour to Mercurial.
|
|
bos@1
|
111
|
|
bos@1
|
112 The key difference between quilt and MQ is that quilt knows nothing
|
|
bos@1
|
113 about revision control systems, while MQ is \emph{integrated} into
|
|
bos@1
|
114 Mercurial. Each patch that you push is represented as a Mercurial
|
|
bos@1
|
115 changeset. Pop a patch, and the changeset goes away.
|
|
bos@1
|
116
|
|
bos@1
|
117 This integration makes understanding patches and debugging their
|
|
bos@1
|
118 effects \emph{enormously} easier. Since every applied patch has an
|
|
bos@1
|
119 associated changeset, you can use \hgcmdargs{log}{\emph{filename}} to
|
|
bos@1
|
120 see which changesets and patches affected a file. You can use the
|
|
bos@1
|
121 \hgext{bisect} extension to binary-search through all changesets and
|
|
bos@1
|
122 applied patches to see where a bug got introduced or fixed. You can
|
|
bos@1
|
123 use the \hgcmd{annotate} command to see which changeset or patch
|
|
bos@1
|
124 modified a particular line of a source file. And so on.
|
|
bos@1
|
125
|
|
bos@1
|
126 Because quilt does not care about revision control tools, it is still
|
|
bos@1
|
127 a tremendously useful piece of software to know about for situations
|
|
bos@1
|
128 where you cannot use Mercurial and MQ.
|
|
bos@2
|
129 \section{Getting started with Mercurial Queues}
|
|
bos@2
|
130 \label{sec:mq:start}
|
|
bos@1
|
131
|
|
bos@3
|
132 Because MQ is implemented as an extension, you must explicitly enable
|
|
bos@3
|
133 before you can use it. (You don't need to download anything; MQ ships
|
|
bos@3
|
134 with the standard Mercurial distribution.) To enable MQ, edit your
|
|
bos@4
|
135 \tildefile{.hgrc} file, and add the lines in figure~\ref{ex:mq:config}.
|
|
bos@2
|
136
|
|
bos@12
|
137 \begin{figure}[ht]
|
|
bos@4
|
138 \begin{codesample4}
|
|
bos@4
|
139 [extensions]
|
|
bos@4
|
140 hgext.mq =
|
|
bos@4
|
141 \end{codesample4}
|
|
bos@4
|
142 \label{ex:mq:config}
|
|
bos@4
|
143 \caption{Contents to add to \tildefile{.hgrc} to enable the MQ extension}
|
|
bos@4
|
144 \end{figure}
|
|
bos@3
|
145
|
|
bos@3
|
146 Once the extension is enabled, it will make a number of new commands
|
|
bos@7
|
147 available. To verify that the extension is working, you can use
|
|
bos@7
|
148 \hgcmd{help} to see if the \hgcmd{qinit} command is now available; see
|
|
bos@7
|
149 the example in figure~\ref{ex:mq:enabled}.
|
|
bos@3
|
150
|
|
bos@12
|
151 \begin{figure}[ht]
|
|
bos@4
|
152 \interaction{mq.qinit-help.help}
|
|
bos@4
|
153 \caption{How to verify that MQ is enabled}
|
|
bos@4
|
154 \label{ex:mq:enabled}
|
|
bos@4
|
155 \end{figure}
|
|
bos@1
|
156
|
|
bos@8
|
157 You can use MQ with \emph{any} Mercurial repository, and its commands
|
|
bos@8
|
158 only operate within that repository. To get started, simply prepare
|
|
bos@8
|
159 the repository using the \hgcmd{qinit} command (see
|
|
bos@7
|
160 figure~\ref{ex:mq:qinit}). This command creates an empty directory
|
|
bos@16
|
161 called \sdirname{.hg/patches}, where MQ will keep its metadata. As
|
|
bos@7
|
162 with many Mercurial commands, the \hgcmd{qinit} command prints nothing
|
|
bos@7
|
163 if it succeeds.
|
|
bos@7
|
164
|
|
bos@12
|
165 \begin{figure}[ht]
|
|
bos@7
|
166 \interaction{mq.tutorial.qinit}
|
|
bos@7
|
167 \caption{Preparing a repository for use with MQ}
|
|
bos@7
|
168 \label{ex:mq:qinit}
|
|
bos@7
|
169 \end{figure}
|
|
bos@7
|
170
|
|
bos@12
|
171 \begin{figure}[ht]
|
|
bos@7
|
172 \interaction{mq.tutorial.qnew}
|
|
bos@7
|
173 \caption{Creating a new patch}
|
|
bos@7
|
174 \label{ex:mq:qnew}
|
|
bos@7
|
175 \end{figure}
|
|
bos@7
|
176
|
|
bos@8
|
177 \subsection{Creating a new patch}
|
|
bos@8
|
178
|
|
bos@8
|
179 To begin work on a new patch, use the \hgcmd{qnew} command. This
|
|
bos@7
|
180 command takes one argument, the name of the patch to create. MQ will
|
|
bos@16
|
181 use this as the name of an actual file in the \sdirname{.hg/patches}
|
|
bos@7
|
182 directory, as you can see in figure~\ref{ex:mq:qnew}.
|
|
bos@7
|
183
|
|
bos@16
|
184 Also newly present in the \sdirname{.hg/patches} directory are two
|
|
bos@16
|
185 other files, \sfilename{series} and \sfilename{status}. The
|
|
bos@16
|
186 \sfilename{series} file lists all of the patches that MQ knows about
|
|
bos@8
|
187 for this repository, with one patch per line. Mercurial uses the
|
|
bos@16
|
188 \sfilename{status} file for internal book-keeping; it tracks all of the
|
|
bos@7
|
189 patches that MQ has \emph{applied} in this repository.
|
|
bos@7
|
190
|
|
bos@7
|
191 \begin{note}
|
|
bos@16
|
192 You may sometimes want to edit the \sfilename{series} file by hand;
|
|
bos@7
|
193 for example, to change the sequence in which some patches are
|
|
bos@16
|
194 applied. However, manually editing the \sfilename{status} file is
|
|
bos@7
|
195 almost always a bad idea, as it's easy to corrupt MQ's idea of what
|
|
bos@7
|
196 is happening.
|
|
bos@7
|
197 \end{note}
|
|
bos@7
|
198
|
|
bos@8
|
199 Once you have created your new patch, you can edit files in the
|
|
bos@8
|
200 working directory as you usually would. All of the normal Mercurial
|
|
bos@8
|
201 commands, such as \hgcmd{diff} and \hgcmd{annotate}, work exactly as
|
|
bos@8
|
202 they did before.
|
|
bos@8
|
203 \subsection{Refreshing a patch}
|
|
bos@8
|
204
|
|
bos@8
|
205 When you reach a point where you want to save your work, use the
|
|
bos@8
|
206 \hgcmd{qrefresh} command (figure~\ref{ex:mq:qnew}) to update the patch
|
|
bos@8
|
207 you are working on. This command folds the changes you have made in
|
|
bos@8
|
208 the working directory into your patch, and updates its corresponding
|
|
bos@8
|
209 changeset to contain those changes.
|
|
bos@8
|
210
|
|
bos@12
|
211 \begin{figure}[ht]
|
|
bos@8
|
212 \interaction{mq.tutorial.qrefresh}
|
|
bos@8
|
213 \caption{Refreshing a patch}
|
|
bos@8
|
214 \label{ex:mq:qrefresh}
|
|
bos@8
|
215 \end{figure}
|
|
bos@8
|
216
|
|
bos@8
|
217 You can run \hgcmd{qrefresh} as often as you like, so it's a good way
|
|
bos@13
|
218 to ``checkpoint'' your work. Refresh your patch at an opportune
|
|
bos@8
|
219 time; try an experiment; and if the experiment doesn't work out,
|
|
bos@8
|
220 \hgcmd{revert} your modifications back to the last time you refreshed.
|
|
bos@8
|
221
|
|
bos@12
|
222 \begin{figure}[ht]
|
|
bos@8
|
223 \interaction{mq.tutorial.qrefresh2}
|
|
bos@8
|
224 \caption{Refresh a patch many times to accumulate changes}
|
|
bos@8
|
225 \label{ex:mq:qrefresh2}
|
|
bos@8
|
226 \end{figure}
|
|
bos@8
|
227
|
|
bos@8
|
228 \subsection{Stacking and tracking patches}
|
|
bos@8
|
229
|
|
bos@8
|
230 Once you have finished working on a patch, or need to work on another,
|
|
bos@8
|
231 you can use the \hgcmd{qnew} command again to create a new patch.
|
|
bos@8
|
232 Mercurial will apply this patch on top of your existing patch. See
|
|
bos@8
|
233 figure~\ref{ex:mq:qnew2} for an example. Notice that the patch
|
|
bos@8
|
234 contains the changes in our prior patch as part of its context (you
|
|
bos@8
|
235 can see this more clearly in the output of \hgcmd{annotate}).
|
|
bos@8
|
236
|
|
bos@12
|
237 \begin{figure}[ht]
|
|
bos@8
|
238 \interaction{mq.tutorial.qnew2}
|
|
bos@8
|
239 \caption{Stacking a second patch on top of the first}
|
|
bos@8
|
240 \label{ex:mq:qnew2}
|
|
bos@8
|
241 \end{figure}
|
|
bos@8
|
242
|
|
bos@8
|
243 So far, with the exception of \hgcmd{qnew} and \hgcmd{qrefresh}, we've
|
|
bos@8
|
244 been careful to only use regular Mercurial commands. However, there
|
|
bos@8
|
245 are more ``natural'' commands you can use when thinking about patches
|
|
bos@8
|
246 with MQ, as illustrated in figure~\ref{ex:mq:qseries}:
|
|
bos@8
|
247
|
|
bos@8
|
248 \begin{itemize}
|
|
bos@8
|
249 \item The \hgcmd{qseries} command lists every patch that MQ knows
|
|
bos@8
|
250 about in this repository, from oldest to newest (most recently
|
|
bos@8
|
251 \emph{created}).
|
|
bos@8
|
252 \item The \hgcmd{qapplied} command lists every patch that MQ has
|
|
bos@8
|
253 \emph{applied} in this repository, again from oldest to newest (most
|
|
bos@8
|
254 recently applied).
|
|
bos@8
|
255 \end{itemize}
|
|
bos@8
|
256
|
|
bos@12
|
257 \begin{figure}[ht]
|
|
bos@8
|
258 \interaction{mq.tutorial.qseries}
|
|
bos@8
|
259 \caption{Understanding the patch stack with \hgcmd{qseries} and
|
|
bos@8
|
260 \hgcmd{qapplied}}
|
|
bos@8
|
261 \label{ex:mq:qseries}
|
|
bos@8
|
262 \end{figure}
|
|
bos@8
|
263
|
|
bos@8
|
264 \subsection{Manipulating the patch stack}
|
|
bos@8
|
265
|
|
bos@8
|
266 The previous discussion implied that there must be a difference
|
|
bos@11
|
267 between ``known'' and ``applied'' patches, and there is. MQ can
|
|
bos@11
|
268 manage a patch without it being applied in the repository.
|
|
bos@8
|
269
|
|
bos@8
|
270 An \emph{applied} patch has a corresponding changeset in the
|
|
bos@8
|
271 repository, and the effects of the patch and changeset are visible in
|
|
bos@8
|
272 the working directory. You can undo the application of a patch using
|
|
bos@12
|
273 the \hgcmd{qpop} command. MQ still \emph{knows about}, or manages, a
|
|
bos@12
|
274 popped patch, but the patch no longer has a corresponding changeset in
|
|
bos@12
|
275 the repository, and the working directory does not contain the changes
|
|
bos@12
|
276 made by the patch. Figure~\ref{fig:mq:stack} illustrates the
|
|
bos@12
|
277 difference between applied and tracked patches.
|
|
bos@12
|
278
|
|
bos@12
|
279 \begin{figure}[ht]
|
|
bos@12
|
280 \centering
|
|
bos@12
|
281 \grafix{mq-stack}
|
|
bos@12
|
282 \caption{Applied and unapplied patches in the MQ patch stack}
|
|
bos@12
|
283 \label{fig:mq:stack}
|
|
bos@8
|
284 \end{figure}
|
|
bos@8
|
285
|
|
bos@8
|
286 You can reapply an unapplied, or popped, patch using the \hgcmd{qpush}
|
|
bos@8
|
287 command. This creates a new changeset to correspond to the patch, and
|
|
bos@8
|
288 the patch's changes once again become present in the working
|
|
bos@8
|
289 directory. See figure~\ref{ex:mq:qpop} for examples of \hgcmd{qpop}
|
|
bos@8
|
290 and \hgcmd{qpush} in action. Notice that once we have popped a patch
|
|
bos@8
|
291 or two patches, the output of \hgcmd{qseries} remains the same, while
|
|
bos@8
|
292 that of \hgcmd{qapplied} has changed.
|
|
bos@8
|
293
|
|
bos@12
|
294 \begin{figure}[ht]
|
|
bos@12
|
295 \interaction{mq.tutorial.qpop}
|
|
bos@12
|
296 \caption{Modifying the stack of applied patches}
|
|
bos@12
|
297 \label{ex:mq:qpop}
|
|
bos@11
|
298 \end{figure}
|
|
bos@11
|
299
|
|
bos@8
|
300 MQ does not limit you to pushing or popping one patch. You can have
|
|
bos@8
|
301 no patches, all of them, or any number in between applied at some
|
|
bos@8
|
302 point in time.
|
|
bos@8
|
303
|
|
bos@13
|
304 \subsection{Working on several patches at once}
|
|
bos@13
|
305
|
|
bos@13
|
306 The \hgcmd{qrefresh} command always refreshes the \emph{topmost}
|
|
bos@13
|
307 applied patch. This means that you can suspend work on one patch (by
|
|
bos@13
|
308 refreshing it), pop or push to make a different patch the top, and
|
|
bos@13
|
309 work on \emph{that} patch for a while.
|
|
bos@13
|
310
|
|
bos@13
|
311 Here's an example that illustrates how you can use this ability.
|
|
bos@13
|
312 Let's say you're developing a new feature as two patches. The first
|
|
bos@13
|
313 is a change to the core of your software, and the second--layered on
|
|
bos@13
|
314 top of the first--changes the user interface to use the code you just
|
|
bos@13
|
315 added to the core. If you notice a bug in the core while you're
|
|
bos@13
|
316 working on the UI patch, it's easy to fix the core. Simply
|
|
bos@13
|
317 \hgcmd{qrefresh} the UI patch to save your in-progress changes, and
|
|
bos@13
|
318 \hgcmd{qpop} down to the core patch. Fix the core bug,
|
|
bos@13
|
319 \hgcmd{qrefresh} the core patch, and \hgcmd{qpush} back to the UI
|
|
bos@13
|
320 patch to continue where you left off.
|
|
bos@13
|
321
|
|
bos@14
|
322 \section{Mercurial Queues and GNU patch}
|
|
bos@15
|
323 \label{sec:mq:patch}
|
|
bos@14
|
324
|
|
bos@16
|
325 MQ uses the GNU \command{patch} command to apply patches. Because MQ
|
|
bos@16
|
326 doesn't hide its patch-oriented nature, it is helpful to understand
|
|
bos@16
|
327 the data that MQ and \command{patch} work with, and a few aspects of
|
|
bos@16
|
328 how \command{patch} operates.
|
|
bos@14
|
329
|
|
bos@15
|
330 The \command{diff} command generates a list of modifications by
|
|
bos@15
|
331 comparing two files. The \command{patch} command applies a list of
|
|
bos@15
|
332 modifications to a file. The kinds of files that \command{diff} and
|
|
bos@15
|
333 \command{patch} work with are referred to as both ``diffs'' and
|
|
bos@15
|
334 ``patches;'' there is no difference between a diff and a patch.
|
|
bos@15
|
335
|
|
bos@14
|
336 A patch file can start with arbitrary text; MQ uses this text as the
|
|
bos@14
|
337 commit message when creating changesets. It treats the first line
|
|
bos@14
|
338 that starts with the string ``\texttt{diff~-}'' as the separator
|
|
bos@14
|
339 between header and content.
|
|
bos@14
|
340
|
|
bos@15
|
341 MQ works with \emph{unified} diffs (\command{patch} can accept several
|
|
bos@15
|
342 other diff formats, but MQ doesn't). A unified diff contains two
|
|
bos@14
|
343 kinds of header. The \emph{file header} describes the file being
|
|
bos@14
|
344 modified; it contains the name of the file to modify. When
|
|
bos@15
|
345 \command{patch} sees a new file header, it looks for a file with that
|
|
bos@14
|
346 name to start modifying.
|
|
bos@14
|
347
|
|
bos@15
|
348 After the file header comes a series of \emph{hunks}. Each hunk
|
|
bos@15
|
349 starts with a header; this identifies the range of line numbers within
|
|
bos@15
|
350 the file that the hunk should modify. Following the header, a hunk
|
|
bos@15
|
351 starts and ends with a few (usually three) lines of text from the
|
|
bos@15
|
352 unmodified file; these are called the \emph{context} for the hunk.
|
|
bos@15
|
353 Each unmodified line begins with a space characters. Within the hunk,
|
|
bos@15
|
354 a line that begins with ``\texttt{-}'' means ``remove this line,''
|
|
bos@15
|
355 while a line that begins with ``\texttt{+}'' means ``insert this
|
|
bos@15
|
356 line.'' For example, a line that is modified is represented by one
|
|
bos@15
|
357 deletion and one insertion.
|
|
bos@15
|
358
|
|
bos@15
|
359 The \command{diff} command runs hunks together when there's not enough
|
|
bos@15
|
360 context between modifications to justify
|
|
bos@14
|
361
|
|
bos@14
|
362 When \command{patch} applies a hunk, it tries a handful of
|
|
bos@14
|
363 successively less accurate strategies to try to make the hunk apply.
|
|
bos@14
|
364 This falling-back technique often makes it possible to take a patch
|
|
bos@14
|
365 that was generated against an old version of a file, and apply it
|
|
bos@14
|
366 against a newer version of that file.
|
|
bos@14
|
367
|
|
bos@14
|
368 First, \command{patch} tries an exact match, where the line numbers,
|
|
bos@14
|
369 the context, and the text to be modified must apply exactly. If it
|
|
bos@14
|
370 cannot make an exact match, it tries to find an exact match for the
|
|
bos@14
|
371 context, without honouring the line numbering information. If this
|
|
bos@14
|
372 succeeds, it prints a line of output saying that the hunk was applied,
|
|
bos@14
|
373 but at some \emph{offset} from the original line number.
|
|
bos@14
|
374
|
|
bos@14
|
375 If a context-only match fails, \command{patch} removes the first and
|
|
bos@14
|
376 last lines of the context, and tries a \emph{reduced} context-only
|
|
bos@14
|
377 match. If the hunk with reduced context succeeds, it prints a message
|
|
bos@14
|
378 saying that it applied the hunk with a \emph{fuzz factor} (the number
|
|
bos@14
|
379 after the fuzz factor indicates how many lines of context
|
|
bos@14
|
380 \command{patch} had to trim before the patch applied).
|
|
bos@14
|
381
|
|
bos@14
|
382 When neither of these techniques works, \command{patch} prints a
|
|
bos@14
|
383 message saying that the hunk in question was rejected. It saves
|
|
bos@17
|
384 rejected hunks (also simply called ``rejects'') to a file with the
|
|
bos@17
|
385 same name, and an added \sfilename{.rej} extension. It also saves an
|
|
bos@17
|
386 unmodified copy of the file with a \sfilename{.orig} extension; the
|
|
bos@17
|
387 copy of the file without any extensions will contain any changes made
|
|
bos@17
|
388 by hunks that \emph{did} apply cleanly. If you have a patch that
|
|
bos@17
|
389 modifies \filename{foo} with six hunks, and one of them fails to
|
|
bos@17
|
390 apply, you will have: an unmodified \filename{foo.orig}, a
|
|
bos@17
|
391 \filename{foo.rej} containing one hunk, and \filename{foo}, containing
|
|
bos@17
|
392 the changes made by the five successful five hunks.
|
|
bos@14
|
393
|
|
bos@14
|
394 \subsection{Beware the fuzz}
|
|
bos@14
|
395
|
|
bos@14
|
396 While applying a hunk at an offset, or with a fuzz factor, will often
|
|
bos@14
|
397 be completely successful, these inexact techniques naturally leave
|
|
bos@14
|
398 open the possibility of corrupting the patched file. The most common
|
|
bos@14
|
399 cases typically involve applying a patch twice, or at an incorrect
|
|
bos@14
|
400 location in the file. If \command{patch} or \hgcmd{qpush} ever
|
|
bos@14
|
401 mentions an offset or fuzz factor, you should make sure that the
|
|
bos@14
|
402 modified files are correct afterwards.
|
|
bos@14
|
403
|
|
bos@14
|
404 It's often a good idea to refresh a patch that has applied with an
|
|
bos@14
|
405 offset or fuzz factor; refreshing the patch generates new context
|
|
bos@14
|
406 information that will make it apply cleanly. I say ``often,'' not
|
|
bos@14
|
407 ``always,'' because sometimes refreshing a patch will make it fail to
|
|
bos@14
|
408 apply against a different revision of the underlying files. In some
|
|
bos@14
|
409 cases, such as when you're maintaining a patch that must sit on top of
|
|
bos@14
|
410 multiple versions of a source tree, it's acceptable to have a patch
|
|
bos@14
|
411 apply with some fuzz, provided you've verified the results of the
|
|
bos@14
|
412 patching process in such cases.
|
|
bos@14
|
413
|
|
bos@15
|
414 \subsection{Handling rejection}
|
|
bos@15
|
415
|
|
bos@15
|
416 If \hgcmd{qpush} fails to apply a patch, it will print an error
|
|
bos@16
|
417 message and exit. If it has left \sfilename{.rej} files behind, it is
|
|
bos@15
|
418 usually best to fix up the rejected hunks before you push more patches
|
|
bos@15
|
419 or do any further work.
|
|
bos@15
|
420
|
|
bos@15
|
421 If your patch \emph{used to} apply cleanly, and no longer does because
|
|
bos@15
|
422 you've changed the underlying code that your patches are based on,
|
|
bos@17
|
423 Mercurial Queues can help; see section~\ref{sec:mq:merge} for details.
|
|
bos@15
|
424
|
|
bos@15
|
425 Unfortunately, there aren't any great techniques for dealing with
|
|
bos@16
|
426 rejected hunks. Most often, you'll need to view the \sfilename{.rej}
|
|
bos@15
|
427 file and edit the target file, applying the rejected hunks by hand.
|
|
bos@15
|
428
|
|
bos@16
|
429 If you're feeling adventurous, Neil Brown, a Linux kernel hacker,
|
|
bos@16
|
430 wrote a tool called \command{wiggle}~\cite{web:wiggle}, which is more
|
|
bos@16
|
431 vigorous than \command{patch} in its attempts to make a patch apply.
|
|
bos@15
|
432
|
|
bos@15
|
433 Another Linux kernel hacker, Chris Mason (the author of Mercurial
|
|
bos@15
|
434 Queues), wrote a similar tool called \command{rej}~\cite{web:rej},
|
|
bos@15
|
435 which takes a simple approach to automating the application of hunks
|
|
bos@15
|
436 rejected by \command{patch}. \command{rej} can help with four common
|
|
bos@15
|
437 reasons that a hunk may be rejected:
|
|
bos@15
|
438
|
|
bos@15
|
439 \begin{itemize}
|
|
bos@15
|
440 \item The context in the middle of a hunk has changed.
|
|
bos@15
|
441 \item A hunk is missing some context at the beginning or end.
|
|
bos@15
|
442 \item A large hunk might apply better--either entirely or in part--if
|
|
bos@15
|
443 it was broken up into smaller hunks.
|
|
bos@15
|
444 \item A hunk removes lines with slightly different content than those
|
|
bos@15
|
445 currently present in the file.
|
|
bos@15
|
446 \end{itemize}
|
|
bos@15
|
447
|
|
bos@15
|
448 If you use \command{wiggle} or \command{rej}, you should be doubly
|
|
bos@15
|
449 careful to check your results when you're done.
|
|
bos@15
|
450
|
|
bos@17
|
451 \section{Getting the best performance out of MQ}
|
|
bos@17
|
452
|
|
bos@17
|
453 MQ is very efficient at handling a large number of patches. I ran
|
|
bos@17
|
454 some performance experiments in mid-2006 for a talk that I gave at the
|
|
bos@17
|
455 2006 EuroPython conference~\cite{web:europython}. I used as my data
|
|
bos@17
|
456 set the Linux 2.6.17-mm1 patch series, which consists of 1,738
|
|
bos@17
|
457 patches. I applied thes on top of a Linux kernel repository
|
|
bos@17
|
458 containing all 27,472 revisions between Linux 2.6.12-rc2 and Linux
|
|
bos@17
|
459 2.6.17.
|
|
bos@17
|
460
|
|
bos@17
|
461 On my old, slow laptop, I was able to
|
|
bos@17
|
462 \hgcmdargs{qpush}{\hgopt{qpush}{-a}} all 1,738 patches in 3.5 minutes,
|
|
bos@17
|
463 and \hgcmdargs{qpop}{\hgopt{qpop}{-a}} them all in 30 seconds. I
|
|
bos@17
|
464 could \hgcmd{qrefresh} one of the biggest patches (which made 22,779
|
|
bos@17
|
465 lines of changes to 287 files) in 6.6 seconds.
|
|
bos@17
|
466
|
|
bos@17
|
467 Clearly, MQ is well suited to working in large trees, but there are a
|
|
bos@17
|
468 few tricks you can use to get the best performance of it.
|
|
bos@17
|
469
|
|
bos@17
|
470 First of all, try to ``batch'' operations together. Every time you
|
|
bos@17
|
471 run \hgcmd{qpush} or \hgcmd{qpop}, these commands scan the working
|
|
bos@17
|
472 directory once to make sure you haven't made some changes and then
|
|
bos@17
|
473 forgotten to run \hgcmd{qrefresh}. On a small tree, the time that
|
|
bos@17
|
474 this scan takes is unnoticeable. However, on a medium-sized tree
|
|
bos@17
|
475 (containing tens of thousands of files), it can take a second or more.
|
|
bos@17
|
476
|
|
bos@17
|
477 The \hgcmd{qpush} and \hgcmd{qpop} commands allow you to push and pop
|
|
bos@17
|
478 multiple patches at a time. You can identify the ``destination
|
|
bos@17
|
479 patch'' that you want to end up at. When you \hgcmd{qpush} with a
|
|
bos@17
|
480 destination specified, it will push patches until that patch is at the
|
|
bos@17
|
481 top of the applied stack. When you \hgcmd{qpop} to a destination, MQ
|
|
bos@17
|
482 will pop patches until the destination patch \emph{is no longer}
|
|
bos@17
|
483 applied.
|
|
bos@17
|
484
|
|
bos@17
|
485 You can identify a destination patch using either the name of the
|
|
bos@17
|
486 patch, or by number. If you use numeric addressing, patches are
|
|
bos@17
|
487 counted from zero; this means that the first patch is zero, the second
|
|
bos@17
|
488 is one, and so on.
|
|
bos@17
|
489
|
|
bos@15
|
490 \section{Updating your patches when the underlying code changes}
|
|
bos@15
|
491 \label{sec:mq:merge}
|
|
bos@15
|
492
|
|
bos@17
|
493 It's common to have a stack of patches on top of an underlying
|
|
bos@17
|
494 repository that you don't modify directly. If you're working on
|
|
bos@17
|
495 changes to third-party code, or on a feature that is taking longer to
|
|
bos@17
|
496 develop than the rate of change of the code beneath, you will often
|
|
bos@17
|
497 need to sync up with the underlying code, and fix up any hunks in your
|
|
bos@17
|
498 patches that no longer apply. This is called \emph{rebasing} your
|
|
bos@17
|
499 patch series.
|
|
bos@17
|
500
|
|
bos@17
|
501 The simplest way to do this is to \hgcmdargs{qpop}{\hgopt{qpop}{-a}}
|
|
bos@17
|
502 your patches, then \hgcmd{pull} changes into the underlying
|
|
bos@17
|
503 repository, and finally \hgcmdargs{qpush}{\hgopt{qpop}{-a}} your
|
|
bos@17
|
504 patches again. MQ will stop pushing any time it runs across a patch
|
|
bos@17
|
505 that fails to apply during conflicts, allowing you to fix your
|
|
bos@17
|
506 conflicts, \hgcmd{qrefresh} the affected patch, and continue pushing
|
|
bos@17
|
507 until you have fixed your entire stack.
|
|
bos@17
|
508
|
|
bos@17
|
509 This approach is easy to use and works well if you don't expect
|
|
bos@17
|
510 changes to the underlying code to affect how well your patches apply.
|
|
bos@17
|
511 If your patch stack touches code that is modified frequently or
|
|
bos@17
|
512 invasively in the underlying repository, however, fixing up rejected
|
|
bos@17
|
513 hunks by hand quickly becomes tiresome.
|
|
bos@17
|
514
|
|
bos@17
|
515 It's possible to partially automate the rebasing process. If your
|
|
bos@17
|
516 patches apply cleanly against some revision of the underlying repo, MQ
|
|
bos@17
|
517 can use this information to help you to resolve conflicts between your
|
|
bos@17
|
518 patches and a different revision.
|
|
bos@17
|
519
|
|
bos@17
|
520 The process is a little involved.
|
|
bos@17
|
521 \begin{enumerate}
|
|
bos@17
|
522 \item To begin, \hgcmdargs{qpush}{-a} all of your patches on top of
|
|
bos@17
|
523 the revision where you know that they apply cleanly.
|
|
bos@17
|
524 \item Save a backup copy of your patch directory using
|
|
bos@17
|
525 \hgcmdargs{qsave}{\hgopt{qsave}{-e} \hgopt{qsave}{-c}}. This prints
|
|
bos@17
|
526 the name of the directory that it has saved the patches in. It will
|
|
bos@17
|
527 save the patches to a directory called
|
|
bos@17
|
528 \sdirname{.hg/patches.\emph{N}}, where \texttt{\emph{N}} is a small
|
|
bos@17
|
529 integer. It also commits a ``save changeset'' on top of your
|
|
bos@17
|
530 applied patches; this is for internal book-keeping, and records the
|
|
bos@17
|
531 states of the \sfilename{series} and \sfilename{status} files.
|
|
bos@17
|
532 \item Use \hgcmd{pull} to bring new changes into the underlying
|
|
bos@17
|
533 repository. (Don't run \hgcmdargs{pull}{-u}; see below for why.)
|
|
bos@17
|
534 \item Update to the new tip revision, using
|
|
bos@17
|
535 \hgcmdargs{update}{\hgopt{update}{-C}} to override the patches you
|
|
bos@17
|
536 have pushed.
|
|
bos@17
|
537 \item Merge all patches using \hgcmdargs{qpush}{\hgopt{qpush}{-m}
|
|
bos@17
|
538 \hgopt{qpush}{-a}}. The \hgopt{qpush}{-m} option to \hgcmd{qpush}
|
|
bos@17
|
539 tells MQ to perform a three-way merge if the patch fails to apply.
|
|
bos@17
|
540 \end{enumerate}
|
|
bos@17
|
541
|
|
bos@17
|
542 During the \hgcmdargs{qpush}{\hgopt{qpush}{-m}}, each patch in the
|
|
bos@17
|
543 \sfilename{series} file is applied normally. If a patch applies with
|
|
bos@17
|
544 fuzz or rejects, MQ looks at the queue you \hgcmd{qsave}d, and
|
|
bos@17
|
545 performs a three-way merge with the corresponding changeset. This
|
|
bos@17
|
546 merge uses Mercurial's normal merge machinery, so it may pop up a GUI
|
|
bos@17
|
547 merge tool to help you to resolve problems.
|
|
bos@17
|
548
|
|
bos@17
|
549 When you finish resolving the effects of a patch, MQ refreshes your
|
|
bos@17
|
550 patch based on the result of the merge.
|
|
bos@17
|
551
|
|
bos@17
|
552 At the end of this process, your repository will have one extra head
|
|
bos@17
|
553 from the old patch queue, and a copy of the old patch queue will be in
|
|
bos@17
|
554 \sdirname{.hg/patches.\emph{N}}. You can remove the extra head using
|
|
bos@17
|
555 \hgcmdargs{qpop}{\hgopt{qpop}{-a} \hgopt{qpop}{-n} patches.\emph{N}}
|
|
bos@17
|
556 or \hgcmd{strip}. You can delete \sdirname{.hg/patches.\emph{N}} once
|
|
bos@17
|
557 you are sure that you no longer need it as a backup.
|
|
bos@13
|
558
|
|
bos@16
|
559 \section{Managing patches in a repository}
|
|
bos@16
|
560
|
|
bos@16
|
561 Because MQ's \sdirname{.hg/patches} directory resides outside a
|
|
bos@16
|
562 Mercurial repository's working directory, the ``underlying'' Mercurial
|
|
bos@16
|
563 repository knows nothing about the management or presence of patches.
|
|
bos@16
|
564
|
|
bos@16
|
565 This presents the interesting possibility of managing the contents of
|
|
bos@16
|
566 the patch directory as a Mercurial repository in its own right. This
|
|
bos@16
|
567 can be a useful way to work. For example, you can work on a patch for
|
|
bos@16
|
568 a while, \hgcmd{qrefresh} it, then \hgcmd{commit} the current state of
|
|
bos@16
|
569 the patch. This lets you ``roll back'' to that version of the patch
|
|
bos@16
|
570 later on.
|
|
bos@16
|
571
|
|
bos@16
|
572 In addition, you can then share different versions of the same patch
|
|
bos@16
|
573 stack among multiple underlying repositories. I use this when I am
|
|
bos@16
|
574 developing a Linux kernel feature. I have a pristine copy of my
|
|
bos@16
|
575 kernel sources for each of several CPU architectures, and a cloned
|
|
bos@16
|
576 repository under each that contains the patches I am working on. When
|
|
bos@16
|
577 I want to test a change on a different architecture, I push my current
|
|
bos@16
|
578 patches to the patch repository associated with that kernel tree, pop
|
|
bos@16
|
579 and push all of my patches, and build and test that kernel.
|
|
bos@16
|
580
|
|
bos@16
|
581 Managing patches in a repository makes it possible for multiple
|
|
bos@16
|
582 developers to work on the same patch series without colliding with
|
|
bos@16
|
583 each other, all on top of an underlying source base that they may or
|
|
bos@16
|
584 may not control.
|
|
bos@16
|
585
|
|
bos@17
|
586 \subsection{MQ support for patch repositories}
|
|
bos@16
|
587
|
|
bos@16
|
588 MQ helps you to work with the \sdirname{.hg/patches} directory as a
|
|
bos@16
|
589 repository; when you prepare a repository for working with patches
|
|
bos@17
|
590 using \hgcmd{qinit}, you can pass the \hgopt{qinit}{-c} option to
|
|
bos@16
|
591 create the \sdirname{.hg/patches} directory as a Mercurial repository.
|
|
bos@16
|
592
|
|
bos@16
|
593 \begin{note}
|
|
bos@16
|
594 If you forget to use the \hgopt{qinit}{-c} option, you can simply go
|
|
bos@16
|
595 into the \sdirname{.hg/patches} directory at any time and run
|
|
bos@16
|
596 \hgcmd{init}. Don't forget to add an entry for the
|
|
bos@17
|
597 \sfilename{status} file to the \sfilename{.hgignore} file, though
|
|
bos@17
|
598 (\hgcmdargs{qinit}{\hgopt{qinit}{-c}} does this for you
|
|
bos@17
|
599 automatically); you \emph{really} don't want to manage the
|
|
bos@17
|
600 \sfilename{status} file.
|
|
bos@16
|
601 \end{note}
|
|
bos@16
|
602
|
|
bos@16
|
603 As a convenience, if MQ notices that the \dirname{.hg/patches}
|
|
bos@16
|
604 directory is a repository, it will automatically \hgcmd{add} every
|
|
bos@16
|
605 patch that you create and import.
|
|
bos@16
|
606
|
|
bos@16
|
607 Finally, MQ provides a shortcut command, \hgcmd{qcommit}, that runs
|
|
bos@16
|
608 \hgcmd{commit} in the \sdirname{.hg/patches} directory. This saves
|
|
bos@16
|
609 some cumbersome typing.
|
|
bos@16
|
610
|
|
bos@16
|
611 \subsection{A few things to watch out for}
|
|
bos@16
|
612
|
|
bos@16
|
613 MQ's support for working with a repository full of patches is limited
|
|
bos@16
|
614 in a few small respects.
|
|
bos@16
|
615
|
|
bos@16
|
616 MQ cannot automatically detect changes that you make to the patch
|
|
bos@16
|
617 directory. If you \hgcmd{pull}, manually edit, or \hgcmd{update}
|
|
bos@16
|
618 changes to patches or the \sfilename{series} file, you will have to
|
|
bos@17
|
619 \hgcmdargs{qpop}{\hgopt{qpop}{-a}} and then
|
|
bos@17
|
620 \hgcmdargs{qpush}{\hgopt{qpush}{-a}} in the underlying repository to
|
|
bos@17
|
621 see those changes show up there. If you forget to do this, you can
|
|
bos@17
|
622 confuse MQ's idea of which patches are applied.
|
|
bos@16
|
623
|
|
bos@16
|
624 \section{Commands for working with patches}
|
|
bos@16
|
625
|
|
bos@16
|
626 Once you've been working with patches for a while, you'll find
|
|
bos@16
|
627 yourself hungry for tools that will help you to understand and
|
|
bos@16
|
628 manipulate the patches you're dealing with.
|
|
bos@16
|
629
|
|
bos@16
|
630 The \command{diffstat} command~\cite{web:diffstat} generates a
|
|
bos@16
|
631 histogram of the modifications made to each file in a patch. It
|
|
bos@16
|
632 provides a good way to ``get a sense of'' a patch--which files it
|
|
bos@16
|
633 affects, and how much change it introduces to each file and as a
|
|
bos@16
|
634 whole. (I find that it's a good idea to use \command{diffstat}'s
|
|
bos@16
|
635 \texttt{-p} option as a matter of course, as otherwise it will try to
|
|
bos@16
|
636 do clever things with prefixes of file names that inevitably confuse
|
|
bos@16
|
637 at least me.)
|
|
bos@16
|
638
|
|
bos@16
|
639 The \package{patchutils} package~\cite{web:patchutils} is invaluable.
|
|
bos@16
|
640 It provides a set of small utilities that follow the ``Unix
|
|
bos@16
|
641 philosophy;'' each does one useful thing with a patch. The
|
|
bos@16
|
642 \package{patchutils} command I use most is \command{filterdiff}, which
|
|
bos@16
|
643 extracts subsets from a patch file. For example, given a patch that
|
|
bos@16
|
644 modifies hundreds of files across dozens of directories, a single
|
|
bos@16
|
645 invocation of \command{filterdiff} can generate a smaller patch that
|
|
bos@16
|
646 only touches files whose names match a particular glob pattern.
|
|
bos@16
|
647
|
|
bos@1
|
648 %%% Local Variables:
|
|
bos@1
|
649 %%% mode: latex
|
|
bos@1
|
650 %%% TeX-master: "00book"
|
|
bos@1
|
651 %%% End:
|
