| rev |
line source |
|
bos@104
|
1 \chapter{Advanced uses of Mercurial Queues}
|
|
bos@104
|
2
|
|
bos@104
|
3 While it's easy to pick up straightforward uses of Mercurial Queues,
|
|
bos@104
|
4 use of a little discipline and some of MQ's less frequently used
|
|
bos@104
|
5 capabilities makes it possible to work in complicated development
|
|
bos@104
|
6 environments.
|
|
bos@104
|
7
|
|
bos@105
|
8 In this chapter, I will use as an example a technique I have used to
|
|
bos@105
|
9 manage the development of an Infiniband device driver for the Linux
|
|
bos@105
|
10 kernel. The driver in question is large (at least as drivers go),
|
|
bos@105
|
11 with 25,000 lines of code spread across 35 source files. It is
|
|
bos@105
|
12 maintained by a small team of developers.
|
|
bos@104
|
13
|
|
bos@104
|
14 While much of the material in this chapter is specific to Linux, the
|
|
bos@104
|
15 same principles apply to any code base for which you're not the
|
|
bos@104
|
16 primary owner, and upon which you need to do a lot of development.
|
|
bos@104
|
17
|
|
bos@104
|
18 \section{The problem of many targets}
|
|
bos@104
|
19
|
|
bos@104
|
20 The Linux kernel changes rapidly, and has never been internally
|
|
bos@104
|
21 stable; developers frequently make drastic changes between releases.
|
|
bos@104
|
22 This means that a version of the driver that works well with a
|
|
bos@104
|
23 particular released version of the kernel will not even \emph{compile}
|
|
bos@104
|
24 correctly against, typically, any other version.
|
|
bos@104
|
25
|
|
bos@104
|
26 To maintain a driver, we have to keep a number of distinct versions of
|
|
bos@104
|
27 Linux in mind.
|
|
bos@104
|
28 \begin{itemize}
|
|
bos@104
|
29 \item One target is the main Linux kernel development tree.
|
|
bos@104
|
30 Maintenance of the code is in this case partly shared by other
|
|
bos@104
|
31 developers in the kernel community, who make ``drive-by''
|
|
bos@104
|
32 modifications to the driver as they develop and refine kernel
|
|
bos@104
|
33 subsystems.
|
|
bos@104
|
34 \item We also maintain a number of ``backports'' to older versions of
|
|
bos@104
|
35 the Linux kernel, to support the needs of customers who are running
|
|
bos@105
|
36 older Linux distributions that do not incorporate our drivers. (To
|
|
bos@105
|
37 \emph{backport} a piece of code is to modify it to work in an older
|
|
bos@105
|
38 version of its target environment than the version it was developed
|
|
bos@105
|
39 for.)
|
|
bos@104
|
40 \item Finally, we make software releases on a schedule that is
|
|
bos@104
|
41 necessarily not aligned with those used by Linux distributors and
|
|
bos@104
|
42 kernel developers, so that we can deliver new features to customers
|
|
bos@104
|
43 without forcing them to upgrade their entire kernels or
|
|
bos@104
|
44 distributions.
|
|
bos@104
|
45 \end{itemize}
|
|
bos@104
|
46
|
|
bos@104
|
47 \subsection{Tempting approaches that don't work well}
|
|
bos@104
|
48
|
|
bos@104
|
49 There are two ``standard'' ways to maintain a piece of software that
|
|
bos@104
|
50 has to target many different environments.
|
|
bos@104
|
51
|
|
bos@104
|
52 The first is to maintain a number of branches, each intended for a
|
|
bos@104
|
53 single target. The trouble with this approach is that you must
|
|
bos@104
|
54 maintain iron discipline in the flow of changes between repositories.
|
|
bos@104
|
55 A new feature or bug fix must start life in a ``pristine'' repository,
|
|
bos@104
|
56 then percolate out to every backport repository. Backport changes are
|
|
bos@104
|
57 more limited in the branches they should propagate to; a backport
|
|
bos@104
|
58 change that is applied to a branch where it doesn't belong will
|
|
bos@104
|
59 probably stop the driver from compiling.
|
|
bos@104
|
60
|
|
bos@104
|
61 The second is to maintain a single source tree filled with conditional
|
|
bos@104
|
62 statements that turn chunks of code on or off depending on the
|
|
bos@104
|
63 intended target. Because these ``ifdefs'' are not allowed in the
|
|
bos@104
|
64 Linux kernel tree, a manual or automatic process must be followed to
|
|
bos@104
|
65 strip them out and yield a clean tree. A code base maintained in this
|
|
bos@104
|
66 fashion rapidly becomes a rat's nest of conditional blocks that are
|
|
bos@104
|
67 difficult to understand and maintain.
|
|
bos@104
|
68
|
|
bos@104
|
69 Neither of these approaches is well suited to a situation where you
|
|
bos@104
|
70 don't ``own'' the canonical copy of a source tree. In the case of a
|
|
bos@104
|
71 Linux driver that is distributed with the standard kernel, Linus's
|
|
bos@104
|
72 tree contains the copy of the code that will be treated by the world
|
|
bos@104
|
73 as canonical. The upstream version of ``my'' driver can be modified
|
|
bos@104
|
74 by people I don't know, without me even finding out about it until
|
|
bos@104
|
75 after the changes show up in Linus's tree.
|
|
bos@104
|
76
|
|
bos@104
|
77 These approaches have the added weakness of making it difficult to
|
|
bos@104
|
78 generate well-formed patches to submit upstream.
|
|
bos@104
|
79
|
|
bos@104
|
80 In principle, Mercurial Queues seems like a good candidate to manage a
|
|
bos@104
|
81 development scenario such as the above. While this is indeed the
|
|
bos@104
|
82 case, MQ contains a few added features that make the job more
|
|
bos@104
|
83 pleasant.
|
|
bos@104
|
84
|
|
bos@105
|
85 \section{Conditionally applying patches with
|
|
bos@105
|
86 guards}
|
|
bos@104
|
87
|
|
bos@104
|
88 Perhaps the best way to maintain sanity with so many targets is to be
|
|
bos@104
|
89 able to choose specific patches to apply for a given situation. MQ
|
|
bos@104
|
90 provides a feature called ``guards'' (which originates with quilt's
|
|
bos@104
|
91 \texttt{guards} command) that does just this. To start off, let's
|
|
bos@104
|
92 create a simple repository for experimenting in.
|
|
bos@104
|
93 \interaction{mq.guards.init}
|
|
bos@104
|
94 This gives us a tiny repository that contains two patches that don't
|
|
bos@104
|
95 have any dependencies on each other, because they touch different files.
|
|
bos@104
|
96
|
|
bos@104
|
97 The idea behind conditional application is that you can ``tag'' a
|
|
bos@104
|
98 patch with a \emph{guard}, which is simply a text string of your
|
|
bos@104
|
99 choosing, then tell MQ to select specific guards to use when applying
|
|
bos@104
|
100 patches. MQ will then either apply, or skip over, a guarded patch,
|
|
bos@104
|
101 depending on the guards that you have selected.
|
|
bos@104
|
102
|
|
bos@104
|
103 A patch can have an arbitrary number of guards;
|
|
bos@104
|
104 each one is \emph{positive} (``apply this patch if this guard is
|
|
bos@104
|
105 selected'') or \emph{negative} (``skip this patch if this guard is
|
|
bos@104
|
106 selected''). A patch with no guards is always applied.
|
|
bos@104
|
107
|
|
bos@104
|
108 \section{Controlling the guards on a patch}
|
|
bos@104
|
109
|
|
bos@104
|
110 The \hgcmd{qguard} command lets you determine which guards should
|
|
bos@104
|
111 apply to a patch, or display the guards that are already in effect.
|
|
bos@104
|
112 Without any arguments, it displays the guards on the current topmost
|
|
bos@104
|
113 patch.
|
|
bos@104
|
114 \interaction{mq.guards.qguard}
|
|
bos@104
|
115 To set a positive guard on a patch, prefix the name of the guard with
|
|
bos@104
|
116 a ``\texttt{+}''.
|
|
bos@104
|
117 \interaction{mq.guards.qguard.pos}
|
|
bos@104
|
118 To set a negative guard on a patch, prefix the name of the guard with
|
|
bos@104
|
119 a ``\texttt{-}''.
|
|
bos@104
|
120 \interaction{mq.guards.qguard.neg}
|
|
bos@104
|
121
|
|
bos@104
|
122 \begin{note}
|
|
bos@104
|
123 The \hgcmd{qguard} command \emph{sets} the guards on a patch; it
|
|
bos@104
|
124 doesn't \emph{modify} them. What this means is that if you run
|
|
bos@104
|
125 \hgcmdargs{qguard}{+a +b} on a patch, then \hgcmdargs{qguard}{+c} on
|
|
bos@104
|
126 the same patch, the \emph{only} guard that will be set on it
|
|
bos@104
|
127 afterwards is \texttt{+c}.
|
|
bos@104
|
128 \end{note}
|
|
bos@104
|
129
|
|
bos@104
|
130 Mercurial stores guards in the \sfilename{series} file; the form in
|
|
bos@104
|
131 which they are stored is easy both to understand and to edit by hand.
|
|
bos@104
|
132 (In other words, you don't have to use the \hgcmd{qguard} command if
|
|
bos@104
|
133 you don't want to; it's okay to simply edit the \sfilename{series}
|
|
bos@104
|
134 file.)
|
|
bos@104
|
135 \interaction{mq.guards.series}
|
|
bos@104
|
136
|
|
bos@104
|
137 \section{Selecting the guards to use}
|
|
bos@104
|
138
|
|
bos@104
|
139 The \hgcmd{qselect} command determines which guards are active at a
|
|
bos@104
|
140 given time. The effect of this is to determine which patches MQ will
|
|
bos@104
|
141 apply the next time you run \hgcmd{qpush}. It has no other effect; in
|
|
bos@104
|
142 particular, it doesn't do anything to patches that are already
|
|
bos@104
|
143 applied.
|
|
bos@104
|
144
|
|
bos@104
|
145 With no arguments, the \hgcmd{qselect} command lists the guards
|
|
bos@104
|
146 currently in effect, one per line of output. Each argument is treated
|
|
bos@104
|
147 as the name of a guard to apply.
|
|
bos@104
|
148 \interaction{mq.guards.qselect.foo}
|
|
bos@104
|
149 In case you're interested, the currently selected guards are stored in
|
|
bos@104
|
150 the \sfilename{guards} file.
|
|
bos@104
|
151 \interaction{mq.guards.qselect.cat}
|
|
bos@104
|
152 We can see the effect the selected guards have when we run
|
|
bos@104
|
153 \hgcmd{qpush}.
|
|
bos@104
|
154 \interaction{mq.guards.qselect.qpush}
|
|
bos@104
|
155
|
|
bos@104
|
156 A guard cannot start with a ``\texttt{+}'' or ``\texttt{-}''
|
|
bos@105
|
157 character. The name of a guard must start with an alphabetic
|
|
bos@105
|
158 character (upper or lower case) or an underscore. The rest of the
|
|
bos@105
|
159 guard's name can contain any of these characters, or a digit. These
|
|
bos@105
|
160 rules are similar to those used for variable naming in most popular
|
|
bos@105
|
161 programming languages. If you try to use a guard with an invalid
|
|
bos@105
|
162 name, MQ will complain:
|
|
bos@104
|
163 \interaction{mq.guards.qselect.error}
|
|
bos@104
|
164 Changing the selected guards changes the patches that are applied.
|
|
bos@104
|
165 \interaction{mq.guards.qselect.quux}
|
|
bos@105
|
166 You can see in the example below that negative guards take precedence
|
|
bos@105
|
167 over positive guards.
|
|
bos@104
|
168 \interaction{mq.guards.qselect.foobar}
|
|
bos@104
|
169
|
|
bos@105
|
170 \section{MQ's rules for applying patches}
|
|
bos@105
|
171
|
|
bos@105
|
172 The rules that MQ uses when deciding whether to apply a patch
|
|
bos@105
|
173 are as follows.
|
|
bos@105
|
174 \begin{itemize}
|
|
bos@105
|
175 \item A patch that has no guards is always applied.
|
|
bos@105
|
176 \item If the patch has any negative guard that matches any currently
|
|
bos@105
|
177 selected guard, the patch is skipped.
|
|
bos@105
|
178 \item If the patch has any positive guard that matches any currently
|
|
bos@105
|
179 selected guard, the patch is applied.
|
|
bos@105
|
180 \item If the patch has positive or negative guards, but none matches
|
|
bos@105
|
181 any currently selected guard, the patch is skipped.
|
|
bos@105
|
182 \end{itemize}
|
|
bos@105
|
183
|
|
bos@105
|
184 \section{Trimming the work environment}
|
|
bos@105
|
185
|
|
bos@105
|
186 In working on the device driver I mentioned earlier, I don't apply the
|
|
bos@105
|
187 patches to a normal Linux kernel tree. Instead, I use a repository
|
|
bos@105
|
188 that contains only a snapshot of the source files and headers that are
|
|
bos@105
|
189 relevant to Infiniband development. This repository is~1\% the size
|
|
bos@105
|
190 of a kernel repository, so it's easier to work with.
|
|
bos@105
|
191
|
|
bos@105
|
192 I then choose a ``base'' version on top of which the patches are
|
|
bos@105
|
193 applied. This is a snapshot of the Linux kernel tree as of a revision
|
|
bos@105
|
194 of my choosing. When I take the snapshot, I record the changeset ID
|
|
bos@105
|
195 from the kernel repository in the commit message. Since the snapshot
|
|
bos@105
|
196 preserves the ``shape'' and content of the relevant parts of the
|
|
bos@105
|
197 kernel tree, I can apply my patches on top of either my tiny
|
|
bos@105
|
198 repository or a normal kernel tree.
|
|
bos@105
|
199
|
|
bos@105
|
200 Normally, the base tree atop which the patches apply should be a
|
|
bos@105
|
201 snapshot of a very recent upstream tree. This best facilitates the
|
|
bos@105
|
202 development of patches that can easily be submitted upstream with few
|
|
bos@105
|
203 or no modifications.
|
|
bos@105
|
204
|
|
bos@105
|
205 \section{Dividing up the \sfilename{series} file}
|
|
bos@105
|
206
|
|
bos@105
|
207 I categorise the patches in the \sfilename{series} file into a number
|
|
bos@105
|
208 of logical groups. Each section of like patches begins with a block
|
|
bos@105
|
209 of comments that describes the purpose of the patches that follow.
|
|
bos@105
|
210
|
|
bos@105
|
211 The sequence of patch groups that I maintain follows. The ordering of
|
|
bos@105
|
212 these groups is important; I'll describe why after I introduce the
|
|
bos@105
|
213 groups.
|
|
bos@105
|
214 \begin{itemize}
|
|
bos@105
|
215 \item The ``accepted'' group. Patches that the development team has
|
|
bos@105
|
216 submitted to the maintainer of the Infiniband subsystem, and which
|
|
bos@105
|
217 he has accepted, but which are not present in the snapshot that the
|
|
bos@105
|
218 tiny repository is based on. These are ``read only'' patches,
|
|
bos@105
|
219 present only to transform the tree into a similar state as it is in
|
|
bos@105
|
220 the upstream maintainer's repository.
|
|
bos@105
|
221 \item The ``rework'' group. Patches that I have submitted, but that
|
|
bos@105
|
222 the upstream maintainer has requested modifications to before he
|
|
bos@105
|
223 will accept them.
|
|
bos@105
|
224 \item The ``pending'' group. Patches that I have not yet submitted to
|
|
bos@105
|
225 the upstream maintainer, but which we have finished working on.
|
|
bos@105
|
226 These will be ``read only'' for a while. If the upstream maintainer
|
|
bos@105
|
227 accepts them upon submission, I'll move them to the end of the
|
|
bos@105
|
228 ``accepted'' group. If he requests that I modify any, I'll move
|
|
bos@105
|
229 them to the beginning of the ``rework'' group.
|
|
bos@105
|
230 \item The ``in progress'' group. Patches that are actively being
|
|
bos@105
|
231 developed, and should not be submitted anywhere yet.
|
|
bos@105
|
232 \item The ``backport'' group. Patches that adapt the source tree to
|
|
bos@105
|
233 older versions of the kernel tree.
|
|
bos@105
|
234 \item The ``do not ship'' group. Patches that for some reason should
|
|
bos@105
|
235 never be submitted upstream. For example, one such patch might
|
|
bos@105
|
236 change embedded driver identification strings to make it easier to
|
|
bos@105
|
237 distinguish, in the field, between an out-of-tree version of the
|
|
bos@105
|
238 driver and a version shipped by a distribution vendor.
|
|
bos@105
|
239 \end{itemize}
|
|
bos@105
|
240
|
|
bos@105
|
241 Now to return to the reasons for ordering groups of patches in this
|
|
bos@105
|
242 way. We would like the lowest patches in the stack to be as stable as
|
|
bos@105
|
243 possible, so that we will not need to rework higher patches due to
|
|
bos@105
|
244 changes in context. Putting patches that will never be changed first
|
|
bos@105
|
245 in the \sfilename{series} file serves this purpose.
|
|
bos@105
|
246
|
|
bos@105
|
247 We would also like the patches that we know we'll need to modify to be
|
|
bos@105
|
248 applied on top of a source tree that resembles the upstream tree as
|
|
bos@105
|
249 closely as possible. This is why we keep accepted patches around for
|
|
bos@105
|
250 a while.
|
|
bos@105
|
251
|
|
bos@105
|
252 The ``backport'' and ``do not ship'' patches float at the end of the
|
|
bos@105
|
253 \sfilename{series} file in part because they'll never be shipped
|
|
bos@105
|
254 upstream. Additionally, the backport patches must be applied on top
|
|
bos@105
|
255 of all other patches.
|
|
bos@105
|
256
|
|
bos@104
|
257 %%% Local Variables:
|
|
bos@104
|
258 %%% mode: latex
|
|
bos@104
|
259 %%% TeX-master: "00book"
|
|
bos@104
|
260 %%% End:
|
