| rev |
line source |
|
bos@108
|
1 \chapter{Behind the scenes}
|
|
jeffpc@56
|
2 \label{chap:concepts}
|
|
jeffpc@56
|
3
|
|
bos@108
|
4 Unlike many revision control systems, the concepts upon which
|
|
bos@108
|
5 Mercurial is built are simple enough that it's easy to understand how
|
|
bos@108
|
6 the software really works. Knowing this certainly isn't necessary,
|
|
bos@108
|
7 but I find it useful to have a ``mental model'' of what's going on.
|
|
jeffpc@56
|
8
|
|
bos@109
|
9 This understanding gives me confidence that Mercurial has been
|
|
bos@109
|
10 carefully designed to be both \emph{safe} and \emph{efficient}. And
|
|
bos@111
|
11 just as importantly, if it's easy for me to retain a good idea of what
|
|
bos@111
|
12 the software is doing when I perform a revision control task, I'm less
|
|
bos@111
|
13 likely to be surprised by its behaviour.
|
|
bos@109
|
14
|
|
bos@112
|
15 In this chapter, we'll initially cover the core concepts behind
|
|
bos@112
|
16 Mercurial's design, then continue to discuss some of the interesting
|
|
bos@112
|
17 details of its implementation.
|
|
bos@112
|
18
|
|
bos@109
|
19 \section{Mercurial's historical record}
|
|
bos@109
|
20
|
|
bos@109
|
21 \subsection{Tracking the history of a single file}
|
|
jeffpc@56
|
22
|
|
bos@108
|
23 When Mercurial tracks modifications to a file, it stores the history
|
|
bos@108
|
24 of that file in a metadata object called a \emph{filelog}. Each entry
|
|
bos@108
|
25 in the filelog contains enough information to reconstruct one revision
|
|
bos@108
|
26 of the file that is being tracked. Filelogs are stored as files in
|
|
bos@244
|
27 the \sdirname{.hg/store/data} directory. A filelog contains two kinds
|
|
bos@244
|
28 of information: revision data, and an index to help Mercurial to find
|
|
bos@244
|
29 a revision efficiently.
|
|
jeffpc@56
|
30
|
|
bos@109
|
31 A file that is large, or has a lot of history, has its filelog stored
|
|
bos@109
|
32 in separate data (``\texttt{.d}'' suffix) and index (``\texttt{.i}''
|
|
bos@109
|
33 suffix) files. For small files without much history, the revision
|
|
bos@109
|
34 data and index are combined in a single ``\texttt{.i}'' file. The
|
|
bos@109
|
35 correspondence between a file in the working directory and the filelog
|
|
bos@109
|
36 that tracks its history in the repository is illustrated in
|
|
bos@109
|
37 figure~\ref{fig:concepts:filelog}.
|
|
jeffpc@56
|
38
|
|
jeffpc@56
|
39 \begin{figure}[ht]
|
|
bos@108
|
40 \centering
|
|
bos@108
|
41 \grafix{filelog}
|
|
bos@108
|
42 \caption{Relationships between files in working directory and
|
|
bos@108
|
43 filelogs in repository}
|
|
bos@108
|
44 \label{fig:concepts:filelog}
|
|
jeffpc@56
|
45 \end{figure}
|
|
jeffpc@56
|
46
|
|
bos@109
|
47 \subsection{Managing tracked files}
|
|
bos@109
|
48
|
|
bos@109
|
49 Mercurial uses a structure called a \emph{manifest} to collect
|
|
bos@109
|
50 together information about the files that it tracks. Each entry in
|
|
bos@109
|
51 the manifest contains information about the files present in a single
|
|
bos@109
|
52 changeset. An entry records which files are present in the changeset,
|
|
bos@109
|
53 the revision of each file, and a few other pieces of file metadata.
|
|
bos@109
|
54
|
|
bos@109
|
55 \subsection{Recording changeset information}
|
|
bos@109
|
56
|
|
bos@109
|
57 The \emph{changelog} contains information about each changeset. Each
|
|
bos@109
|
58 revision records who committed a change, the changeset comment, other
|
|
bos@109
|
59 pieces of changeset-related information, and the revision of the
|
|
bos@109
|
60 manifest to use.
|
|
bos@109
|
61
|
|
bos@109
|
62 \subsection{Relationships between revisions}
|
|
bos@109
|
63
|
|
bos@109
|
64 Within a changelog, a manifest, or a filelog, each revision stores a
|
|
bos@109
|
65 pointer to its immediate parent (or to its two parents, if it's a
|
|
bos@109
|
66 merge revision). As I mentioned above, there are also relationships
|
|
bos@109
|
67 between revisions \emph{across} these structures, and they are
|
|
bos@109
|
68 hierarchical in nature.
|
|
bos@109
|
69
|
|
bos@109
|
70 For every changeset in a repository, there is exactly one revision
|
|
bos@109
|
71 stored in the changelog. Each revision of the changelog contains a
|
|
bos@109
|
72 pointer to a single revision of the manifest. A revision of the
|
|
bos@109
|
73 manifest stores a pointer to a single revision of each filelog tracked
|
|
bos@109
|
74 when that changeset was created. These relationships are illustrated
|
|
bos@109
|
75 in figure~\ref{fig:concepts:metadata}.
|
|
bos@109
|
76
|
|
bos@109
|
77 \begin{figure}[ht]
|
|
bos@109
|
78 \centering
|
|
bos@109
|
79 \grafix{metadata}
|
|
bos@109
|
80 \caption{Metadata relationships}
|
|
bos@109
|
81 \label{fig:concepts:metadata}
|
|
bos@109
|
82 \end{figure}
|
|
bos@109
|
83
|
|
bos@110
|
84 As the illustration shows, there is \emph{not} a ``one to one''
|
|
bos@110
|
85 relationship between revisions in the changelog, manifest, or filelog.
|
|
bos@110
|
86 If the manifest hasn't changed between two changesets, the changelog
|
|
bos@110
|
87 entries for those changesets will point to the same revision of the
|
|
bos@110
|
88 manifest. If a file that Mercurial tracks hasn't changed between two
|
|
bos@110
|
89 changesets, the entry for that file in the two revisions of the
|
|
bos@110
|
90 manifest will point to the same revision of its filelog.
|
|
bos@110
|
91
|
|
bos@110
|
92 \section{Safe, efficient storage}
|
|
bos@109
|
93
|
|
bos@109
|
94 The underpinnings of changelogs, manifests, and filelogs are provided
|
|
bos@109
|
95 by a single structure called the \emph{revlog}.
|
|
bos@109
|
96
|
|
bos@109
|
97 \subsection{Efficient storage}
|
|
bos@109
|
98
|
|
bos@109
|
99 The revlog provides efficient storage of revisions using a
|
|
bos@109
|
100 \emph{delta} mechanism. Instead of storing a complete copy of a file
|
|
bos@109
|
101 for each revision, it stores the changes needed to transform an older
|
|
bos@109
|
102 revision into the new revision. For many kinds of file data, these
|
|
bos@109
|
103 deltas are typically a fraction of a percent of the size of a full
|
|
bos@109
|
104 copy of a file.
|
|
bos@109
|
105
|
|
bos@109
|
106 Some obsolete revision control systems can only work with deltas of
|
|
bos@109
|
107 text files. They must either store binary files as complete snapshots
|
|
bos@109
|
108 or encoded into a text representation, both of which are wasteful
|
|
bos@109
|
109 approaches. Mercurial can efficiently handle deltas of files with
|
|
bos@109
|
110 arbitrary binary contents; it doesn't need to treat text as special.
|
|
bos@109
|
111
|
|
bos@109
|
112 \subsection{Safe operation}
|
|
bos@121
|
113 \label{sec:concepts:txn}
|
|
bos@109
|
114
|
|
bos@109
|
115 Mercurial only ever \emph{appends} data to the end of a revlog file.
|
|
bos@109
|
116 It never modifies a section of a file after it has written it. This
|
|
bos@109
|
117 is both more robust and efficient than schemes that need to modify or
|
|
bos@109
|
118 rewrite data.
|
|
bos@109
|
119
|
|
bos@109
|
120 In addition, Mercurial treats every write as part of a
|
|
bos@109
|
121 \emph{transaction} that can span a number of files. A transaction is
|
|
bos@109
|
122 \emph{atomic}: either the entire transaction succeeds and its effects
|
|
bos@109
|
123 are all visible to readers in one go, or the whole thing is undone.
|
|
bos@109
|
124 This guarantee of atomicity means that if you're running two copies of
|
|
bos@109
|
125 Mercurial, where one is reading data and one is writing it, the reader
|
|
bos@109
|
126 will never see a partially written result that might confuse it.
|
|
bos@109
|
127
|
|
bos@109
|
128 The fact that Mercurial only appends to files makes it easier to
|
|
bos@109
|
129 provide this transactional guarantee. The easier it is to do stuff
|
|
bos@109
|
130 like this, the more confident you should be that it's done correctly.
|
|
bos@109
|
131
|
|
bos@109
|
132 \subsection{Fast retrieval}
|
|
bos@109
|
133
|
|
bos@109
|
134 Mercurial cleverly avoids a pitfall common to all earlier
|
|
bos@109
|
135 revision control systems: the problem of \emph{inefficient retrieval}.
|
|
bos@109
|
136 Most revision control systems store the contents of a revision as an
|
|
bos@109
|
137 incremental series of modifications against a ``snapshot''. To
|
|
bos@109
|
138 reconstruct a specific revision, you must first read the snapshot, and
|
|
bos@109
|
139 then every one of the revisions between the snapshot and your target
|
|
bos@109
|
140 revision. The more history that a file accumulates, the more
|
|
bos@109
|
141 revisions you must read, hence the longer it takes to reconstruct a
|
|
bos@109
|
142 particular revision.
|
|
bos@109
|
143
|
|
bos@110
|
144 \begin{figure}[ht]
|
|
bos@110
|
145 \centering
|
|
bos@110
|
146 \grafix{snapshot}
|
|
bos@110
|
147 \caption{Snapshot of a revlog, with incremental deltas}
|
|
bos@110
|
148 \label{fig:concepts:snapshot}
|
|
bos@110
|
149 \end{figure}
|
|
bos@110
|
150
|
|
bos@109
|
151 The innovation that Mercurial applies to this problem is simple but
|
|
bos@109
|
152 effective. Once the cumulative amount of delta information stored
|
|
bos@109
|
153 since the last snapshot exceeds a fixed threshold, it stores a new
|
|
bos@109
|
154 snapshot (compressed, of course), instead of another delta. This
|
|
bos@109
|
155 makes it possible to reconstruct \emph{any} revision of a file
|
|
bos@110
|
156 quickly. This approach works so well that it has since been copied by
|
|
bos@110
|
157 several other revision control systems.
|
|
bos@110
|
158
|
|
bos@110
|
159 Figure~\ref{fig:concepts:snapshot} illustrates the idea. In an entry
|
|
bos@110
|
160 in a revlog's index file, Mercurial stores the range of entries from
|
|
bos@110
|
161 the data file that it must read to reconstruct a particular revision.
|
|
bos@109
|
162
|
|
bos@109
|
163 \subsubsection{Aside: the influence of video compression}
|
|
bos@109
|
164
|
|
bos@109
|
165 If you're familiar with video compression or have ever watched a TV
|
|
bos@109
|
166 feed through a digital cable or satellite service, you may know that
|
|
bos@109
|
167 most video compression schemes store each frame of video as a delta
|
|
bos@109
|
168 against its predecessor frame. In addition, these schemes use
|
|
bos@109
|
169 ``lossy'' compression techniques to increase the compression ratio, so
|
|
bos@109
|
170 visual errors accumulate over the course of a number of inter-frame
|
|
bos@109
|
171 deltas.
|
|
bos@109
|
172
|
|
bos@109
|
173 Because it's possible for a video stream to ``drop out'' occasionally
|
|
bos@109
|
174 due to signal glitches, and to limit the accumulation of artefacts
|
|
bos@109
|
175 introduced by the lossy compression process, video encoders
|
|
bos@109
|
176 periodically insert a complete frame (called a ``key frame'') into the
|
|
bos@109
|
177 video stream; the next delta is generated against that frame. This
|
|
bos@109
|
178 means that if the video signal gets interrupted, it will resume once
|
|
bos@109
|
179 the next key frame is received. Also, the accumulation of encoding
|
|
bos@109
|
180 errors restarts anew with each key frame.
|
|
bos@109
|
181
|
|
bos@112
|
182 \subsection{Identification and strong integrity}
|
|
bos@109
|
183
|
|
bos@109
|
184 Along with delta or snapshot information, a revlog entry contains a
|
|
bos@109
|
185 cryptographic hash of the data that it represents. This makes it
|
|
bos@109
|
186 difficult to forge the contents of a revision, and easy to detect
|
|
bos@112
|
187 accidental corruption.
|
|
bos@112
|
188
|
|
bos@112
|
189 Hashes provide more than a mere check against corruption; they are
|
|
bos@112
|
190 used as the identifiers for revisions. The changeset identification
|
|
bos@111
|
191 hashes that you see as an end user are from revisions of the
|
|
bos@112
|
192 changelog. Although filelogs and the manifest also use hashes,
|
|
bos@112
|
193 Mercurial only uses these behind the scenes.
|
|
bos@112
|
194
|
|
bos@112
|
195 Mercurial verifies that hashes are correct when it retrieves file
|
|
bos@112
|
196 revisions and when it pulls changes from another repository. If it
|
|
bos@112
|
197 encounters an integrity problem, it will complain and stop whatever
|
|
bos@112
|
198 it's doing.
|
|
bos@109
|
199
|
|
bos@109
|
200 In addition to the effect it has on retrieval efficiency, Mercurial's
|
|
bos@109
|
201 use of periodic snapshots makes it more robust against partial data
|
|
bos@109
|
202 corruption. If a revlog becomes partly corrupted due to a hardware
|
|
bos@109
|
203 error or system bug, it's often possible to reconstruct some or most
|
|
bos@109
|
204 revisions from the uncorrupted sections of the revlog, both before and
|
|
bos@109
|
205 after the corrupted section. This would not be possible with a
|
|
bos@109
|
206 delta-only storage model.
|
|
bos@109
|
207
|
|
bos@115
|
208 \section{Revision history, branching,
|
|
bos@115
|
209 and merging}
|
|
bos@115
|
210
|
|
bos@115
|
211 Every entry in a Mercurial revlog knows the identity of its immediate
|
|
bos@115
|
212 ancestor revision, usually referred to as its \emph{parent}. In fact,
|
|
bos@115
|
213 a revision contains room for not one parent, but two. Mercurial uses
|
|
bos@115
|
214 a special hash, called the ``null ID'', to represent the idea ``there
|
|
bos@115
|
215 is no parent here''. This hash is simply a string of zeroes.
|
|
bos@115
|
216
|
|
bos@115
|
217 In figure~\ref{fig:concepts:revlog}, you can see an example of the
|
|
bos@115
|
218 conceptual structure of a revlog. Filelogs, manifests, and changelogs
|
|
bos@115
|
219 all have this same structure; they differ only in the kind of data
|
|
bos@115
|
220 stored in each delta or snapshot.
|
|
bos@115
|
221
|
|
bos@115
|
222 The first revision in a revlog (at the bottom of the image) has the
|
|
bos@115
|
223 null ID in both of its parent slots. For a ``normal'' revision, its
|
|
bos@115
|
224 first parent slot contains the ID of its parent revision, and its
|
|
bos@115
|
225 second contains the null ID, indicating that the revision has only one
|
|
bos@115
|
226 real parent. Any two revisions that have the same parent ID are
|
|
bos@115
|
227 branches. A revision that represents a merge between branches has two
|
|
bos@115
|
228 normal revision IDs in its parent slots.
|
|
bos@115
|
229
|
|
bos@115
|
230 \begin{figure}[ht]
|
|
bos@115
|
231 \centering
|
|
bos@115
|
232 \grafix{revlog}
|
|
bos@115
|
233 \caption{}
|
|
bos@115
|
234 \label{fig:concepts:revlog}
|
|
bos@115
|
235 \end{figure}
|
|
bos@115
|
236
|
|
bos@110
|
237 \section{The working directory}
|
|
bos@110
|
238
|
|
bos@113
|
239 In the working directory, Mercurial stores a snapshot of the files
|
|
bos@113
|
240 from the repository as of a particular changeset.
|
|
bos@113
|
241
|
|
bos@113
|
242 The working directory ``knows'' which changeset it contains. When you
|
|
bos@113
|
243 update the working directory to contain a particular changeset,
|
|
bos@113
|
244 Mercurial looks up the appropriate revision of the manifest to find
|
|
bos@113
|
245 out which files it was tracking at the time that changeset was
|
|
bos@113
|
246 committed, and which revision of each file was then current. It then
|
|
bos@113
|
247 recreates a copy of each of those files, with the same contents it had
|
|
bos@113
|
248 when the changeset was committed.
|
|
bos@113
|
249
|
|
bos@113
|
250 The \emph{dirstate} contains Mercurial's knowledge of the working
|
|
bos@113
|
251 directory. This details which changeset the working directory is
|
|
bos@113
|
252 updated to, and all of the files that Mercurial is tracking in the
|
|
bos@113
|
253 working directory.
|
|
bos@113
|
254
|
|
bos@113
|
255 Just as a revision of a revlog has room for two parents, so that it
|
|
bos@113
|
256 can represent either a normal revision (with one parent) or a merge of
|
|
bos@113
|
257 two earlier revisions, the dirstate has slots for two parents. When
|
|
bos@113
|
258 you use the \hgcmd{update} command, the changeset that you update to
|
|
bos@113
|
259 is stored in the ``first parent'' slot, and the null ID in the second.
|
|
bos@113
|
260 When you \hgcmd{merge} with another changeset, the first parent
|
|
bos@113
|
261 remains unchanged, and the second parent is filled in with the
|
|
bos@113
|
262 changeset you're merging with. The \hgcmd{parents} command tells you
|
|
bos@113
|
263 what the parents of the dirstate are.
|
|
bos@113
|
264
|
|
bos@113
|
265 \subsection{What happens when you commit}
|
|
bos@113
|
266
|
|
bos@113
|
267 The dirstate stores parent information for more than just book-keeping
|
|
bos@113
|
268 purposes. Mercurial uses the parents of the dirstate as \emph{the
|
|
bos@113
|
269 parents of a new changeset} when you perform a commit.
|
|
bos@113
|
270
|
|
bos@113
|
271 \begin{figure}[ht]
|
|
bos@113
|
272 \centering
|
|
bos@113
|
273 \grafix{wdir}
|
|
bos@113
|
274 \caption{The working directory can have two parents}
|
|
bos@113
|
275 \label{fig:concepts:wdir}
|
|
bos@113
|
276 \end{figure}
|
|
bos@113
|
277
|
|
bos@113
|
278 Figure~\ref{fig:concepts:wdir} shows the normal state of the working
|
|
bos@113
|
279 directory, where it has a single changeset as parent. That changeset
|
|
bos@113
|
280 is the \emph{tip}, the newest changeset in the repository that has no
|
|
bos@113
|
281 children.
|
|
bos@113
|
282
|
|
bos@113
|
283 \begin{figure}[ht]
|
|
bos@113
|
284 \centering
|
|
bos@113
|
285 \grafix{wdir-after-commit}
|
|
bos@113
|
286 \caption{The working directory gains new parents after a commit}
|
|
bos@113
|
287 \label{fig:concepts:wdir-after-commit}
|
|
bos@113
|
288 \end{figure}
|
|
bos@113
|
289
|
|
bos@113
|
290 It's useful to think of the working directory as ``the changeset I'm
|
|
bos@113
|
291 about to commit''. Any files that you tell Mercurial that you've
|
|
bos@113
|
292 added, removed, renamed, or copied will be reflected in that
|
|
bos@113
|
293 changeset, as will modifications to any files that Mercurial is
|
|
bos@113
|
294 already tracking; the new changeset will have the parents of the
|
|
bos@113
|
295 working directory as its parents.
|
|
bos@113
|
296
|
|
bos@113
|
297 After a commit, Mercurial will update the parents of the working
|
|
bos@113
|
298 directory, so that the first parent is the ID of the new changeset,
|
|
bos@115
|
299 and the second is the null ID. This is shown in
|
|
bos@115
|
300 figure~\ref{fig:concepts:wdir-after-commit}. Mercurial doesn't touch
|
|
bos@115
|
301 any of the files in the working directory when you commit; it just
|
|
bos@115
|
302 modifies the dirstate to note its new parents.
|
|
bos@115
|
303
|
|
bos@115
|
304 \subsection{Creating a new head}
|
|
bos@115
|
305
|
|
bos@115
|
306 It's perfectly normal to update the working directory to a changeset
|
|
bos@115
|
307 other than the current tip. For example, you might want to know what
|
|
bos@115
|
308 your project looked like last Tuesday, or you could be looking through
|
|
bos@115
|
309 changesets to see which one introduced a bug. In cases like this, the
|
|
bos@115
|
310 natural thing to do is update the working directory to the changeset
|
|
bos@115
|
311 you're interested in, and then examine the files in the working
|
|
bos@115
|
312 directory directly to see their contents as they werea when you
|
|
bos@115
|
313 committed that changeset. The effect of this is shown in
|
|
bos@115
|
314 figure~\ref{fig:concepts:wdir-pre-branch}.
|
|
bos@115
|
315
|
|
bos@115
|
316 \begin{figure}[ht]
|
|
bos@115
|
317 \centering
|
|
bos@115
|
318 \grafix{wdir-pre-branch}
|
|
bos@115
|
319 \caption{The working directory, updated to an older changeset}
|
|
bos@115
|
320 \label{fig:concepts:wdir-pre-branch}
|
|
bos@115
|
321 \end{figure}
|
|
bos@115
|
322
|
|
bos@115
|
323 Having updated the working directory to an older changeset, what
|
|
bos@115
|
324 happens if you make some changes, and then commit? Mercurial behaves
|
|
bos@115
|
325 in the same way as I outlined above. The parents of the working
|
|
bos@115
|
326 directory become the parents of the new changeset. This new changeset
|
|
bos@115
|
327 has no children, so it becomes the new tip. And the repository now
|
|
bos@115
|
328 contains two changesets that have no children; we call these
|
|
bos@115
|
329 \emph{heads}. You can see the structure that this creates in
|
|
bos@115
|
330 figure~\ref{fig:concepts:wdir-branch}.
|
|
bos@115
|
331
|
|
bos@115
|
332 \begin{figure}[ht]
|
|
bos@115
|
333 \centering
|
|
bos@115
|
334 \grafix{wdir-branch}
|
|
bos@115
|
335 \caption{After a commit made while synced to an older changeset}
|
|
bos@115
|
336 \label{fig:concepts:wdir-branch}
|
|
bos@115
|
337 \end{figure}
|
|
bos@115
|
338
|
|
bos@115
|
339 \begin{note}
|
|
bos@115
|
340 If you're new to Mercurial, you should keep in mind a common
|
|
bos@115
|
341 ``error'', which is to use the \hgcmd{pull} command without any
|
|
bos@115
|
342 options. By default, the \hgcmd{pull} command \emph{does not}
|
|
bos@115
|
343 update the working directory, so you'll bring new changesets into
|
|
bos@115
|
344 your repository, but the working directory will stay synced at the
|
|
bos@115
|
345 same changeset as before the pull. If you make some changes and
|
|
bos@115
|
346 commit afterwards, you'll thus create a new head, because your
|
|
bos@115
|
347 working directory isn't synced to whatever the current tip is.
|
|
bos@115
|
348
|
|
bos@115
|
349 I put the word ``error'' in quotes because all that you need to do
|
|
bos@115
|
350 to rectify this situation is \hgcmd{merge}, then \hgcmd{commit}. In
|
|
bos@115
|
351 other words, this almost never has negative consequences; it just
|
|
bos@115
|
352 surprises people. I'll discuss other ways to avoid this behaviour,
|
|
bos@115
|
353 and why Mercurial behaves in this initially surprising way, later
|
|
bos@115
|
354 on.
|
|
bos@115
|
355 \end{note}
|
|
bos@115
|
356
|
|
bos@115
|
357 \subsection{Merging heads}
|
|
bos@115
|
358
|
|
bos@115
|
359 When you run the \hgcmd{merge} command, Mercurial leaves the first
|
|
bos@115
|
360 parent of the working directory unchanged, and sets the second parent
|
|
bos@115
|
361 to the changeset you're merging with, as shown in
|
|
bos@115
|
362 figure~\ref{fig:concepts:wdir-merge}.
|
|
bos@115
|
363
|
|
bos@115
|
364 \begin{figure}[ht]
|
|
bos@115
|
365 \centering
|
|
bos@115
|
366 \grafix{wdir-merge}
|
|
bos@245
|
367 \caption{Merging two heads}
|
|
bos@115
|
368 \label{fig:concepts:wdir-merge}
|
|
bos@115
|
369 \end{figure}
|
|
bos@115
|
370
|
|
bos@115
|
371 Mercurial also has to modify the working directory, to merge the files
|
|
bos@115
|
372 managed in the two changesets. Simplified a little, the merging
|
|
bos@115
|
373 process goes like this, for every file in the manifests of both
|
|
bos@115
|
374 changesets.
|
|
bos@115
|
375 \begin{itemize}
|
|
bos@115
|
376 \item If neither changeset has modified a file, do nothing with that
|
|
bos@115
|
377 file.
|
|
bos@115
|
378 \item If one changeset has modified a file, and the other hasn't,
|
|
bos@115
|
379 create the modified copy of the file in the working directory.
|
|
bos@115
|
380 \item If one changeset has removed a file, and the other hasn't (or
|
|
bos@115
|
381 has also deleted it), delete the file from the working directory.
|
|
bos@115
|
382 \item If one changeset has removed a file, but the other has modified
|
|
bos@115
|
383 the file, ask the user what to do: keep the modified file, or remove
|
|
bos@115
|
384 it?
|
|
bos@115
|
385 \item If both changesets have modified a file, invoke an external
|
|
bos@115
|
386 merge program to choose the new contents for the merged file. This
|
|
bos@115
|
387 may require input from the user.
|
|
bos@115
|
388 \item If one changeset has modified a file, and the other has renamed
|
|
bos@115
|
389 or copied the file, make sure that the changes follow the new name
|
|
bos@115
|
390 of the file.
|
|
bos@115
|
391 \end{itemize}
|
|
bos@115
|
392 There are more details---merging has plenty of corner cases---but
|
|
bos@115
|
393 these are the most common choices that are involved in a merge. As
|
|
bos@115
|
394 you can see, most cases are completely automatic, and indeed most
|
|
bos@115
|
395 merges finish automatically, without requiring your input to resolve
|
|
bos@115
|
396 any conflicts.
|
|
bos@115
|
397
|
|
bos@115
|
398 When you're thinking about what happens when you commit after a merge,
|
|
bos@115
|
399 once again the working directory is ``the changeset I'm about to
|
|
bos@115
|
400 commit''. After the \hgcmd{merge} command completes, the working
|
|
bos@115
|
401 directory has two parents; these will become the parents of the new
|
|
bos@115
|
402 changeset.
|
|
bos@115
|
403
|
|
bos@115
|
404 Mercurial lets you perform multiple merges, but you must commit the
|
|
bos@115
|
405 results of each individual merge as you go. This is necessary because
|
|
bos@115
|
406 Mercurial only tracks two parents for both revisions and the working
|
|
bos@115
|
407 directory. While it would be technically possible to merge multiple
|
|
bos@115
|
408 changesets at once, the prospect of user confusion and making a
|
|
bos@115
|
409 terrible mess of a merge immediately becomes overwhelming.
|
|
bos@112
|
410
|
|
bos@110
|
411 \section{Other interesting design features}
|
|
bos@110
|
412
|
|
bos@110
|
413 In the sections above, I've tried to highlight some of the most
|
|
bos@110
|
414 important aspects of Mercurial's design, to illustrate that it pays
|
|
bos@110
|
415 careful attention to reliability and performance. However, the
|
|
bos@110
|
416 attention to detail doesn't stop there. There are a number of other
|
|
bos@110
|
417 aspects of Mercurial's construction that I personally find
|
|
bos@110
|
418 interesting. I'll detail a few of them here, separate from the ``big
|
|
bos@110
|
419 ticket'' items above, so that if you're interested, you can gain a
|
|
bos@110
|
420 better idea of the amount of thinking that goes into a well-designed
|
|
bos@110
|
421 system.
|
|
bos@110
|
422
|
|
bos@110
|
423 \subsection{Clever compression}
|
|
bos@110
|
424
|
|
bos@110
|
425 When appropriate, Mercurial will store both snapshots and deltas in
|
|
bos@110
|
426 compressed form. It does this by always \emph{trying to} compress a
|
|
bos@110
|
427 snapshot or delta, but only storing the compressed version if it's
|
|
bos@110
|
428 smaller than the uncompressed version.
|
|
bos@110
|
429
|
|
bos@110
|
430 This means that Mercurial does ``the right thing'' when storing a file
|
|
bos@110
|
431 whose native form is compressed, such as a \texttt{zip} archive or a
|
|
bos@110
|
432 JPEG image. When these types of files are compressed a second time,
|
|
bos@110
|
433 the resulting file is usually bigger than the once-compressed form,
|
|
bos@110
|
434 and so Mercurial will store the plain \texttt{zip} or JPEG.
|
|
bos@110
|
435
|
|
bos@110
|
436 Deltas between revisions of a compressed file are usually larger than
|
|
bos@110
|
437 snapshots of the file, and Mercurial again does ``the right thing'' in
|
|
bos@110
|
438 these cases. It finds that such a delta exceeds the threshold at
|
|
bos@110
|
439 which it should store a complete snapshot of the file, so it stores
|
|
bos@110
|
440 the snapshot, again saving space compared to a naive delta-only
|
|
bos@110
|
441 approach.
|
|
bos@110
|
442
|
|
bos@110
|
443 \subsubsection{Network recompression}
|
|
bos@110
|
444
|
|
bos@110
|
445 When storing revisions on disk, Mercurial uses the ``deflate''
|
|
bos@110
|
446 compression algorithm (the same one used by the popular \texttt{zip}
|
|
bos@110
|
447 archive format), which balances good speed with a respectable
|
|
bos@110
|
448 compression ratio. However, when transmitting revision data over a
|
|
bos@110
|
449 network connection, Mercurial uncompresses the compressed revision
|
|
bos@110
|
450 data.
|
|
bos@110
|
451
|
|
bos@110
|
452 If the connection is over HTTP, Mercurial recompresses the entire
|
|
bos@246
|
453 stream of data using a compression algorithm that gives a better
|
|
bos@110
|
454 compression ratio (the Burrows-Wheeler algorithm from the widely used
|
|
bos@110
|
455 \texttt{bzip2} compression package). This combination of algorithm
|
|
bos@110
|
456 and compression of the entire stream (instead of a revision at a time)
|
|
bos@110
|
457 substantially reduces the number of bytes to be transferred, yielding
|
|
bos@110
|
458 better network performance over almost all kinds of network.
|
|
bos@110
|
459
|
|
bos@110
|
460 (If the connection is over \command{ssh}, Mercurial \emph{doesn't}
|
|
bos@110
|
461 recompress the stream, because \command{ssh} can already do this
|
|
bos@110
|
462 itself.)
|
|
bos@110
|
463
|
|
bos@109
|
464 \subsection{Read/write ordering and atomicity}
|
|
bos@109
|
465
|
|
bos@109
|
466 Appending to files isn't the whole story when it comes to guaranteeing
|
|
bos@109
|
467 that a reader won't see a partial write. If you recall
|
|
bos@109
|
468 figure~\ref{fig:concepts:metadata}, revisions in the changelog point to
|
|
bos@109
|
469 revisions in the manifest, and revisions in the manifest point to
|
|
bos@109
|
470 revisions in filelogs. This hierarchy is deliberate.
|
|
bos@109
|
471
|
|
bos@109
|
472 A writer starts a transaction by writing filelog and manifest data,
|
|
bos@109
|
473 and doesn't write any changelog data until those are finished. A
|
|
bos@109
|
474 reader starts by reading changelog data, then manifest data, followed
|
|
bos@109
|
475 by filelog data.
|
|
bos@109
|
476
|
|
bos@109
|
477 Since the writer has always finished writing filelog and manifest data
|
|
bos@109
|
478 before it writes to the changelog, a reader will never read a pointer
|
|
bos@109
|
479 to a partially written manifest revision from the changelog, and it will
|
|
bos@109
|
480 never read a pointer to a partially written filelog revision from the
|
|
bos@109
|
481 manifest.
|
|
bos@109
|
482
|
|
bos@109
|
483 \subsection{Concurrent access}
|
|
bos@109
|
484
|
|
bos@109
|
485 The read/write ordering and atomicity guarantees mean that Mercurial
|
|
bos@109
|
486 never needs to \emph{lock} a repository when it's reading data, even
|
|
bos@109
|
487 if the repository is being written to while the read is occurring.
|
|
bos@109
|
488 This has a big effect on scalability; you can have an arbitrary number
|
|
bos@109
|
489 of Mercurial processes safely reading data from a repository safely
|
|
bos@109
|
490 all at once, no matter whether it's being written to or not.
|
|
bos@109
|
491
|
|
bos@109
|
492 The lockless nature of reading means that if you're sharing a
|
|
bos@109
|
493 repository on a multi-user system, you don't need to grant other local
|
|
bos@109
|
494 users permission to \emph{write} to your repository in order for them
|
|
bos@109
|
495 to be able to clone it or pull changes from it; they only need
|
|
bos@109
|
496 \emph{read} permission. (This is \emph{not} a common feature among
|
|
bos@109
|
497 revision control systems, so don't take it for granted! Most require
|
|
bos@109
|
498 readers to be able to lock a repository to access it safely, and this
|
|
bos@109
|
499 requires write permission on at least one directory, which of course
|
|
bos@109
|
500 makes for all kinds of nasty and annoying security and administrative
|
|
bos@109
|
501 problems.)
|
|
bos@109
|
502
|
|
bos@110
|
503 Mercurial uses locks to ensure that only one process can write to a
|
|
bos@110
|
504 repository at a time (the locking mechanism is safe even over
|
|
bos@110
|
505 filesystems that are notoriously hostile to locking, such as NFS). If
|
|
bos@110
|
506 a repository is locked, a writer will wait for a while to retry if the
|
|
bos@110
|
507 repository becomes unlocked, but if the repository remains locked for
|
|
bos@110
|
508 too long, the process attempting to write will time out after a while.
|
|
bos@110
|
509 This means that your daily automated scripts won't get stuck forever
|
|
bos@110
|
510 and pile up if a system crashes unnoticed, for example. (Yes, the
|
|
bos@110
|
511 timeout is configurable, from zero to infinity.)
|
|
bos@110
|
512
|
|
bos@110
|
513 \subsubsection{Safe dirstate access}
|
|
bos@110
|
514
|
|
bos@110
|
515 As with revision data, Mercurial doesn't take a lock to read the
|
|
bos@110
|
516 dirstate file; it does acquire a lock to write it. To avoid the
|
|
bos@110
|
517 possibility of reading a partially written copy of the dirstate file,
|
|
bos@110
|
518 Mercurial writes to a file with a unique name in the same directory as
|
|
bos@110
|
519 the dirstate file, then renames the temporary file atomically to
|
|
bos@110
|
520 \filename{dirstate}. The file named \filename{dirstate} is thus
|
|
bos@110
|
521 guaranteed to be complete, not partially written.
|
|
bos@109
|
522
|
|
bos@111
|
523 \subsection{Avoiding seeks}
|
|
bos@111
|
524
|
|
bos@111
|
525 Critical to Mercurial's performance is the avoidance of seeks of the
|
|
bos@111
|
526 disk head, since any seek is far more expensive than even a
|
|
bos@111
|
527 comparatively large read operation.
|
|
bos@111
|
528
|
|
bos@111
|
529 This is why, for example, the dirstate is stored in a single file. If
|
|
bos@111
|
530 there were a dirstate file per directory that Mercurial tracked, the
|
|
bos@111
|
531 disk would seek once per directory. Instead, Mercurial reads the
|
|
bos@111
|
532 entire single dirstate file in one step.
|
|
bos@111
|
533
|
|
bos@111
|
534 Mercurial also uses a ``copy on write'' scheme when cloning a
|
|
bos@111
|
535 repository on local storage. Instead of copying every revlog file
|
|
bos@111
|
536 from the old repository into the new repository, it makes a ``hard
|
|
bos@111
|
537 link'', which is a shorthand way to say ``these two names point to the
|
|
bos@111
|
538 same file''. When Mercurial is about to write to one of a revlog's
|
|
bos@111
|
539 files, it checks to see if the number of names pointing at the file is
|
|
bos@111
|
540 greater than one. If it is, more than one repository is using the
|
|
bos@111
|
541 file, so Mercurial makes a new copy of the file that is private to
|
|
bos@111
|
542 this repository.
|
|
bos@111
|
543
|
|
bos@111
|
544 A few revision control developers have pointed out that this idea of
|
|
bos@111
|
545 making a complete private copy of a file is not very efficient in its
|
|
bos@111
|
546 use of storage. While this is true, storage is cheap, and this method
|
|
bos@111
|
547 gives the highest performance while deferring most book-keeping to the
|
|
bos@111
|
548 operating system. An alternative scheme would most likely reduce
|
|
bos@111
|
549 performance and increase the complexity of the software, each of which
|
|
bos@111
|
550 is much more important to the ``feel'' of day-to-day use.
|
|
bos@109
|
551
|
|
bos@115
|
552 \subsection{Other contents of the dirstate}
|
|
bos@115
|
553
|
|
bos@115
|
554 Because Mercurial doesn't force you to tell it when you're modifying a
|
|
bos@115
|
555 file, it uses the dirstate to store some extra information so it can
|
|
bos@115
|
556 determine efficiently whether you have modified a file. For each file
|
|
bos@115
|
557 in the working directory, it stores the time that it last modified the
|
|
bos@115
|
558 file itself, and the size of the file at that time.
|
|
bos@115
|
559
|
|
bos@115
|
560 When you explicitly \hgcmd{add}, \hgcmd{remove}, \hgcmd{rename} or
|
|
bos@116
|
561 \hgcmd{copy} files, Mercurial updates the dirstate so that it knows
|
|
bos@116
|
562 what to do with those files when you commit.
|
|
bos@115
|
563
|
|
bos@115
|
564 When Mercurial is checking the states of files in the working
|
|
bos@115
|
565 directory, it first checks a file's modification time. If that has
|
|
bos@115
|
566 not changed, the file must not have been modified. If the file's size
|
|
bos@115
|
567 has changed, the file must have been modified. If the modification
|
|
bos@115
|
568 time has changed, but the size has not, only then does Mercurial need
|
|
bos@115
|
569 to read the actual contents of the file to see if they've changed.
|
|
bos@115
|
570 Storing these few extra pieces of information dramatically reduces the
|
|
bos@115
|
571 amount of data that Mercurial needs to read, which yields large
|
|
bos@115
|
572 performance improvements compared to other revision control systems.
|
|
bos@115
|
573
|
|
jeffpc@56
|
574 %%% Local Variables:
|
|
jeffpc@56
|
575 %%% mode: latex
|
|
jeffpc@56
|
576 %%% TeX-master: "00book"
|
|
jeffpc@56
|
577 %%% End:
|
