hgbook
view en/ch14-hgext.xml @ 563:44d1363234d2
Move example output files into examples/results
| author | Bryan O'Sullivan <bos@serpentine.com> |
|---|---|
| date | Mon Mar 09 21:37:47 2009 -0700 (2009-03-09) |
| parents | b90b024729f1 |
| children | 8fcd44708f41 |
line source
1 <!-- vim: set filetype=docbkxml shiftwidth=2 autoindent expandtab tw=77 : -->
3 <chapter id="chap:hgext">
4 <title>Adding functionality with extensions</title>
6 <para>While the core of Mercurial is quite complete from a
7 functionality standpoint, it's deliberately shorn of fancy
8 features. This approach of preserving simplicity keeps the
9 software easy to deal with for both maintainers and users.</para>
11 <para>However, Mercurial doesn't box you in with an inflexible
12 command set: you can add features to it as
13 <emphasis>extensions</emphasis> (sometimes known as
14 <emphasis>plugins</emphasis>). We've already discussed a few of
15 these extensions in earlier chapters.</para>
16 <itemizedlist>
17 <listitem><para>Section <xref linkend="sec:tour-merge:fetch"/>
18 covers the <literal role="hg-ext">fetch</literal> extension;
19 this combines pulling new changes and merging them with local
20 changes into a single command, <command
21 role="hg-ext-fetch">fetch</command>.</para>
22 </listitem>
23 <listitem><para>In chapter <xref linkend="chap:hook"/>, we covered
24 several extensions that are useful for hook-related
25 functionality: <literal role="hg-ext">acl</literal> adds
26 access control lists; <literal
27 role="hg-ext">bugzilla</literal> adds integration with the
28 Bugzilla bug tracking system; and <literal
29 role="hg-ext">notify</literal> sends notification emails on
30 new changes.</para>
31 </listitem>
32 <listitem><para>The Mercurial Queues patch management extension is
33 so invaluable that it merits two chapters and an appendix all
34 to itself. Chapter <xref linkend="chap:mq"/> covers the
35 basics; chapter <xref
36 linkend="chap:mq-collab"/> discusses advanced topics;
37 and appendix <xref linkend="chap:mqref"/> goes into detail on
38 each
39 command.</para>
40 </listitem></itemizedlist>
42 <para>In this chapter, we'll cover some of the other extensions that
43 are available for Mercurial, and briefly touch on some of the
44 machinery you'll need to know about if you want to write an
45 extension of your own.</para>
46 <itemizedlist>
47 <listitem><para>In section <xref linkend="sec:hgext:inotify"/>,
48 we'll discuss the possibility of <emphasis>huge</emphasis>
49 performance improvements using the <literal
50 role="hg-ext">inotify</literal> extension.</para>
51 </listitem></itemizedlist>
53 <sect1 id="sec:hgext:inotify">
54 <title>Improve performance with the <literal
55 role="hg-ext">inotify</literal> extension</title>
57 <para>Are you interested in having some of the most common
58 Mercurial operations run as much as a hundred times faster?
59 Read on!</para>
61 <para>Mercurial has great performance under normal circumstances.
62 For example, when you run the <command role="hg-cmd">hg
63 status</command> command, Mercurial has to scan almost every
64 directory and file in your repository so that it can display
65 file status. Many other Mercurial commands need to do the same
66 work behind the scenes; for example, the <command
67 role="hg-cmd">hg diff</command> command uses the status
68 machinery to avoid doing an expensive comparison operation on
69 files that obviously haven't changed.</para>
71 <para>Because obtaining file status is crucial to good
72 performance, the authors of Mercurial have optimised this code
73 to within an inch of its life. However, there's no avoiding the
74 fact that when you run <command role="hg-cmd">hg
75 status</command>, Mercurial is going to have to perform at
76 least one expensive system call for each managed file to
77 determine whether it's changed since the last time Mercurial
78 checked. For a sufficiently large repository, this can take a
79 long time.</para>
81 <para>To put a number on the magnitude of this effect, I created a
82 repository containing 150,000 managed files. I timed <command
83 role="hg-cmd">hg status</command> as taking ten seconds to
84 run, even when <emphasis>none</emphasis> of those files had been
85 modified.</para>
87 <para>Many modern operating systems contain a file notification
88 facility. If a program signs up to an appropriate service, the
89 operating system will notify it every time a file of interest is
90 created, modified, or deleted. On Linux systems, the kernel
91 component that does this is called
92 <literal>inotify</literal>.</para>
94 <para>Mercurial's <literal role="hg-ext">inotify</literal>
95 extension talks to the kernel's <literal>inotify</literal>
96 component to optimise <command role="hg-cmd">hg status</command>
97 commands. The extension has two components. A daemon sits in
98 the background and receives notifications from the
99 <literal>inotify</literal> subsystem. It also listens for
100 connections from a regular Mercurial command. The extension
101 modifies Mercurial's behaviour so that instead of scanning the
102 filesystem, it queries the daemon. Since the daemon has perfect
103 information about the state of the repository, it can respond
104 with a result instantaneously, avoiding the need to scan every
105 directory and file in the repository.</para>
107 <para>Recall the ten seconds that I measured plain Mercurial as
108 taking to run <command role="hg-cmd">hg status</command> on a
109 150,000 file repository. With the <literal
110 role="hg-ext">inotify</literal> extension enabled, the time
111 dropped to 0.1 seconds, a factor of <emphasis>one
112 hundred</emphasis> faster.</para>
114 <para>Before we continue, please pay attention to some
115 caveats.</para>
116 <itemizedlist>
117 <listitem><para>The <literal role="hg-ext">inotify</literal>
118 extension is Linux-specific. Because it interfaces directly
119 to the Linux kernel's <literal>inotify</literal> subsystem,
120 it does not work on other operating systems.</para>
121 </listitem>
122 <listitem><para>It should work on any Linux distribution that
123 was released after early 2005. Older distributions are
124 likely to have a kernel that lacks
125 <literal>inotify</literal>, or a version of
126 <literal>glibc</literal> that does not have the necessary
127 interfacing support.</para>
128 </listitem>
129 <listitem><para>Not all filesystems are suitable for use with
130 the <literal role="hg-ext">inotify</literal> extension.
131 Network filesystems such as NFS are a non-starter, for
132 example, particularly if you're running Mercurial on several
133 systems, all mounting the same network filesystem. The
134 kernel's <literal>inotify</literal> system has no way of
135 knowing about changes made on another system. Most local
136 filesystems (e.g. ext3, XFS, ReiserFS) should work
137 fine.</para>
138 </listitem></itemizedlist>
140 <para>The <literal role="hg-ext">inotify</literal> extension is
141 not yet shipped with Mercurial as of May 2007, so it's a little
142 more involved to set up than other extensions. But the
143 performance improvement is worth it!</para>
145 <para>The extension currently comes in two parts: a set of patches
146 to the Mercurial source code, and a library of Python bindings
147 to the <literal>inotify</literal> subsystem.</para>
148 <note>
149 <para> There are <emphasis>two</emphasis> Python
150 <literal>inotify</literal> binding libraries. One of them is
151 called <literal>pyinotify</literal>, and is packaged by some
152 Linux distributions as <literal>python-inotify</literal>.
153 This is <emphasis>not</emphasis> the one you'll need, as it is
154 too buggy and inefficient to be practical.</para>
155 </note>
156 <para>To get going, it's best to already have a functioning copy
157 of Mercurial installed.</para>
158 <note>
159 <para> If you follow the instructions below, you'll be
160 <emphasis>replacing</emphasis> and overwriting any existing
161 installation of Mercurial that you might already have, using
162 the latest <quote>bleeding edge</quote> Mercurial code. Don't
163 say you weren't warned!</para>
164 </note>
165 <orderedlist>
166 <listitem><para>Clone the Python <literal>inotify</literal>
167 binding repository. Build and install it.</para>
168 <programlisting>
169 hg clone http://hg.kublai.com/python/inotify cd inotify
170 python setup.py build --force sudo python setup.py install
171 --skip-build
172 </programlisting>
173 </listitem>
174 <listitem><para>Clone the <filename
175 class="directory">crew</filename> Mercurial repository.
176 Clone the <literal role="hg-ext">inotify</literal> patch
177 repository so that Mercurial Queues will be able to apply
178 patches to your cope of the <filename
179 class="directory">crew</filename> repository.</para>
180 <programlisting>
181 hg clone http://hg.intevation.org/mercurial/crew hg clone
182 crew inotify hg clone
183 http://hg.kublai.com/mercurial/patches/inotify
184 inotify/.hg/patches
185 </programlisting>
186 </listitem>
187 <listitem><para>Make sure that you have the Mercurial Queues
188 extension, <literal role="hg-ext">mq</literal>, enabled. If
189 you've never used MQ, read section <xref
190 linkend="sec:mq:start"/> to get started
191 quickly.</para>
192 </listitem>
193 <listitem><para>Go into the <filename
194 class="directory">inotify</filename> repo, and apply all
195 of the <literal role="hg-ext">inotify</literal> patches
196 using the <option role="hg-ext-mq-cmd-qpush-opt">hg
197 -a</option> option to the <command
198 role="hg-ext-mq">qpush</command> command.</para>
199 <programlisting>
200 cd inotify hg qpush -a
201 </programlisting>
202 </listitem>
203 <listitem><para> If you get an error message from <command
204 role="hg-ext-mq">qpush</command>, you should not continue.
205 Instead, ask for help.</para>
206 </listitem>
207 <listitem><para>Build and install the patched version of
208 Mercurial.</para>
209 <programlisting>
210 python setup.py build --force sudo python setup.py install
211 --skip-build
212 </programlisting>
213 </listitem>
214 </orderedlist>
215 <para>Once you've build a suitably patched version of Mercurial,
216 all you need to do to enable the <literal
217 role="hg-ext">inotify</literal> extension is add an entry to
218 your <filename role="special"> /.hgrc</filename>.</para>
219 <programlisting>[extensions] inotify =</programlisting>
220 <para>When the <literal role="hg-ext">inotify</literal> extension
221 is enabled, Mercurial will automatically and transparently start
222 the status daemon the first time you run a command that needs
223 status in a repository. It runs one status daemon per
224 repository.</para>
226 <para>The status daemon is started silently, and runs in the
227 background. If you look at a list of running processes after
228 you've enabled the <literal role="hg-ext">inotify</literal>
229 extension and run a few commands in different repositories,
230 you'll thus see a few <literal>hg</literal> processes sitting
231 around, waiting for updates from the kernel and queries from
232 Mercurial.</para>
234 <para>The first time you run a Mercurial command in a repository
235 when you have the <literal role="hg-ext">inotify</literal>
236 extension enabled, it will run with about the same performance
237 as a normal Mercurial command. This is because the status
238 daemon needs to perform a normal status scan so that it has a
239 baseline against which to apply later updates from the kernel.
240 However, <emphasis>every</emphasis> subsequent command that does
241 any kind of status check should be noticeably faster on
242 repositories of even fairly modest size. Better yet, the bigger
243 your repository is, the greater a performance advantage you'll
244 see. The <literal role="hg-ext">inotify</literal> daemon makes
245 status operations almost instantaneous on repositories of all
246 sizes!</para>
248 <para>If you like, you can manually start a status daemon using
249 the <command role="hg-ext-inotify">inserve</command> command.
250 This gives you slightly finer control over how the daemon ought
251 to run. This command will of course only be available when the
252 <literal role="hg-ext">inotify</literal> extension is
253 enabled.</para>
255 <para>When you're using the <literal
256 role="hg-ext">inotify</literal> extension, you should notice
257 <emphasis>no difference at all</emphasis> in Mercurial's
258 behaviour, with the sole exception of status-related commands
259 running a whole lot faster than they used to. You should
260 specifically expect that commands will not print different
261 output; neither should they give different results. If either of
262 these situations occurs, please report a bug.</para>
264 </sect1>
265 <sect1 id="sec:hgext:extdiff">
266 <title>Flexible diff support with the <literal
267 role="hg-ext">extdiff</literal> extension</title>
269 <para>Mercurial's built-in <command role="hg-cmd">hg
270 diff</command> command outputs plaintext unified diffs. <!--
271 &interaction.extdiff.diff; --> If you would like to use an
272 external tool to display modifications, you'll want to use the
273 <literal role="hg-ext">extdiff</literal> extension. This will
274 let you use, for example, a graphical diff tool.</para>
276 <para>The <literal role="hg-ext">extdiff</literal> extension is
277 bundled with Mercurial, so it's easy to set up. In the <literal
278 role="rc-extensions">extensions</literal> section of your
279 <filename role="special"> /.hgrc</filename>, simply add a
280 one-line entry to enable the extension.</para>
281 <programlisting>[extensions] extdiff =</programlisting>
282 <para>This introduces a command named <command
283 role="hg-ext-extdiff">extdiff</command>, which by default uses
284 your system's <command>diff</command> command to generate a
285 unified diff in the same form as the built-in <command
286 role="hg-cmd">hg diff</command> command. <!--
287 &interaction.extdiff.extdiff; --> The result won't be exactly
288 the same as with the built-in <command role="hg-cmd">hg
289 diff</command> variations, because the output of
290 <command>diff</command> varies from one system to another, even
291 when passed the same options.</para>
293 <para>As the <quote><literal>making snapshot</literal></quote>
294 lines of output above imply, the <command
295 role="hg-ext-extdiff">extdiff</command> command works by
296 creating two snapshots of your source tree. The first snapshot
297 is of the source revision; the second, of the target revision or
298 working directory. The <command
299 role="hg-ext-extdiff">extdiff</command> command generates
300 these snapshots in a temporary directory, passes the name of
301 each directory to an external diff viewer, then deletes the
302 temporary directory. For efficiency, it only snapshots the
303 directories and files that have changed between the two
304 revisions.</para>
306 <para>Snapshot directory names have the same base name as your
307 repository. If your repository path is <filename
308 class="directory">/quux/bar/foo</filename>, then <filename
309 class="directory">foo</filename> will be the name of each
310 snapshot directory. Each snapshot directory name has its
311 changeset ID appended, if appropriate. If a snapshot is of
312 revision <literal>a631aca1083f</literal>, the directory will be
313 named <filename class="directory">foo.a631aca1083f</filename>.
314 A snapshot of the working directory won't have a changeset ID
315 appended, so it would just be <filename
316 class="directory">foo</filename> in this example. To see what
317 this looks like in practice, look again at the <command
318 role="hg-ext-extdiff">extdiff</command> example above. Notice
319 that the diff has the snapshot directory names embedded in its
320 header.</para>
322 <para>The <command role="hg-ext-extdiff">extdiff</command> command
323 accepts two important options. The <option
324 role="hg-ext-extdiff-cmd-extdiff-opt">hg -p</option> option
325 lets you choose a program to view differences with, instead of
326 <command>diff</command>. With the <option
327 role="hg-ext-extdiff-cmd-extdiff-opt">hg -o</option> option,
328 you can change the options that <command
329 role="hg-ext-extdiff">extdiff</command> passes to the program
330 (by default, these options are
331 <quote><literal>-Npru</literal></quote>, which only make sense
332 if you're running <command>diff</command>). In other respects,
333 the <command role="hg-ext-extdiff">extdiff</command> command
334 acts similarly to the built-in <command role="hg-cmd">hg
335 diff</command> command: you use the same option names, syntax,
336 and arguments to specify the revisions you want, the files you
337 want, and so on.</para>
339 <para>As an example, here's how to run the normal system
340 <command>diff</command> command, getting it to generate context
341 diffs (using the <option role="cmd-opt-diff">-c</option> option)
342 instead of unified diffs, and five lines of context instead of
343 the default three (passing <literal>5</literal> as the argument
344 to the <option role="cmd-opt-diff">-C</option> option). <!--
345 &interaction.extdiff.extdiff-ctx; --></para>
347 <para>Launching a visual diff tool is just as easy. Here's how to
348 launch the <command>kdiff3</command> viewer.</para>
349 <programlisting>hg extdiff -p kdiff3 -o</programlisting>
351 <para>If your diff viewing command can't deal with directories,
352 you can easily work around this with a little scripting. For an
353 example of such scripting in action with the <literal
354 role="hg-ext">mq</literal> extension and the
355 <command>interdiff</command> command, see section <xref
356 linkend="mq-collab:tips:interdiff"/>.</para>
358 <sect2>
359 <title>Defining command aliases</title>
361 <para>It can be cumbersome to remember the options to both the
362 <command role="hg-ext-extdiff">extdiff</command> command and
363 the diff viewer you want to use, so the <literal
364 role="hg-ext">extdiff</literal> extension lets you define
365 <emphasis>new</emphasis> commands that will invoke your diff
366 viewer with exactly the right options.</para>
368 <para>All you need to do is edit your <filename role="special">
369 /.hgrc</filename>, and add a section named <literal
370 role="rc-extdiff">extdiff</literal>. Inside this section,
371 you can define multiple commands. Here's how to add a
372 <literal>kdiff3</literal> command. Once you've defined this,
373 you can type <quote><literal>hg kdiff3</literal></quote> and
374 the <literal role="hg-ext">extdiff</literal> extension will
375 run <command>kdiff3</command> for you.</para>
376 <programlisting>[extdiff] cmd.kdiff3 =</programlisting>
377 <para>If you leave the right hand side of the definition empty,
378 as above, the <literal role="hg-ext">extdiff</literal>
379 extension uses the name of the command you defined as the name
380 of the external program to run. But these names don't have to
381 be the same. Here, we define a command named
382 <quote><literal>hg wibble</literal></quote>, which runs
383 <command>kdiff3</command>.</para>
384 <programlisting>[extdiff] cmd.wibble = kdiff3</programlisting>
386 <para>You can also specify the default options that you want to
387 invoke your diff viewing program with. The prefix to use is
388 <quote><literal>opts.</literal></quote>, followed by the name
389 of the command to which the options apply. This example
390 defines a <quote><literal>hg vimdiff</literal></quote> command
391 that runs the <command>vim</command> editor's
392 <literal>DirDiff</literal> extension.</para>
393 <programlisting>[extdiff] cmd.vimdiff = vim opts.vimdiff = -f
394 '+next' '+execute "DirDiff" argv(0) argv(1)'</programlisting>
396 </sect2>
397 </sect1>
398 <sect1 id="sec:hgext:transplant">
399 <title>Cherrypicking changes with the <literal
400 role="hg-ext">transplant</literal> extension</title>
402 <para>Need to have a long chat with Brendan about this.</para>
404 </sect1>
405 <sect1 id="sec:hgext:patchbomb">
406 <title>Send changes via email with the <literal
407 role="hg-ext">patchbomb</literal> extension</title>
409 <para>Many projects have a culture of <quote>change
410 review</quote>, in which people send their modifications to a
411 mailing list for others to read and comment on before they
412 commit the final version to a shared repository. Some projects
413 have people who act as gatekeepers; they apply changes from
414 other people to a repository to which those others don't have
415 access.</para>
417 <para>Mercurial makes it easy to send changes over email for
418 review or application, via its <literal
419 role="hg-ext">patchbomb</literal> extension. The extension is
420 so named because changes are formatted as patches, and it's usual
421 to send one changeset per email message. Sending a long series
422 of changes by email is thus much like <quote>bombing</quote> the
423 recipient's inbox, hence <quote>patchbomb</quote>.</para>
425 <para>As usual, the basic configuration of the <literal
426 role="hg-ext">patchbomb</literal> extension takes just one or
427 two lines in your <filename role="special">
428 /.hgrc</filename>.</para>
429 <programlisting>[extensions] patchbomb =</programlisting>
430 <para>Once you've enabled the extension, you will have a new
431 command available, named <command
432 role="hg-ext-patchbomb">email</command>.</para>
434 <para>The safest and best way to invoke the <command
435 role="hg-ext-patchbomb">email</command> command is to
436 <emphasis>always</emphasis> run it first with the <option
437 role="hg-ext-patchbomb-cmd-email-opt">hg -n</option> option.
438 This will show you what the command <emphasis>would</emphasis>
439 send, without actually sending anything. Once you've had a
440 quick glance over the changes and verified that you are sending
441 the right ones, you can rerun the same command, with the <option
442 role="hg-ext-patchbomb-cmd-email-opt">hg -n</option> option
443 removed.</para>
445 <para>The <command role="hg-ext-patchbomb">email</command> command
446 accepts the same kind of revision syntax as every other
447 Mercurial command. For example, this command will send every
448 revision between 7 and <literal>tip</literal>, inclusive.</para>
449 <programlisting>hg email -n 7:tip</programlisting>
450 <para>You can also specify a <emphasis>repository</emphasis> to
451 compare with. If you provide a repository but no revisions, the
452 <command role="hg-ext-patchbomb">email</command> command will
453 send all revisions in the local repository that are not present
454 in the remote repository. If you additionally specify revisions
455 or a branch name (the latter using the <option
456 role="hg-ext-patchbomb-cmd-email-opt">hg -b</option> option),
457 this will constrain the revisions sent.</para>
459 <para>It's perfectly safe to run the <command
460 role="hg-ext-patchbomb">email</command> command without the
461 names of the people you want to send to: if you do this, it will
462 just prompt you for those values interactively. (If you're
463 using a Linux or Unix-like system, you should have enhanced
464 <literal>readline</literal>-style editing capabilities when
465 entering those headers, too, which is useful.)</para>
467 <para>When you are sending just one revision, the <command
468 role="hg-ext-patchbomb">email</command> command will by
469 default use the first line of the changeset description as the
470 subject of the single email message it sends.</para>
472 <para>If you send multiple revisions, the <command
473 role="hg-ext-patchbomb">email</command> command will usually
474 send one message per changeset. It will preface the series with
475 an introductory message, in which you should describe the
476 purpose of the series of changes you're sending.</para>
478 <sect2>
479 <title>Changing the behaviour of patchbombs</title>
481 <para>Not every project has exactly the same conventions for
482 sending changes in email; the <literal
483 role="hg-ext">patchbomb</literal> extension tries to
484 accommodate a number of variations through command line
485 options.</para>
486 <itemizedlist>
487 <listitem><para>You can write a subject for the introductory
488 message on the command line using the <option
489 role="hg-ext-patchbomb-cmd-email-opt">hg -s</option>
490 option. This takes one argument, the text of the subject
491 to use.</para>
492 </listitem>
493 <listitem><para>To change the email address from which the
494 messages originate, use the <option
495 role="hg-ext-patchbomb-cmd-email-opt">hg -f</option>
496 option. This takes one argument, the email address to
497 use.</para>
498 </listitem>
499 <listitem><para>The default behaviour is to send unified diffs
500 (see section <xref linkend="sec:mq:patch"/> for a
501 description of the
502 format), one per message. You can send a binary bundle
503 instead with the <option
504 role="hg-ext-patchbomb-cmd-email-opt">hg -b</option>
505 option.</para>
506 </listitem>
507 <listitem><para>Unified diffs are normally prefaced with a
508 metadata header. You can omit this, and send unadorned
509 diffs, with the <option
510 role="hg-ext-patchbomb-cmd-email-opt">hg
511 --plain</option> option.</para>
512 </listitem>
513 <listitem><para>Diffs are normally sent <quote>inline</quote>,
514 in the same body part as the description of a patch. This
515 makes it easiest for the largest number of readers to
516 quote and respond to parts of a diff, as some mail clients
517 will only quote the first MIME body part in a message. If
518 you'd prefer to send the description and the diff in
519 separate body parts, use the <option
520 role="hg-ext-patchbomb-cmd-email-opt">hg -a</option>
521 option.</para>
522 </listitem>
523 <listitem><para>Instead of sending mail messages, you can
524 write them to an <literal>mbox</literal>-format mail
525 folder using the <option
526 role="hg-ext-patchbomb-cmd-email-opt">hg -m</option>
527 option. That option takes one argument, the name of the
528 file to write to.</para>
529 </listitem>
530 <listitem><para>If you would like to add a
531 <command>diffstat</command>-format summary to each patch,
532 and one to the introductory message, use the <option
533 role="hg-ext-patchbomb-cmd-email-opt">hg -d</option>
534 option. The <command>diffstat</command> command displays
535 a table containing the name of each file patched, the
536 number of lines affected, and a histogram showing how much
537 each file is modified. This gives readers a qualitative
538 glance at how complex a patch is.</para>
539 </listitem></itemizedlist>
541 </sect2>
542 </sect1>
543 </chapter>
545 <!--
546 local variables:
547 sgml-parent-document: ("00book.xml" "book" "chapter")
548 end:
549 -->