hgbook
view en/ch14-hgext.xml @ 567:8fcd44708f41
Uncomment all the mangled interaction examples.
| author | Bryan O'Sullivan <bos@serpentine.com> |
|---|---|
| date | Mon Mar 09 23:22:09 2009 -0700 (2009-03-09) |
| parents | 21c62e09b99f |
| children | 13513d2a128d |
line source
1 <!-- vim: set filetype=docbkxml shiftwidth=2 autoindent expandtab tw=77 : -->
3 <chapter id="chap:hgext">
4 <title>Adding functionality with extensions</title>
6 <para>While the core of Mercurial is quite complete from a
7 functionality standpoint, it's deliberately shorn of fancy
8 features. This approach of preserving simplicity keeps the
9 software easy to deal with for both maintainers and users.</para>
11 <para>However, Mercurial doesn't box you in with an inflexible
12 command set: you can add features to it as
13 <emphasis>extensions</emphasis> (sometimes known as
14 <emphasis>plugins</emphasis>). We've already discussed a few of
15 these extensions in earlier chapters.</para>
16 <itemizedlist>
17 <listitem><para>Section <xref linkend="sec:tour-merge:fetch"/>
18 covers the <literal role="hg-ext">fetch</literal> extension;
19 this combines pulling new changes and merging them with local
20 changes into a single command, <command
21 role="hg-ext-fetch">fetch</command>.</para>
22 </listitem>
23 <listitem><para>In chapter <xref linkend="chap:hook"/>, we covered
24 several extensions that are useful for hook-related
25 functionality: <literal role="hg-ext">acl</literal> adds
26 access control lists; <literal
27 role="hg-ext">bugzilla</literal> adds integration with the
28 Bugzilla bug tracking system; and <literal
29 role="hg-ext">notify</literal> sends notification emails on
30 new changes.</para>
31 </listitem>
32 <listitem><para>The Mercurial Queues patch management extension is
33 so invaluable that it merits two chapters and an appendix all
34 to itself. Chapter <xref linkend="chap:mq"/> covers the
35 basics; chapter <xref
36 linkend="chap:mq-collab"/> discusses advanced topics;
37 and appendix <xref linkend="chap:mqref"/> goes into detail on
38 each
39 command.</para>
40 </listitem></itemizedlist>
42 <para>In this chapter, we'll cover some of the other extensions that
43 are available for Mercurial, and briefly touch on some of the
44 machinery you'll need to know about if you want to write an
45 extension of your own.</para>
46 <itemizedlist>
47 <listitem><para>In section <xref linkend="sec:hgext:inotify"/>,
48 we'll discuss the possibility of <emphasis>huge</emphasis>
49 performance improvements using the <literal
50 role="hg-ext">inotify</literal> extension.</para>
51 </listitem></itemizedlist>
53 <sect1 id="sec:hgext:inotify">
54 <title>Improve performance with the <literal
55 role="hg-ext">inotify</literal> extension</title>
57 <para>Are you interested in having some of the most common
58 Mercurial operations run as much as a hundred times faster?
59 Read on!</para>
61 <para>Mercurial has great performance under normal circumstances.
62 For example, when you run the <command role="hg-cmd">hg
63 status</command> command, Mercurial has to scan almost every
64 directory and file in your repository so that it can display
65 file status. Many other Mercurial commands need to do the same
66 work behind the scenes; for example, the <command
67 role="hg-cmd">hg diff</command> command uses the status
68 machinery to avoid doing an expensive comparison operation on
69 files that obviously haven't changed.</para>
71 <para>Because obtaining file status is crucial to good
72 performance, the authors of Mercurial have optimised this code
73 to within an inch of its life. However, there's no avoiding the
74 fact that when you run <command role="hg-cmd">hg
75 status</command>, Mercurial is going to have to perform at
76 least one expensive system call for each managed file to
77 determine whether it's changed since the last time Mercurial
78 checked. For a sufficiently large repository, this can take a
79 long time.</para>
81 <para>To put a number on the magnitude of this effect, I created a
82 repository containing 150,000 managed files. I timed <command
83 role="hg-cmd">hg status</command> as taking ten seconds to
84 run, even when <emphasis>none</emphasis> of those files had been
85 modified.</para>
87 <para>Many modern operating systems contain a file notification
88 facility. If a program signs up to an appropriate service, the
89 operating system will notify it every time a file of interest is
90 created, modified, or deleted. On Linux systems, the kernel
91 component that does this is called
92 <literal>inotify</literal>.</para>
94 <para>Mercurial's <literal role="hg-ext">inotify</literal>
95 extension talks to the kernel's <literal>inotify</literal>
96 component to optimise <command role="hg-cmd">hg status</command>
97 commands. The extension has two components. A daemon sits in
98 the background and receives notifications from the
99 <literal>inotify</literal> subsystem. It also listens for
100 connections from a regular Mercurial command. The extension
101 modifies Mercurial's behaviour so that instead of scanning the
102 filesystem, it queries the daemon. Since the daemon has perfect
103 information about the state of the repository, it can respond
104 with a result instantaneously, avoiding the need to scan every
105 directory and file in the repository.</para>
107 <para>Recall the ten seconds that I measured plain Mercurial as
108 taking to run <command role="hg-cmd">hg status</command> on a
109 150,000 file repository. With the <literal
110 role="hg-ext">inotify</literal> extension enabled, the time
111 dropped to 0.1 seconds, a factor of <emphasis>one
112 hundred</emphasis> faster.</para>
114 <para>Before we continue, please pay attention to some
115 caveats.</para>
116 <itemizedlist>
117 <listitem><para>The <literal role="hg-ext">inotify</literal>
118 extension is Linux-specific. Because it interfaces directly
119 to the Linux kernel's <literal>inotify</literal> subsystem,
120 it does not work on other operating systems.</para>
121 </listitem>
122 <listitem><para>It should work on any Linux distribution that
123 was released after early 2005. Older distributions are
124 likely to have a kernel that lacks
125 <literal>inotify</literal>, or a version of
126 <literal>glibc</literal> that does not have the necessary
127 interfacing support.</para>
128 </listitem>
129 <listitem><para>Not all filesystems are suitable for use with
130 the <literal role="hg-ext">inotify</literal> extension.
131 Network filesystems such as NFS are a non-starter, for
132 example, particularly if you're running Mercurial on several
133 systems, all mounting the same network filesystem. The
134 kernel's <literal>inotify</literal> system has no way of
135 knowing about changes made on another system. Most local
136 filesystems (e.g. ext3, XFS, ReiserFS) should work
137 fine.</para>
138 </listitem></itemizedlist>
140 <para>The <literal role="hg-ext">inotify</literal> extension is
141 not yet shipped with Mercurial as of May 2007, so it's a little
142 more involved to set up than other extensions. But the
143 performance improvement is worth it!</para>
145 <para>The extension currently comes in two parts: a set of patches
146 to the Mercurial source code, and a library of Python bindings
147 to the <literal>inotify</literal> subsystem.</para>
148 <note>
149 <para> There are <emphasis>two</emphasis> Python
150 <literal>inotify</literal> binding libraries. One of them is
151 called <literal>pyinotify</literal>, and is packaged by some
152 Linux distributions as <literal>python-inotify</literal>.
153 This is <emphasis>not</emphasis> the one you'll need, as it is
154 too buggy and inefficient to be practical.</para>
155 </note>
156 <para>To get going, it's best to already have a functioning copy
157 of Mercurial installed.</para>
158 <note>
159 <para> If you follow the instructions below, you'll be
160 <emphasis>replacing</emphasis> and overwriting any existing
161 installation of Mercurial that you might already have, using
162 the latest <quote>bleeding edge</quote> Mercurial code. Don't
163 say you weren't warned!</para>
164 </note>
165 <orderedlist>
166 <listitem><para>Clone the Python <literal>inotify</literal>
167 binding repository. Build and install it.</para>
168 <programlisting>
169 hg clone http://hg.kublai.com/python/inotify cd inotify
170 python setup.py build --force sudo python setup.py install
171 --skip-build
172 </programlisting>
173 </listitem>
174 <listitem><para>Clone the <filename
175 class="directory">crew</filename> Mercurial repository.
176 Clone the <literal role="hg-ext">inotify</literal> patch
177 repository so that Mercurial Queues will be able to apply
178 patches to your cope of the <filename
179 class="directory">crew</filename> repository.</para>
180 <programlisting>
181 hg clone http://hg.intevation.org/mercurial/crew hg clone
182 crew inotify hg clone
183 http://hg.kublai.com/mercurial/patches/inotify
184 inotify/.hg/patches
185 </programlisting>
186 </listitem>
187 <listitem><para>Make sure that you have the Mercurial Queues
188 extension, <literal role="hg-ext">mq</literal>, enabled. If
189 you've never used MQ, read section <xref
190 linkend="sec:mq:start"/> to get started
191 quickly.</para>
192 </listitem>
193 <listitem><para>Go into the <filename
194 class="directory">inotify</filename> repo, and apply all
195 of the <literal role="hg-ext">inotify</literal> patches
196 using the <option role="hg-ext-mq-cmd-qpush-opt">hg
197 -a</option> option to the <command
198 role="hg-ext-mq">qpush</command> command.</para>
199 <programlisting>
200 cd inotify hg qpush -a
201 </programlisting>
202 </listitem>
203 <listitem><para> If you get an error message from <command
204 role="hg-ext-mq">qpush</command>, you should not continue.
205 Instead, ask for help.</para>
206 </listitem>
207 <listitem><para>Build and install the patched version of
208 Mercurial.</para>
209 <programlisting>
210 python setup.py build --force sudo python setup.py install
211 --skip-build
212 </programlisting>
213 </listitem>
214 </orderedlist>
215 <para>Once you've build a suitably patched version of Mercurial,
216 all you need to do to enable the <literal
217 role="hg-ext">inotify</literal> extension is add an entry to
218 your <filename role="special"> /.hgrc</filename>.</para>
219 <programlisting>[extensions] inotify =</programlisting>
220 <para>When the <literal role="hg-ext">inotify</literal> extension
221 is enabled, Mercurial will automatically and transparently start
222 the status daemon the first time you run a command that needs
223 status in a repository. It runs one status daemon per
224 repository.</para>
226 <para>The status daemon is started silently, and runs in the
227 background. If you look at a list of running processes after
228 you've enabled the <literal role="hg-ext">inotify</literal>
229 extension and run a few commands in different repositories,
230 you'll thus see a few <literal>hg</literal> processes sitting
231 around, waiting for updates from the kernel and queries from
232 Mercurial.</para>
234 <para>The first time you run a Mercurial command in a repository
235 when you have the <literal role="hg-ext">inotify</literal>
236 extension enabled, it will run with about the same performance
237 as a normal Mercurial command. This is because the status
238 daemon needs to perform a normal status scan so that it has a
239 baseline against which to apply later updates from the kernel.
240 However, <emphasis>every</emphasis> subsequent command that does
241 any kind of status check should be noticeably faster on
242 repositories of even fairly modest size. Better yet, the bigger
243 your repository is, the greater a performance advantage you'll
244 see. The <literal role="hg-ext">inotify</literal> daemon makes
245 status operations almost instantaneous on repositories of all
246 sizes!</para>
248 <para>If you like, you can manually start a status daemon using
249 the <command role="hg-ext-inotify">inserve</command> command.
250 This gives you slightly finer control over how the daemon ought
251 to run. This command will of course only be available when the
252 <literal role="hg-ext">inotify</literal> extension is
253 enabled.</para>
255 <para>When you're using the <literal
256 role="hg-ext">inotify</literal> extension, you should notice
257 <emphasis>no difference at all</emphasis> in Mercurial's
258 behaviour, with the sole exception of status-related commands
259 running a whole lot faster than they used to. You should
260 specifically expect that commands will not print different
261 output; neither should they give different results. If either of
262 these situations occurs, please report a bug.</para>
264 </sect1>
265 <sect1 id="sec:hgext:extdiff">
266 <title>Flexible diff support with the <literal
267 role="hg-ext">extdiff</literal> extension</title>
269 <para>Mercurial's built-in <command role="hg-cmd">hg
270 diff</command> command outputs plaintext unified diffs.</para>
272 &interaction.extdiff.diff;
274 <para>If you would like to use an external tool to display
275 modifications, you'll want to use the <literal
276 role="hg-ext">extdiff</literal> extension. This will let you
277 use, for example, a graphical diff tool.</para>
279 <para>The <literal role="hg-ext">extdiff</literal> extension is
280 bundled with Mercurial, so it's easy to set up. In the <literal
281 role="rc-extensions">extensions</literal> section of your
282 <filename role="special"> /.hgrc</filename>, simply add a
283 one-line entry to enable the extension.</para>
284 <programlisting>[extensions] extdiff =</programlisting>
285 <para>This introduces a command named <command
286 role="hg-ext-extdiff">extdiff</command>, which by default uses
287 your system's <command>diff</command> command to generate a
288 unified diff in the same form as the built-in <command
289 role="hg-cmd">hg diff</command> command.</para>
291 &interaction.extdiff.extdiff;
293 <para>The result won't be exactly the same as with the built-in
294 <command role="hg-cmd">hg diff</command> variations, because the
295 output of <command>diff</command> varies from one system to
296 another, even when passed the same options.</para>
298 <para>As the <quote><literal>making snapshot</literal></quote>
299 lines of output above imply, the <command
300 role="hg-ext-extdiff">extdiff</command> command works by
301 creating two snapshots of your source tree. The first snapshot
302 is of the source revision; the second, of the target revision or
303 working directory. The <command
304 role="hg-ext-extdiff">extdiff</command> command generates
305 these snapshots in a temporary directory, passes the name of
306 each directory to an external diff viewer, then deletes the
307 temporary directory. For efficiency, it only snapshots the
308 directories and files that have changed between the two
309 revisions.</para>
311 <para>Snapshot directory names have the same base name as your
312 repository. If your repository path is <filename
313 class="directory">/quux/bar/foo</filename>, then <filename
314 class="directory">foo</filename> will be the name of each
315 snapshot directory. Each snapshot directory name has its
316 changeset ID appended, if appropriate. If a snapshot is of
317 revision <literal>a631aca1083f</literal>, the directory will be
318 named <filename class="directory">foo.a631aca1083f</filename>.
319 A snapshot of the working directory won't have a changeset ID
320 appended, so it would just be <filename
321 class="directory">foo</filename> in this example. To see what
322 this looks like in practice, look again at the <command
323 role="hg-ext-extdiff">extdiff</command> example above. Notice
324 that the diff has the snapshot directory names embedded in its
325 header.</para>
327 <para>The <command role="hg-ext-extdiff">extdiff</command> command
328 accepts two important options. The <option
329 role="hg-ext-extdiff-cmd-extdiff-opt">hg -p</option> option
330 lets you choose a program to view differences with, instead of
331 <command>diff</command>. With the <option
332 role="hg-ext-extdiff-cmd-extdiff-opt">hg -o</option> option,
333 you can change the options that <command
334 role="hg-ext-extdiff">extdiff</command> passes to the program
335 (by default, these options are
336 <quote><literal>-Npru</literal></quote>, which only make sense
337 if you're running <command>diff</command>). In other respects,
338 the <command role="hg-ext-extdiff">extdiff</command> command
339 acts similarly to the built-in <command role="hg-cmd">hg
340 diff</command> command: you use the same option names, syntax,
341 and arguments to specify the revisions you want, the files you
342 want, and so on.</para>
344 <para>As an example, here's how to run the normal system
345 <command>diff</command> command, getting it to generate context
346 diffs (using the <option role="cmd-opt-diff">-c</option> option)
347 instead of unified diffs, and five lines of context instead of
348 the default three (passing <literal>5</literal> as the argument
349 to the <option role="cmd-opt-diff">-C</option> option).</para>
351 &interaction.extdiff.extdiff-ctx;
353 <para>Launching a visual diff tool is just as easy. Here's how to
354 launch the <command>kdiff3</command> viewer.</para>
355 <programlisting>hg extdiff -p kdiff3 -o</programlisting>
357 <para>If your diff viewing command can't deal with directories,
358 you can easily work around this with a little scripting. For an
359 example of such scripting in action with the <literal
360 role="hg-ext">mq</literal> extension and the
361 <command>interdiff</command> command, see section <xref
362 linkend="mq-collab:tips:interdiff"/>.</para>
364 <sect2>
365 <title>Defining command aliases</title>
367 <para>It can be cumbersome to remember the options to both the
368 <command role="hg-ext-extdiff">extdiff</command> command and
369 the diff viewer you want to use, so the <literal
370 role="hg-ext">extdiff</literal> extension lets you define
371 <emphasis>new</emphasis> commands that will invoke your diff
372 viewer with exactly the right options.</para>
374 <para>All you need to do is edit your <filename role="special">
375 /.hgrc</filename>, and add a section named <literal
376 role="rc-extdiff">extdiff</literal>. Inside this section,
377 you can define multiple commands. Here's how to add a
378 <literal>kdiff3</literal> command. Once you've defined this,
379 you can type <quote><literal>hg kdiff3</literal></quote> and
380 the <literal role="hg-ext">extdiff</literal> extension will
381 run <command>kdiff3</command> for you.</para>
382 <programlisting>[extdiff] cmd.kdiff3 =</programlisting>
383 <para>If you leave the right hand side of the definition empty,
384 as above, the <literal role="hg-ext">extdiff</literal>
385 extension uses the name of the command you defined as the name
386 of the external program to run. But these names don't have to
387 be the same. Here, we define a command named
388 <quote><literal>hg wibble</literal></quote>, which runs
389 <command>kdiff3</command>.</para>
390 <programlisting>[extdiff] cmd.wibble = kdiff3</programlisting>
392 <para>You can also specify the default options that you want to
393 invoke your diff viewing program with. The prefix to use is
394 <quote><literal>opts.</literal></quote>, followed by the name
395 of the command to which the options apply. This example
396 defines a <quote><literal>hg vimdiff</literal></quote> command
397 that runs the <command>vim</command> editor's
398 <literal>DirDiff</literal> extension.</para>
399 <programlisting>[extdiff] cmd.vimdiff = vim opts.vimdiff = -f
400 '+next' '+execute "DirDiff" argv(0) argv(1)'</programlisting>
402 </sect2>
403 </sect1>
404 <sect1 id="sec:hgext:transplant">
405 <title>Cherrypicking changes with the <literal
406 role="hg-ext">transplant</literal> extension</title>
408 <para>Need to have a long chat with Brendan about this.</para>
410 </sect1>
411 <sect1 id="sec:hgext:patchbomb">
412 <title>Send changes via email with the <literal
413 role="hg-ext">patchbomb</literal> extension</title>
415 <para>Many projects have a culture of <quote>change
416 review</quote>, in which people send their modifications to a
417 mailing list for others to read and comment on before they
418 commit the final version to a shared repository. Some projects
419 have people who act as gatekeepers; they apply changes from
420 other people to a repository to which those others don't have
421 access.</para>
423 <para>Mercurial makes it easy to send changes over email for
424 review or application, via its <literal
425 role="hg-ext">patchbomb</literal> extension. The extension is
426 so named because changes are formatted as patches, and it's usual
427 to send one changeset per email message. Sending a long series
428 of changes by email is thus much like <quote>bombing</quote> the
429 recipient's inbox, hence <quote>patchbomb</quote>.</para>
431 <para>As usual, the basic configuration of the <literal
432 role="hg-ext">patchbomb</literal> extension takes just one or
433 two lines in your <filename role="special">
434 /.hgrc</filename>.</para>
435 <programlisting>[extensions] patchbomb =</programlisting>
436 <para>Once you've enabled the extension, you will have a new
437 command available, named <command
438 role="hg-ext-patchbomb">email</command>.</para>
440 <para>The safest and best way to invoke the <command
441 role="hg-ext-patchbomb">email</command> command is to
442 <emphasis>always</emphasis> run it first with the <option
443 role="hg-ext-patchbomb-cmd-email-opt">hg -n</option> option.
444 This will show you what the command <emphasis>would</emphasis>
445 send, without actually sending anything. Once you've had a
446 quick glance over the changes and verified that you are sending
447 the right ones, you can rerun the same command, with the <option
448 role="hg-ext-patchbomb-cmd-email-opt">hg -n</option> option
449 removed.</para>
451 <para>The <command role="hg-ext-patchbomb">email</command> command
452 accepts the same kind of revision syntax as every other
453 Mercurial command. For example, this command will send every
454 revision between 7 and <literal>tip</literal>, inclusive.</para>
455 <programlisting>hg email -n 7:tip</programlisting>
456 <para>You can also specify a <emphasis>repository</emphasis> to
457 compare with. If you provide a repository but no revisions, the
458 <command role="hg-ext-patchbomb">email</command> command will
459 send all revisions in the local repository that are not present
460 in the remote repository. If you additionally specify revisions
461 or a branch name (the latter using the <option
462 role="hg-ext-patchbomb-cmd-email-opt">hg -b</option> option),
463 this will constrain the revisions sent.</para>
465 <para>It's perfectly safe to run the <command
466 role="hg-ext-patchbomb">email</command> command without the
467 names of the people you want to send to: if you do this, it will
468 just prompt you for those values interactively. (If you're
469 using a Linux or Unix-like system, you should have enhanced
470 <literal>readline</literal>-style editing capabilities when
471 entering those headers, too, which is useful.)</para>
473 <para>When you are sending just one revision, the <command
474 role="hg-ext-patchbomb">email</command> command will by
475 default use the first line of the changeset description as the
476 subject of the single email message it sends.</para>
478 <para>If you send multiple revisions, the <command
479 role="hg-ext-patchbomb">email</command> command will usually
480 send one message per changeset. It will preface the series with
481 an introductory message, in which you should describe the
482 purpose of the series of changes you're sending.</para>
484 <sect2>
485 <title>Changing the behaviour of patchbombs</title>
487 <para>Not every project has exactly the same conventions for
488 sending changes in email; the <literal
489 role="hg-ext">patchbomb</literal> extension tries to
490 accommodate a number of variations through command line
491 options.</para>
492 <itemizedlist>
493 <listitem><para>You can write a subject for the introductory
494 message on the command line using the <option
495 role="hg-ext-patchbomb-cmd-email-opt">hg -s</option>
496 option. This takes one argument, the text of the subject
497 to use.</para>
498 </listitem>
499 <listitem><para>To change the email address from which the
500 messages originate, use the <option
501 role="hg-ext-patchbomb-cmd-email-opt">hg -f</option>
502 option. This takes one argument, the email address to
503 use.</para>
504 </listitem>
505 <listitem><para>The default behaviour is to send unified diffs
506 (see section <xref linkend="sec:mq:patch"/> for a
507 description of the
508 format), one per message. You can send a binary bundle
509 instead with the <option
510 role="hg-ext-patchbomb-cmd-email-opt">hg -b</option>
511 option.</para>
512 </listitem>
513 <listitem><para>Unified diffs are normally prefaced with a
514 metadata header. You can omit this, and send unadorned
515 diffs, with the <option
516 role="hg-ext-patchbomb-cmd-email-opt">hg
517 --plain</option> option.</para>
518 </listitem>
519 <listitem><para>Diffs are normally sent <quote>inline</quote>,
520 in the same body part as the description of a patch. This
521 makes it easiest for the largest number of readers to
522 quote and respond to parts of a diff, as some mail clients
523 will only quote the first MIME body part in a message. If
524 you'd prefer to send the description and the diff in
525 separate body parts, use the <option
526 role="hg-ext-patchbomb-cmd-email-opt">hg -a</option>
527 option.</para>
528 </listitem>
529 <listitem><para>Instead of sending mail messages, you can
530 write them to an <literal>mbox</literal>-format mail
531 folder using the <option
532 role="hg-ext-patchbomb-cmd-email-opt">hg -m</option>
533 option. That option takes one argument, the name of the
534 file to write to.</para>
535 </listitem>
536 <listitem><para>If you would like to add a
537 <command>diffstat</command>-format summary to each patch,
538 and one to the introductory message, use the <option
539 role="hg-ext-patchbomb-cmd-email-opt">hg -d</option>
540 option. The <command>diffstat</command> command displays
541 a table containing the name of each file patched, the
542 number of lines affected, and a histogram showing how much
543 each file is modified. This gives readers a qualitative
544 glance at how complex a patch is.</para>
545 </listitem></itemizedlist>
547 </sect2>
548 </sect1>
549 </chapter>
551 <!--
552 local variables:
553 sgml-parent-document: ("00book.xml" "book" "chapter")
554 end:
555 -->