igor@402: \chapter{File names and pattern matching}
igor@402: \label{chap:names}
igor@402: 
igor@402: Mercurial provides mechanisms that let you work with file names in a
igor@402: consistent and expressive way.
igor@402: 
igor@402: \section{Simple file naming}
igor@402: 
igor@402: Mercurial uses a unified piece of machinery ``under the hood'' to
igor@402: handle file names.  Every command behaves uniformly with respect to
igor@402: file names.  The way in which commands work with file names is as
igor@402: follows.
igor@402: 
igor@402: If you explicitly name real files on the command line, Mercurial works
igor@402: with exactly those files, as you would expect.
igor@402: \interaction{filenames.files}
igor@402: 
igor@402: When you provide a directory name, Mercurial will interpret this as
igor@402: ``operate on every file in this directory and its subdirectories''.
igor@402: Mercurial traverses the files and subdirectories in a directory in
igor@402: alphabetical order.  When it encounters a subdirectory, it will
igor@402: traverse that subdirectory before continuing with the current
igor@402: directory.
igor@402: \interaction{filenames.dirs}
igor@402: 
igor@402: \section{Running commands without any file names}
igor@402: 
igor@402: Mercurial's commands that work with file names have useful default
igor@402: behaviours when you invoke them without providing any file names or
igor@402: patterns.  What kind of behaviour you should expect depends on what
igor@402: the command does.  Here are a few rules of thumb you can use to
igor@402: predict what a command is likely to do if you don't give it any names
igor@402: to work with.
igor@402: \begin{itemize}
igor@402: \item Most commands will operate on the entire working directory.
igor@402:   This is what the \hgcmd{add} command does, for example.
igor@402: \item If the command has effects that are difficult or impossible to
igor@402:   reverse, it will force you to explicitly provide at least one name
igor@402:   or pattern (see below).  This protects you from accidentally
igor@402:   deleting files by running \hgcmd{remove} with no arguments, for
igor@402:   example.
igor@402: \end{itemize}
igor@402: 
igor@402: It's easy to work around these default behaviours if they don't suit
igor@402: you.  If a command normally operates on the whole working directory,
igor@402: you can invoke it on just the current directory and its subdirectories
igor@402: by giving it the name ``\dirname{.}''.
igor@402: \interaction{filenames.wdir-subdir}
igor@402: 
igor@402: Along the same lines, some commands normally print file names relative
igor@402: to the root of the repository, even if you're invoking them from a
igor@402: subdirectory.  Such a command will print file names relative to your
igor@402: subdirectory if you give it explicit names.  Here, we're going to run
igor@402: \hgcmd{status} from a subdirectory, and get it to operate on the
igor@402: entire working directory while printing file names relative to our
igor@402: subdirectory, by passing it the output of the \hgcmd{root} command.
igor@402: \interaction{filenames.wdir-relname}
igor@402: 
igor@402: \section{Telling you what's going on}
igor@402: 
igor@402: The \hgcmd{add} example in the preceding section illustrates something
igor@402: else that's helpful about Mercurial commands.  If a command operates
igor@402: on a file that you didn't name explicitly on the command line, it will
igor@402: usually print the name of the file, so that you will not be surprised
igor@402: what's going on.
igor@402: 
igor@402: The principle here is of \emph{least surprise}.  If you've exactly
igor@402: named a file on the command line, there's no point in repeating it
igor@402: back at you.  If Mercurial is acting on a file \emph{implicitly},
igor@402: because you provided no names, or a directory, or a pattern (see
igor@402: below), it's safest to tell you what it's doing.
igor@402: 
igor@402: For commands that behave this way, you can silence them using the
igor@402: \hggopt{-q} option.  You can also get them to print the name of every
igor@402: file, even those you've named explicitly, using the \hggopt{-v}
igor@402: option.
igor@402: 
igor@402: \section{Using patterns to identify files}
igor@402: 
igor@402: In addition to working with file and directory names, Mercurial lets
igor@402: you use \emph{patterns} to identify files.  Mercurial's pattern
igor@402: handling is expressive.
igor@402: 
igor@402: On Unix-like systems (Linux, MacOS, etc.), the job of matching file
igor@402: names to patterns normally falls to the shell.  On these systems, you
igor@402: must explicitly tell Mercurial that a name is a pattern.  On Windows,
igor@402: the shell does not expand patterns, so Mercurial will automatically
igor@402: identify names that are patterns, and expand them for you.
igor@402: 
igor@402: To provide a pattern in place of a regular name on the command line,
igor@402: the mechanism is simple:
igor@402: \begin{codesample2}
igor@402:   syntax:patternbody
igor@402: \end{codesample2}
igor@402: That is, a pattern is identified by a short text string that says what
igor@402: kind of pattern this is, followed by a colon, followed by the actual
igor@402: pattern.
igor@402: 
igor@402: Mercurial supports two kinds of pattern syntax.  The most frequently
igor@402: used is called \texttt{glob}; this is the same kind of pattern
igor@402: matching used by the Unix shell, and should be familiar to Windows
igor@402: command prompt users, too.  
igor@402: 
igor@402: When Mercurial does automatic pattern matching on Windows, it uses
igor@402: \texttt{glob} syntax.  You can thus omit the ``\texttt{glob:}'' prefix
igor@402: on Windows, but it's safe to use it, too.
igor@402: 
igor@402: The \texttt{re} syntax is more powerful; it lets you specify patterns
igor@402: using regular expressions, also known as regexps.
igor@402: 
igor@402: By the way, in the examples that follow, notice that I'm careful to
igor@402: wrap all of my patterns in quote characters, so that they won't get
igor@402: expanded by the shell before Mercurial sees them.
igor@402: 
igor@402: \subsection{Shell-style \texttt{glob} patterns}
igor@402: 
igor@402: This is an overview of the kinds of patterns you can use when you're
igor@402: matching on glob patterns.
igor@402: 
igor@402: The ``\texttt{*}'' character matches any string, within a single
igor@402: directory.
igor@402: \interaction{filenames.glob.star}
igor@402: 
igor@402: The ``\texttt{**}'' pattern matches any string, and crosses directory
igor@402: boundaries.  It's not a standard Unix glob token, but it's accepted by
igor@402: several popular Unix shells, and is very useful.
igor@402: \interaction{filenames.glob.starstar}
igor@402: 
igor@402: The ``\texttt{?}'' pattern matches any single character.
igor@402: \interaction{filenames.glob.question}
igor@402: 
igor@402: The ``\texttt{[}'' character begins a \emph{character class}.  This
igor@402: matches any single character within the class.  The class ends with a
igor@402: ``\texttt{]}'' character.  A class may contain multiple \emph{range}s
igor@402: of the form ``\texttt{a-f}'', which is shorthand for
igor@402: ``\texttt{abcdef}''.
igor@402: \interaction{filenames.glob.range}
igor@402: If the first character after the ``\texttt{[}'' in a character class
igor@402: is a ``\texttt{!}'', it \emph{negates} the class, making it match any
igor@402: single character not in the class.
igor@402: 
igor@402: A ``\texttt{\{}'' begins a group of subpatterns, where the whole group
igor@402: matches if any subpattern in the group matches.  The ``\texttt{,}''
igor@402: character separates subpatterns, and ``\texttt{\}}'' ends the group.
igor@402: \interaction{filenames.glob.group}
igor@402: 
igor@402: \subsubsection{Watch out!}
igor@402: 
igor@402: Don't forget that if you want to match a pattern in any directory, you
igor@402: should not be using the ``\texttt{*}'' match-any token, as this will
igor@402: only match within one directory.  Instead, use the ``\texttt{**}''
igor@402: token.  This small example illustrates the difference between the two.
igor@402: \interaction{filenames.glob.star-starstar}
igor@402: 
igor@402: \subsection{Regular expression matching with \texttt{re} patterns}
igor@402: 
igor@402: Mercurial accepts the same regular expression syntax as the Python
igor@402: programming language (it uses Python's regexp engine internally).
igor@402: This is based on the Perl language's regexp syntax, which is the most
igor@402: popular dialect in use (it's also used in Java, for example).
igor@402: 
igor@402: I won't discuss Mercurial's regexp dialect in any detail here, as
igor@402: regexps are not often used.  Perl-style regexps are in any case
igor@402: already exhaustively documented on a multitude of web sites, and in
igor@402: many books.  Instead, I will focus here on a few things you should
igor@402: know if you find yourself needing to use regexps with Mercurial.
igor@402: 
igor@402: A regexp is matched against an entire file name, relative to the root
igor@402: of the repository.  In other words, even if you're already in
igor@402: subbdirectory \dirname{foo}, if you want to match files under this
igor@402: directory, your pattern must start with ``\texttt{foo/}''.
igor@402: 
igor@402: One thing to note, if you're familiar with Perl-style regexps, is that
igor@402: Mercurial's are \emph{rooted}.  That is, a regexp starts matching
igor@402: against the beginning of a string; it doesn't look for a match
igor@402: anywhere within the string.  To match anywhere in a string, start
igor@402: your pattern with ``\texttt{.*}''.
igor@402: 
igor@402: \section{Filtering files}
igor@402: 
igor@402: Not only does Mercurial give you a variety of ways to specify files;
igor@402: it lets you further winnow those files using \emph{filters}.  Commands
igor@402: that work with file names accept two filtering options.
igor@402: \begin{itemize}
igor@402: \item \hggopt{-I}, or \hggopt{--include}, lets you specify a pattern
igor@402:   that file names must match in order to be processed.
igor@402: \item \hggopt{-X}, or \hggopt{--exclude}, gives you a way to
igor@402:   \emph{avoid} processing files, if they match this pattern.
igor@402: \end{itemize}
igor@402: You can provide multiple \hggopt{-I} and \hggopt{-X} options on the
igor@402: command line, and intermix them as you please.  Mercurial interprets
igor@402: the patterns you provide using glob syntax by default (but you can use
igor@402: regexps if you need to).
igor@402: 
igor@402: You can read a \hggopt{-I} filter as ``process only the files that
igor@402: match this filter''.
igor@402: \interaction{filenames.filter.include}
igor@402: The \hggopt{-X} filter is best read as ``process only the files that
igor@402: don't match this pattern''.
igor@402: \interaction{filenames.filter.exclude}
igor@402: 
igor@402: \section{Ignoring unwanted files and directories}
igor@402: 
igor@402: XXX.
igor@402: 
igor@402: \section{Case sensitivity}
igor@402: \label{sec:names:case}
igor@402: 
igor@402: If you're working in a mixed development environment that contains
igor@402: both Linux (or other Unix) systems and Macs or Windows systems, you
igor@402: should keep in the back of your mind the knowledge that they treat the
igor@402: case (``N'' versus ``n'') of file names in incompatible ways.  This is
igor@402: not very likely to affect you, and it's easy to deal with if it does,
igor@402: but it could surprise you if you don't know about it.
igor@402: 
igor@402: Operating systems and filesystems differ in the way they handle the
igor@402: \emph{case} of characters in file and directory names.  There are
igor@402: three common ways to handle case in names.
igor@402: \begin{itemize}
igor@402: \item Completely case insensitive.  Uppercase and lowercase versions
igor@402:   of a letter are treated as identical, both when creating a file and
igor@402:   during subsequent accesses.  This is common on older DOS-based
igor@402:   systems.
igor@402: \item Case preserving, but insensitive.  When a file or directory is
igor@402:   created, the case of its name is stored, and can be retrieved and
igor@402:   displayed by the operating system.  When an existing file is being
igor@402:   looked up, its case is ignored.  This is the standard arrangement on
igor@402:   Windows and MacOS.  The names \filename{foo} and \filename{FoO}
igor@402:   identify the same file.  This treatment of uppercase and lowercase
igor@402:   letters as interchangeable is also referred to as \emph{case
igor@402:     folding}.
igor@402: \item Case sensitive.  The case of a name is significant at all times.
igor@402:   The names \filename{foo} and {FoO} identify different files.  This
igor@402:   is the way Linux and Unix systems normally work.
igor@402: \end{itemize}
igor@402: 
igor@402: On Unix-like systems, it is possible to have any or all of the above
igor@402: ways of handling case in action at once.  For example, if you use a
igor@402: USB thumb drive formatted with a FAT32 filesystem on a Linux system,
igor@402: Linux will handle names on that filesystem in a case preserving, but
igor@402: insensitive, way.
igor@402: 
igor@402: \subsection{Safe, portable repository storage}
igor@402: 
igor@402: Mercurial's repository storage mechanism is \emph{case safe}.  It
igor@402: translates file names so that they can be safely stored on both case
igor@402: sensitive and case insensitive filesystems.  This means that you can
igor@402: use normal file copying tools to transfer a Mercurial repository onto,
igor@402: for example, a USB thumb drive, and safely move that drive and
igor@402: repository back and forth between a Mac, a PC running Windows, and a
igor@402: Linux box.
igor@402: 
igor@402: \subsection{Detecting case conflicts}
igor@402: 
igor@402: When operating in the working directory, Mercurial honours the naming
igor@402: policy of the filesystem where the working directory is located.  If
igor@402: the filesystem is case preserving, but insensitive, Mercurial will
igor@402: treat names that differ only in case as the same.
igor@402: 
igor@402: An important aspect of this approach is that it is possible to commit
igor@402: a changeset on a case sensitive (typically Linux or Unix) filesystem
igor@402: that will cause trouble for users on case insensitive (usually Windows
igor@402: and MacOS) users.  If a Linux user commits changes to two files, one
igor@402: named \filename{myfile.c} and the other named \filename{MyFile.C},
igor@402: they will be stored correctly in the repository.  And in the working
igor@402: directories of other Linux users, they will be correctly represented
igor@402: as separate files.
igor@402: 
igor@402: If a Windows or Mac user pulls this change, they will not initially
igor@402: have a problem, because Mercurial's repository storage mechanism is
igor@402: case safe.  However, once they try to \hgcmd{update} the working
igor@402: directory to that changeset, or \hgcmd{merge} with that changeset,
igor@402: Mercurial will spot the conflict between the two file names that the
igor@402: filesystem would treat as the same, and forbid the update or merge
igor@402: from occurring.
igor@402: 
igor@402: \subsection{Fixing a case conflict}
igor@402: 
igor@402: If you are using Windows or a Mac in a mixed environment where some of
igor@402: your collaborators are using Linux or Unix, and Mercurial reports a
igor@402: case folding conflict when you try to \hgcmd{update} or \hgcmd{merge},
igor@402: the procedure to fix the problem is simple.
igor@402: 
igor@402: Just find a nearby Linux or Unix box, clone the problem repository
igor@402: onto it, and use Mercurial's \hgcmd{rename} command to change the
igor@402: names of any offending files or directories so that they will no
igor@402: longer cause case folding conflicts.  Commit this change, \hgcmd{pull}
igor@402: or \hgcmd{push} it across to your Windows or MacOS system, and
igor@402: \hgcmd{update} to the revision with the non-conflicting names.
igor@402: 
igor@402: The changeset with case-conflicting names will remain in your
igor@402: project's history, and you still won't be able to \hgcmd{update} your
igor@402: working directory to that changeset on a Windows or MacOS system, but
igor@402: you can continue development unimpeded.
igor@402: 
igor@402: \begin{note}
igor@402:   Prior to version~0.9.3, Mercurial did not use a case safe repository
igor@402:   storage mechanism, and did not detect case folding conflicts.  If
igor@402:   you are using an older version of Mercurial on Windows or MacOS, I
igor@402:   strongly recommend that you upgrade.
igor@402: \end{note}
igor@402: 
igor@402: %%% Local Variables: 
igor@402: %%% mode: latex
igor@402: %%% TeX-master: "00book"
igor@402: %%% End: