Patterns to exclude files can be loaded from a text file using the
“--exclude-from” option. Whitespace at the beginning or end of lines was
not stripped. Indented comments would be interpreted as a pattern and
a misplaced space at the end of a line--some text editors don't strip
them--could cause an exclusion pattern to not match as desired. With the
recent addition of regular expression support for exclusions the spaces
can be matched if necessary (“^\s” or “\s$”), though it's highly
unlikely that there are many paths deliberately starting or ending with
whitespace.
The existing option to exclude files and directories, “--exclude”, is
implemented using fnmatch[1]. fnmatch matches the slash (“/”) with “*”
and thus makes it impossible to write patterns where a directory with
a given name should be excluded at a specific depth in the directory
hierarchy, but not anywhere else. Consider this structure:
home/
home/aaa
home/aaa/.thumbnails
home/user
home/user/img
home/user/img/.thumbnails
fnmatch incorrectly excludes “home/user/img/.thumbnails” with a pattern
of “home/*/.thumbnails” when the intention is to exclude “.thumbnails”
in all home directories while retaining directories with the same name
in all other locations.
With this change regular expressions are introduced as an additional
pattern syntax. The syntax is selected using a prefix on “--exclude”'s
value. “re:” is for regular expression and “fm:”, the default, selects
fnmatch. Selecting the syntax is necessary when regular expressions are
desired or when the desired fnmatch pattern starts with two alphanumeric
characters followed by a colon (i.e. “aa:something/*”). The exclusion
described above can be implemented as follows:
--exclude 're:^home/[^/]+/\.thumbnails$'
The “--exclude-from” option permits loading exclusions from a text file
where the same prefixes can now be used, e.g. “re:\.tmp$”.
The documentation has been extended and now not only describes the two
pattern styles, but also the file format supported by “--exclude-from”.
This change has been discussed in issue #43 and in change request #497.
[1] https://docs.python.org/3/library/fnmatch.html
Signed-off-by: Michael Hanselmann <public@hansmi.ch>
The two classes for applying inclusion and exclusion patterns contained
unnecessarily duplicated logic. The introduction of a shared base class
allows for easier reuse, especially considering that two more classes
are going to be added in forthcoming changes (regular expressions and
shell-style patterns).
prune and create now both require --verbose --stats to show stats.
it was implemented in this way (and not with print) so you can feed the stats data
into the logging system, too.
delete now says "Archive deleted" in verbose mode (for consistency,
it already said "Repository deleted" when deleting a repo).
also: add helpers.log_multi to comfortably and prettily output a block of log lines
The parsing code for exclude files (given via `--exclude-from`) was not
tested. Its core is factorized into a separate function to facilitate an
easier test. The observable behaviour is unchanged.
Danny Edel