The existing option to exclude files and directories, “--exclude”, is
implemented using fnmatch[1]. fnmatch matches the slash (“/”) with “*”
and thus makes it impossible to write patterns where a directory with
a given name should be excluded at a specific depth in the directory
hierarchy, but not anywhere else. Consider this structure:
home/
home/aaa
home/aaa/.thumbnails
home/user
home/user/img
home/user/img/.thumbnails
fnmatch incorrectly excludes “home/user/img/.thumbnails” with a pattern
of “home/*/.thumbnails” when the intention is to exclude “.thumbnails”
in all home directories while retaining directories with the same name
in all other locations.
With this change regular expressions are introduced as an additional
pattern syntax. The syntax is selected using a prefix on “--exclude”'s
value. “re:” is for regular expression and “fm:”, the default, selects
fnmatch. Selecting the syntax is necessary when regular expressions are
desired or when the desired fnmatch pattern starts with two alphanumeric
characters followed by a colon (i.e. “aa:something/*”). The exclusion
described above can be implemented as follows:
--exclude 're:^home/[^/]+/\.thumbnails$'
The “--exclude-from” option permits loading exclusions from a text file
where the same prefixes can now be used, e.g. “re:\.tmp$”.
The documentation has been extended and now not only describes the two
pattern styles, but also the file format supported by “--exclude-from”.
This change has been discussed in issue #43 and in change request #497.
[1] https://docs.python.org/3/library/fnmatch.html
Signed-off-by: Michael Hanselmann <public@hansmi.ch>
removed --log-level due to overlap with how --verbose works now.
for consistency, added --info as alias to --verbose (as the effect is
setting INFO log level).
also added --debug which sets DEBUG log level.
note: there are no messages emitted at DEBUG level yet.
WARNING is the default (because we want mostly silent behaviour,
except if something serious happens), so we don't need --warning
as an option.
the problem here was that we do not just have changed and unchanged items,
but also a lot of items besides regular files which we just back up "as is" without
determining whether they are changed or not. thus, we can't support changed/unchanged
in a way users would expect them to work.
the A/M/U status only applies to the data content of regular files (compared to the index).
for all items, we ALWAYS save the metadata, there is no changed / not changed detection there.
thus, I replaced this with a --filter option where you can just specify which
status chars you want to see listed in the output.
E.g. --filter AM will only show regular files with A(dded) or M(odified) state, but nothing else.
Not giving --filter defaults to showing all items no matter what status they have.
Output is emitted via logger at info level, so it won't show up except if the logger is at that level.
also: remove mailing list and irc channel address from development docs,
it is enough to have this information on the main page and on the support page.
the generation of those files was causing us way too much pain to
justify automatically generating them all the time.
those will have to be re-generated with `build_api` or `build_usage`
as appropriate, for example when function signatures or commandline
flags change.
see #384
this was making us require mock, which is really a test component and
shouldn't be part of the runtime dependencies. furthermore, it was
making the imports and the code more brittle: it may have been
possible that, through an environment variable, backups could be
corrupted because mock libraries would be configured instead of real
once, which is a risk we shouldn't be taking.
finally, this was used only to build docs, which we will build and
commit to git by hand with a fully working borg when relevant.
see #384.
Moved the list of dependencies to the corresponding subsection.
Collected all preparation steps under one heading.
Added link to the Arch Linux AUR package.
Install docs for OS X.
This is what attic used by default, but borgbackup defaults to "no compression".
I just adjusted the command invocation, so we can keep the example output
(which shows that stuff was compressed).
Also: add FAQ item about compression.
In one case removed the |project_name| and |git_url| variables to fix
the display of the code block. Shouldn't be problematic, as they are not
used consistently in this document anyway.
Put two notes in their own nice '.. note::' blocks.
this still doesn't quite work: our sidebar is gone, so no more useful
links and related projects. we also loose the link to github and the
RTD popup, although the latter still needs to be confirmed on RTD
infra
instead of applying this only to usage generation, use it as a generic
mechanism to disable loading of Cython code.
it may be incomplete: there may be other places where Cython code is
loaded that is not checked, but that is sufficient to build the usage
docs. the environment variable used is documented as such in the
docs/usage.rst.
we also move the check to a helper function and document it
better. this has the unfortunate side effect of moving includes
around, but I can't think of a better way.
this is an unfortunate rewrite of the manpage creation code mentionned
in #208. ideally, this would be rewritten into a class that can
generate both man pages and .rst files.
instead of a boring table of contents, try to show our more exciting README file
it's still a wall of text, but at least all the buzzwords and highlights are there
ideally, the table of contents would be in the sidebar, but i don't know how to do that
link to the locations of different tools when I know them. i marked
the ones I don't know about specially so we can document those as
well.
point to the Github releases for the standalone binaries upload
we stop supporting them, because there are better alternatives:
- use a distribution package (from your linux distribution), if available
- use a pyinstaller binary provided by us (they include all you need in 1 file and
thus have better compatibility properties and are easier to install than a wheel)
- install from source (pypi or git) if everything else fails
while SSH options can be specified through `~/.ssh/config`, some users
may want to use a completely different SSH command for their backups,
without overriding their $PATH variable. it may also be easier to do
ad-hoc configuration and tests that way.
plus, the POLA tells us that users expects something like this to be
supported by commands that talk to ssh. it is supported by rsync, git
and so on.
this is a crude hack for now, and could use a better table of contents
but at least we have some way of linking and showing the different
internal functions
the next phase here is obviously to document that API through the
addition of docstrings. a static api.rst could also be easier to read,
but maybe that could go through some docstrings as well, to be tested
right now, the update_usage script regenerates the usage files at
every call
by moving this into the makefile, we make those files be generated
only when the source file change, which makes testing docs much faster
- reduce redundancy (platforms are documented in README.rst)
- reformat to 80 chars width
- clarify checkpoints
- remove workarounds for stuff that was fixed
sets the default repository to use, e.g. like:
export BORG_REPO=/mnt/backup/repo
borg init
borg create ::archive
borg list
borg mount :: /mnt
fusermount -u /mnt
borg delete ::archive
outdated - it just showed different levels of zlib compression,
but not we additionally have "lzma", "lz4" and "none" compression.
the "usage" and "internals" docs give some hints about them, too.
found out why it could not install llfuse into virtual env: it always complained about
not being able to find fuse.pc - which is part of libfuse-dev / fuse-devel and was missing.
once one adds the fuse dev stuff, llfuse installs to virtual env without problems.
README.rst (shown on github and also at the start of the html docs) shall
be like an elevator speech - convince readers in a very short time.
this is most important, everything else can come after we got the reader's interest.
include README into docs to avoid duplication.
also include CHANGES into docs.
add developer docs, move examples from tox.ini there
add separate support docs
remove glossary, most of what was there can be understood by an admin from context
move attic and compatibility note to the end