this refactors umount code we already used for the testsuite into the platform module's namespace.
also, it exposes that functionality via the cli api, so users can use it via "borg umount <mountpoint>",
which is more consistent than using borg to mount and fusermount -u (or umount) to un-mount.
the rationale is to simplify the README file to the bare
minimum. security researchers will be able to find the contact
information if they look minimally and people installing the software
will find a link where relevant (in binary releases only, since all
the others have other trust paths)
without those changes, all of the toctree document headings do not
show up. they are considered to be "below" the last heading of the
README file.
we also remove the "Notes" section from the readme as there is only
one note, regarding the fork.
we introduce a stub "introduction" element in the toctree, otherwise
it is impossible for the PDF rendered to render the README correctly.
this is to workaround a bug in the PDF renderer.
it seems now that the fork is more of historical value than a current
thing. people interested in the differences between borg and attic can
look in the FAQ, but I do not see why this is present in the README.
a new section regarding compatibility is created to keep that warning
in place.
move the note about free space after the step by step example. it is
unlikely that users will hit out of space conditions on their first
run, and at the end of the example, they will see the not anyways.
this is to make the documentation less scary for new users and easier
to use.
The current snippet throws deprecation warnings:
```
[DEPRECATION WARNING]: Using bare variables is deprecated. Update your
playbooks so that the environment value uses the full variable syntax
('{{auth_users}}').
This feature will be removed in a future
release. Deprecation warnings can be disabled by setting
deprecation_warnings=False in ansible.cfg.
```
we use "debug xxx" subcommands now. docs updated.
also makes "borg help" shorter as not all debug-xxx commands
show up, but just 1 main "debug" command.
all archives, all items are read to build a unified view.
files are represented by a same-name directory with the versions of the file.
A filename suffix computed by adler32(chunkids) is used to disambiguate the versions.
also: refactor code a little, create methods for leaves, inner nodes.
we will not get() objects that have a segment entry larger than MAX_OBJECT_SIZE.
thus we should never produce such entries.
also: introduce repository.MAX_DATA_SIZE that gives the max payload size.
real story: users that are also developpers expect to find out where
to submit issues and pull requests in the development section, but
couldn't. add some meat there and point to the support section for
everything else.
checkpoint items: chunks abc, abcdef, abcdefghi, ....
part items: chunks abc, def, ghi
solves the "restore a big file over unstable connection issue" (not very comfortably, but
one can just extract one part after the other and concatenate them to get the full file)
some cython stuff can't be processed by sphinx.
some platform-dependant modules can't get imported.
also: reordered modules in api.rst so order makes some sense.
literal blocks must be followed by an empty line
suppressed the non-local image uri warning via sphinx config
the links on the resources page must have different label texts
setup.py: do not generate pointless "::\n" - it does not create a literal block if the stuff below is not indented
This fixes#1247. It also regenerates the usage documentation, so that
styling fixex in that section (as well as other existing changes) make
it into the files in docs/.
* sort install list (fedora was out of place)
* add EPEL
* borg is not young anymore, warn about old distros
* more coherent *BSD ports naming
* add raspbian, openindiana and mageia
There are persistent questions why output from options like --list
and --stats doesn't show up. Also, borg currently isn't able to
show *just* the output for a given option (--list, --stats,
--show-rc, --show-version, or --progress), without other INFO level
messages.
The solution is to use more granular loggers, so that messages
specific to a given option goes to a logger designated for that
option. That option-specific logger can then be configured
separately from the regular loggers.
Those option-specific loggers can also be used as a hook in a
BORG_LOGGING_CONF config file to log the --list output to a separate
file, or send --stats output to a network socket where some daemon
could analyze it.
Steps:
- create an option-specific logger for each of the implied output options
- modify the messages specific to each option to go to the correct logger
- if an implied output option is passed, change the option-specific
logger (only) to log at INFO level
- test that root logger messages don't come through option-specific loggers
They shouldn't, per https://docs.python.org/3/howto/logging.html#logging-flow
but test just the same. Particularly test a message that can come from
remote repositories.
Fixes#526, #573, #665, #824
- Instead of very small (5 MB-ish) segment files, use larger ones
- Request asynchronous write-out or write-through (TODO) where it is supported,
to achieve a continuously high throughput for writes
- Instead of depending on ordered writes (write data, commit tag, sync)
for consistency, do a double-sync commit as more serious RDBMS also do
i.e. write data, sync, write commit tag, sync
Since commits are very expensive in Borg at the moment this makes no
difference performance-wise.
New platform APIs: SyncFile, sync_dir
[x] Naive implementation (equivalent to what Borg did before)
[x] Linux implementation
[ ] Windows implementation
[-] OSX implementation (F_FULLSYNC)
Update installation instructions so that the following error won't occur:
gcc: error: /usr/lib/rpm/redhat/redhat-hardened-cc1: No such file or directory
error: command 'gcc' failed with exit status 1
/mnt/backup was confusing as people like to mount their backup disk on /mnt/backup,
but borg init /mnt/backup does not work if that directory already exists because it is
the mountpoint. it would work, if /mnt was the mountpoint, but that is not obvious
and also unusual.
Use with caution: permanent data loss by specifying incorrect patterns
is easily possible. Make a dry run to make sure you got everything right.
borg recreate has many uses:
- Can selectively remove files/dirs from old archives, e.g. to free
space or purging picturarum biggus dickus from history
- Recompress data
- Rechunkify data, to have upgraded Attic / Borg 0.xx archives deduplicate
with Borg 1.x archives. (Or to experiment with chunker-params for
specific use cases
It is interrupt- and resumable.
Chunks are not freed on-the-fly.
Rationale:
Makes only sense when rechunkifying, but logic on which new chunks to
free what input chunks is complicated and *very* delicate.
Future TODOs:
- Refactor tests using py.test fixtures
-- would require porting ArchiverTestCase to py.test: many changes,
this changeset is already borderline too large.
- Possibly add a --target option to not replace the source archive
-- with the target possibly in another Repo
(better than "cp" due to full integrity checking, and deduplication
at the target)
- Detect and skip (unless --always-recompress) already recompressed chunks
Fixes#787#686#630#70 (and probably some I overlooked)
Also see #757 and #770
- Group options
- Nicer list of options in Sphinx
- Deduplicate 'Common options'
(including --help)
The latter is done by explicitly declaring --help in the common_parser,
which is then inherited by the sub-parsers; no change in observable
behaviour.
ubuntu was showing up twice in the list of supported OSes... it seems it was because the line was getting too long, so I removed the "names" and kept only the numbers to keep the line short.
- Archives now have a new metadata field 'comment'
- 'info' command shows a comment if it's present
- 'create' command now has option '--comment' for adding comments to archives.
- A new command 'comment' is added for modifying the comments on existing
archives.
Resolves#842.
it's not recommended to suppress warnings or errors,
but the user may decide this on his own.
note: --warning is not given to borg serve so a <= 1.0.0 borg
will still work as server. it is not needed as it is the default.
Previously, on 'borg diff', the output always had first the modifications, then
additions, and finally removals. Output may be easier to follow if the various
kinds of changes are interleaved. This commit is a simple solution that first
collects the output lines and sorts them by file path before printing. This new
behavior is optional and disabled by default. It can be enabled with '--sort'
command line option.
This option will be especially useful after the planned multi-threading changes
arrive. Multi-threading may shuffle the archive order of files making diff
output hard to follow without sorting.
Resolves#797.
Makes it easy to see paths excluded by --exclude* options for testing of
regexes, and for ongoing monitoring that files desired for backup aren't
getting excluded accidentally.
The main design goals of the new format:
- One file takes exactly one line of output
- The format is easy to read with typical, long list of changes
- Metadata changes are visible and easy to examine
- Unuseful information is not shown
Resolves#757.
