1
0
Fork 0
mirror of https://github.com/borgbackup/borg.git synced 2025-02-24 07:01:59 +00:00
Commit graph

1551 commits

Author SHA1 Message Date
Antoine Beaupré
6b547e6554 no need to flush stderr in my tests 2015-10-17 21:21:32 -04:00
Antoine Beaupré
75c993b875 simplify progress display
we stop enforcing a minimum width for fields, it changes only on
logarithmic boundaries, so not a big problem. string conversion is
implicit

this gives us a little more width for the path
2015-10-17 21:21:32 -04:00
Antoine Beaupré
ecae163072 use all available columns for path in progress
we use the new get_terminal_size() function, with a fallback for
Python 3.2. we default to 80 columns.

then we generate the stats bit and fill the rest with the path, as
previously, but with a possibly larger field.

note that this works with resizes in my test (uxterm)
2015-10-17 21:21:32 -04:00
Antoine Beaupré
7f28244cfe fix conflict between --stats and --progress
the --stats output would be slightly garbled by --progress, because of
the \r that is output at the last line...

example:

    initializing cache
    reading files cache
    processing files
    ------------------------------------------------------------------------------ s/twotone
    Archive name: 2015-10-15-test
2015-10-17 21:21:32 -04:00
Antoine Beaupré
e8cf28f9a0 only show all files found in debug
a single -v flag shouldn't flood the console with all the files in the
path specified, it makes -v basically useless

this way, -v can also be used with --progress to have nicer output:

initializing cache
reading files cache
processing files
  5.20 GB O   2.66 GB C  25.13 MB D 27576 N baz/...
2015-10-17 21:21:31 -04:00
Antoine Beaupré
d8f8076984 factor out status output so it is consistent
as it was, surrogates were not always removed, for example

we may also want to output at different levels or control if we want
to print unchanged files and so on
2015-10-17 21:21:31 -04:00
Antoine Beaupré
b120e5f119 output more progress information
without this, there would be a solid 20 seconds here without any sort
of output on the console, regardless of the verbosity level. this
makes nice incremental messages telling the user that borg is not
stalled (or waiting for a lock, for that matter)

the "processing files" message is a little clunky, as we somewhat
abuse the cache to figure out if we are just starting... but it helps
if there are problems reading the actual files: it tells us the
initialization is basically complete and we're going ahead with the
reading of all the files.
2015-10-17 21:21:31 -04:00
Antoine Beaupré
3827c2b107 also show the number of files processed in --progress 2015-10-17 21:21:31 -04:00
Antoine Beaupré
3eebea8745 explain what the --progress flag does exactly
an alternative to this would be to use more than a letter in the
output, for example:

 orig: 16.82 GB comp: 9.44 GB dedup: 25.86 MB home/
2015-10-17 21:21:31 -04:00
Thomas Waldmann
6712bdf456 include a list of major changes compared to attic into the README, fixes #224 2015-10-17 22:27:59 +02:00
Thomas Waldmann
908e944bdd include the help offered by borg help <topic> in the usage docs, fixes #293
it's not pretty yet, but better than nothing.
2015-10-17 21:53:09 +02:00
Thomas Waldmann
fc78b44923 use --one-file-system (as "du" tool does) instead of --do-not-cross-mountpoints
the old name still works, but emits a deprecation warning suggesting the new name.

this is a followup to 4fd06e2634, which added "-x" (as seen in "du").
2015-10-17 20:55:07 +02:00
TW
0e9c524266 Merge pull request #292 from anarcat/rename-usage
add missing rename usage to docs, closes #279
2015-10-17 20:06:23 +02:00
TW
2dda41dc2a Merge pull request #294 from anarcat/x-option
add -x flag, common to GNU utilities
2015-10-17 20:06:02 +02:00
TW
b0f66e84a9 Merge pull request #295 from yagebu/master
docs: restructure installation.rst + add OS X install instructions
2015-10-17 20:05:36 +02:00
Jakob Schnitzer
462bb41e57 docs: AUTHORS: one borg in the title is enough 2015-10-17 18:40:15 +02:00
Jakob Schnitzer
7cc3becad9 docs: restructure installation.rst, add OSX instructions
Moved the list of dependencies to the corresponding subsection.
Collected all preparation steps under one heading.
Added link to the Arch Linux AUR package.
Install docs for OS X.
2015-10-17 18:40:15 +02:00
Antoine Beaupré
eacb0b9e83 Merge branch 'logging-refactor' into upstream 2015-10-17 12:29:52 -04:00
Antoine Beaupré
4fd06e2634 add -x flag, common to GNU utilities
it should also probably be --one-file-system for coherence with du(1)
2015-10-17 12:23:45 -04:00
Antoine Beaupré
e6c2359a60 add missing rename usage to docs, closes #279 2015-10-17 12:13:59 -04:00
Antoine Beaupré
34d0e0641c make sure hardlink copy doesn't break perms 2015-10-17 00:41:20 -04:00
Antoine Beaupré
4be9c29d0d os.link signature is the same as shutil.copy, use it directly 2015-10-17 00:38:14 -04:00
Antoine Beaupré
aaf72e3861 do not skip all attic tests, some work without now 2015-10-17 00:38:14 -04:00
Antoine Beaupré
6d457aed57 do not upgrade repositories in place by default
instead, we perform the equivalent of `cp -al` on the repository to
keep a backup, and then rewrite the files, breaking the hardlinks as
necessary.

it has to be confirmed that the rest of Borg will also break hardlinks
when operating on files in the repository. if Borg operates in place
on any files of the repository, it could jeoperdize the backup, so
this needs to be verified. I believe that most files are written to a
temporary file and moved into place, however, so the backup should be
safe.

the rationale behind the backup copy is that we want to be extra
careful with user's data by default. the old behavior is retained
through the `--inplace`/`-i` commandline flag. plus, this way we don't
need to tell users to go through extra steps (`cp -a`, in particular)
before running the command.

also, it can take a long time to do the copy of the attic repository
we wish to work on. since `cp -a` doesn't provide progress
information, the new default behavior provides a nicer user experience
of giving an overall impression of the upgrade progress, while
retaining compatibility with Attic by default (in a separate
repository, of course).

this makes the upgrade command much less scary to use and hopefully
will convert drones to the borg collective.

the only place where the default inplace behavior is retained is in
the header_replace() function, to avoid breaking the cache conversion
code and to keep API stability and semantic coherence ("replace" by
defaults means in place).
2015-10-17 00:38:07 -04:00
Thomas Waldmann
e7f25d7821 docs: use zlib,6 compression in some examples, fixes #275
This is what attic used by default, but borgbackup defaults to "no compression".

I just adjusted the command invocation, so we can keep the example output
(which shows that stuff was compressed).

Also: add FAQ item about compression.
2015-10-16 01:18:37 +02:00
Thomas Waldmann
9b10e8a3f3 if borg.exe is not present, do not try to test it 2015-10-16 00:52:23 +02:00
Thomas Waldmann
1a248116db move cmd fixture to archiver test module 2015-10-16 00:18:46 +02:00
Thomas Waldmann
4b7c02775e benchmarks: test with both the binary and the python code
we use forking mode always and either execute python with the archiver module or the "borg.exe" binary.
the cmd fixture alternates between 'python' and 'binary' mode and calls exec_cmd accordingly.
2015-10-16 00:12:02 +02:00
TW
a96770636c Merge pull request #270 from ThomasWaldmann/benchmark
integrate pytest-benchmark, test create, extract, list, delete, info,…
2015-10-15 10:58:20 +02:00
TW
ba7e365238 Merge pull request #273 from yagebu/master
docs: Some small fixes
2015-10-15 10:56:48 +02:00
Thomas Waldmann
7efab2f254 benchmark tests: improve comments 2015-10-14 01:00:25 +02:00
Thomas Waldmann
f8b536890b use pytest-benchmark from pypi 2015-10-14 00:54:32 +02:00
Jakob Schnitzer
da926f2eb4 docs: fix code blocks in installation.rst
In one case removed the |project_name| and |git_url| variables to fix
the display of the code block. Shouldn't be problematic, as they are not
used consistently in this document anyway.

Put two notes in their own nice '.. note::' blocks.
2015-10-13 17:25:52 +02:00
Jakob Schnitzer
45a96fbc8d docs: change name of API section
from 'Borg Backup API documentation' to 'API Documentation'

It is pretty clear which software the API docs are for imho.
2015-10-13 17:02:15 +02:00
Jakob Schnitzer
fb62435fc6 docs: remove 'Additional Notes' from TOC, fix markup
What's covered in the 'Additional Notes' section of the documentation
hardly seems important enough to deserve its own entry in the sidebar.
2015-10-13 16:50:27 +02:00
Jakob Schnitzer
b42bbc6f68 docs: fix code blocks 2015-10-13 16:31:25 +02:00
Antoine Beaupré
256f89e697 add back logo, see #272
it's ugly as hell, but at least we have some logo there. keeping the issue opened for improvements
2015-10-13 08:35:12 -04:00
anarcat
97a9855417 Merge pull request #268 from anarcat/rtd-theme
switch to default rtd theme
2015-10-13 08:27:38 -04:00
Antoine Beaupré
f31b4a639b fix headings in README
this makes it consistent with the rest of the documentation
2015-10-12 23:36:48 -04:00
Antoine Beaupré
6b604ac28c Merge remote-tracking branch 'origin/master' into rtd-theme
Conflicts:
	README.rst
2015-10-12 23:29:45 -04:00
Antoine Beaupré
e59045fd24 move links from the sidebar to the readme
it seems sidebar links are not supported by the RTD theme just yet: https://github.com/snide/sphinx_rtd_theme/issues/254
2015-10-12 23:27:59 -04:00
Thomas Waldmann
08d96099f1 pytest-benchmark: use git master code, fixes py32 issues 2015-10-12 00:52:52 +02:00
Thomas Waldmann
0b9dcbf8e5 Merge branch 'master' of github.com:borgbackup/borg 2015-10-11 21:20:56 +02:00
Thomas Waldmann
8eaf460da3 setup.py: fix bug related to BORG_LZ4_PREFIX processing 2015-10-11 21:20:29 +02:00
Thomas Waldmann
07d9132f05 travis: use tox -r so it rebuilds the tox envs
it seems to keep .tox envs even if one removes stuff from requirements file.
if that stuff causes troubles, as here with pytest-benchmark, the troubles stay...
2015-10-11 17:41:25 +02:00
Thomas Waldmann
9dccc809a9 create a separate requirements file for travis - without pytest-benchmark
pytest-benchmark (and its dependency "statistics") needs py 3.3+.

we do not want to run benchmarks on travis anyway, but even its presence
makes trouble due to this, so we just remove it from there again.

as --benchmark-skip is also removed from tox configuration, this means benchmarks
are run by default when pytest-benchmark is installed.
2015-10-11 17:06:12 +02:00
Thomas Waldmann
a9b1e591ab upgrade pytest-benchmark to a4 2015-10-11 16:25:43 +02:00
Thomas Waldmann
a5a6ba0d77 integrate pytest-benchmark, test create, extract, list, delete, info, check, help, fixes #146
Instead of "realistic data", I chose the test data to be either all-zero (all-ascii-zero to be precise)
or all-random and benchmark them separately.
So we can better determine the cause (deduplication or storage) in case we see some performance regression.

"help" is benchmarked to see the minimum runtime when it basically does nothing.

also:
- refactor archiver execution core functionality into exec_cmd() so it can be used more flexibly
- tox: usually we want to skip benchmarks, only run them if requested manually
- install pytest-benchmark - run tox with "-r" to have it installed into your .tox envs
2015-10-11 16:07:11 +02:00
TW
4d0fa95f20 Merge pull request #267 from anarcat/docs-fixes
API doc fixes
2015-10-10 01:25:53 +02:00
Antoine Beaupré
3c535b9d7a switch to default rtd theme
this still doesn't quite work: our sidebar is gone, so no more useful
links and related projects. we also loose the link to github and the
RTD popup, although the latter still needs to be confirmed on RTD
infra
2015-10-09 16:09:27 -04:00