it seems it is possible that the chunks files are copied but *not*
converted. this may have happened here because the conversion was
interrupted, although the specific scenario is still unclear (but it
did happen during manual tests here). therefore reproducing this
problem seems to be difficult, hence the lack of tests for this
specific issue.
since we consider the header replacement code to be safe, that we
always convert shouldn't pose any additional threat to the existing
borg chunk cache.
this resolves bug #something where the index file could not be
converted, completely breaking conversion.
it seems that, during some refactoring, the index conversion code was
completely dropped. this was missed by the unit tests because the repo
can still be opened by the constructor even though the index is
invalid, so tests need improvements there.
this function was over-coupling the cache system and the statistics
module. they are now almost decoupled insofar as the cache system has
its own rendering system now that is called separately.
furthermore, we have a much more flexible formatting system that is
used coherently between --progress and --stats
the degenerate case here is if we want to change the label in the
statistics summary: in this case we need to override the default
__str__() representation to insert our own label.
i saw the errors in my ways: __format__ is only to customize the
"format mini-language", what comes after ":" in a new string
format. unfortunately, we cannot easily refer to individual fields in
there, short of re-implementing a new formatting language, which seems
silly.
instead, we use properties to extract human-readable versions of the
statistics. more intuitive and certainly a more common pattern than
the exotic __format__().
also add unit tests to prove this all works
we stop enforcing a minimum width for fields, it changes only on
logarithmic boundaries, so not a big problem. string conversion is
implicit
this gives us a little more width for the path
we use the new get_terminal_size() function, with a fallback for
Python 3.2. we default to 80 columns.
then we generate the stats bit and fill the rest with the path, as
previously, but with a possibly larger field.
note that this works with resizes in my test (uxterm)
the --stats output would be slightly garbled by --progress, because of
the \r that is output at the last line...
example:
initializing cache
reading files cache
processing files
------------------------------------------------------------------------------ s/twotone
Archive name: 2015-10-15-test
a single -v flag shouldn't flood the console with all the files in the
path specified, it makes -v basically useless
this way, -v can also be used with --progress to have nicer output:
initializing cache
reading files cache
processing files
5.20 GB O 2.66 GB C 25.13 MB D 27576 N baz/...
as it was, surrogates were not always removed, for example
we may also want to output at different levels or control if we want
to print unchanged files and so on
without this, there would be a solid 20 seconds here without any sort
of output on the console, regardless of the verbosity level. this
makes nice incremental messages telling the user that borg is not
stalled (or waiting for a lock, for that matter)
the "processing files" message is a little clunky, as we somewhat
abuse the cache to figure out if we are just starting... but it helps
if there are problems reading the actual files: it tells us the
initialization is basically complete and we're going ahead with the
reading of all the files.
the old name still works, but emits a deprecation warning suggesting the new name.
this is a followup to 4fd06e2634, which added "-x" (as seen in "du").
Moved the list of dependencies to the corresponding subsection.
Collected all preparation steps under one heading.
Added link to the Arch Linux AUR package.
Install docs for OS X.
instead, we perform the equivalent of `cp -al` on the repository to
keep a backup, and then rewrite the files, breaking the hardlinks as
necessary.
it has to be confirmed that the rest of Borg will also break hardlinks
when operating on files in the repository. if Borg operates in place
on any files of the repository, it could jeoperdize the backup, so
this needs to be verified. I believe that most files are written to a
temporary file and moved into place, however, so the backup should be
safe.
the rationale behind the backup copy is that we want to be extra
careful with user's data by default. the old behavior is retained
through the `--inplace`/`-i` commandline flag. plus, this way we don't
need to tell users to go through extra steps (`cp -a`, in particular)
before running the command.
also, it can take a long time to do the copy of the attic repository
we wish to work on. since `cp -a` doesn't provide progress
information, the new default behavior provides a nicer user experience
of giving an overall impression of the upgrade progress, while
retaining compatibility with Attic by default (in a separate
repository, of course).
this makes the upgrade command much less scary to use and hopefully
will convert drones to the borg collective.
the only place where the default inplace behavior is retained is in
the header_replace() function, to avoid breaking the cache conversion
code and to keep API stability and semantic coherence ("replace" by
defaults means in place).
This is what attic used by default, but borgbackup defaults to "no compression".
I just adjusted the command invocation, so we can keep the example output
(which shows that stuff was compressed).
Also: add FAQ item about compression.
we use forking mode always and either execute python with the archiver module or the "borg.exe" binary.
the cmd fixture alternates between 'python' and 'binary' mode and calls exec_cmd accordingly.
In one case removed the |project_name| and |git_url| variables to fix
the display of the code block. Shouldn't be problematic, as they are not
used consistently in this document anyway.
Put two notes in their own nice '.. note::' blocks.
Antoine Beaupré