Marian Beermann
0e31f78dd6
cache sync: avoid "l" and such as a variable name, note vanilla changes
2017-06-10 10:17:28 +02:00
Marian Beermann
5eb43b8464
cache sync: give overview of the source's structure
2017-06-10 10:17:28 +02:00
Marian Beermann
310a71e4f0
cache sync: use ro_buffer to accept bytes, memoryview, ...
2017-06-10 10:17:28 +02:00
Marian Beermann
5af66dbb12
cache sync: add more refcount tests
2017-06-03 15:02:27 +02:00
Marian Beermann
4faaa7d1fa
RepositoryCache: abort on data corruption
2017-06-03 12:27:35 +02:00
Marian Beermann
b544af2af1
RepositoryCache: checksum decrypted cache
2017-06-03 12:23:23 +02:00
Marian Beermann
5b3667b617
cache sync: macros in all-caps
2017-06-02 19:31:56 +02:00
Marian Beermann
795cdfc9ab
cache sync: move stat initialization to main unpack
2017-06-02 19:30:53 +02:00
Marian Beermann
cb98cb838d
fuse: fix read(2) caching data in metadata cache
...
The OS page cache is responsible for handling this and is much more
empowered to do a good job at that than Borg.
2017-06-02 17:43:15 +02:00
Marian Beermann
67b97f2223
cache sync: cleanup progress handling, unused parameters
2017-06-02 17:43:15 +02:00
Marian Beermann
7f04e00ba2
testsuite: add TestRepositoryCache
2017-06-02 17:43:15 +02:00
Marian Beermann
835b0e5ee0
cache sync/remote: compressed, decrypted cache
2017-06-02 17:43:15 +02:00
Marian Beermann
c786a5941e
CacheSynchronizer: redo as quasi FSM on top of unpack.h
...
This is a (relatively) simple state machine running in the
data callbacks invoked by the msgpack unpacking stack machine
(the same machine is used in msgpack-c and msgpack-python,
changes are minor and cosmetic, e.g. removal of msgpack_unpack_object,
removal of the C++ template thus porting to C and so on).
Compared to the previous solution this has multiple advantages
- msgpack-c dependency is removed
- this approach is faster and requires fewer and smaller
memory allocations
Testability of the two solutions does not differ in my
professional opinion(tm).
Two other changes were rolled up; _hashindex.c can be compiled
without Python.h again (handy for fuzzing and testing);
a "small" bug in the cache sync was fixed which allocated too
large archive indices, leading to excessive archive.chunks.d
disk usage (that actually gave me an idea).
2017-06-02 17:43:15 +02:00
Marian Beermann
bf895950ac
RepositoryCache: limit cache size
...
Unbounded cache size is inacceptable.
I don't see why a full-fledged repository needs to be used here, either,
since this cache requires none of the consistency or durability guarantees
made by it (and bought with a performance impact).
A notable issue is that posix_fadvise is slow (for some reason) on tmpfs,
which could eat 30-35 % of the total CPU time of a cache sync.
2017-06-02 17:43:15 +02:00
Marian Beermann
167875b753
cache sync: fix n^2 behaviour in lookup_name
2017-06-02 17:43:14 +02:00
Marian Beermann
9f8b967a6f
cache sync: initialize master index to known capacity
2017-06-02 17:43:14 +02:00
Marian Beermann
740898d83b
CacheSynchronizer
2017-06-02 17:43:14 +02:00
enkore
47894072ad
Merge pull request #2580 from enkore/issue/1101.integration.xxh
...
1101.integration.xxh
2017-06-02 16:23:54 +02:00
enkore
d3373bebf1
Merge pull request #2588 from enkore/docs/quota
...
docs: quotas: clarify compatbility; only relevant to serve side
2017-06-02 16:22:20 +02:00
Marian Beermann
d51f2bbbae
docs: quotas: local repo disclaimer
...
...
2017-06-02 14:50:51 +02:00
Marian Beermann
089224975b
docs: quotas: clarify compatbility; only relevant to serve side
...
also cf. "Enforcing the quota":
The quota is enforcible only if *all* :ref:`borg_serve` versions
accessible to clients support quotas
2017-06-02 13:12:43 +02:00
Marian Beermann
6c91a750d1
algorithms: rename crc32 to checksums
2017-06-01 21:26:42 +02:00
Marian Beermann
0221e31058
file_integrity: use xxh64
2017-06-01 21:26:42 +02:00
enkore
23d591c1c3
Merge pull request #2565 from enkore/issue/2517
...
Implement storage quotas
2017-06-01 14:39:17 +02:00
Marian Beermann
bcf4b4492b
testsuite: add test for parse_storage_quota
2017-05-31 20:28:17 +02:00
enkore
1b66d8d109
Merge pull request #2576 from enkore/pr/2573+split
...
BORG_PASSCOMMAND: use same cmd-string splitting as BORG_RSH
2017-05-31 19:41:42 +02:00
Marian Beermann
4e6a771ee7
BORG_PASSCOMMAND: use same cmd-string splitting as BORG_RSH
2017-05-31 19:41:17 +02:00
enkore
349a4ade7c
Merge pull request #2568 from enkore/issue/1101.integration.cache
...
1101.integration.cache
2017-05-31 19:34:43 +02:00
TuXicc
578b76af3a
Added BORG_PASSCOMMAND environment variable ( #2573 )
2017-05-31 19:25:21 +02:00
Marian Beermann
f8b48dc8d7
remote: propagate Error.traceback correctly
2017-05-31 18:48:48 +02:00
Marian Beermann
4edf77788d
Implement storage quotas
2017-05-31 18:36:03 +02:00
Marian Beermann
9032aa062b
testsuite: simplify ArchiverCorruptionTestCase
2017-05-31 18:08:20 +02:00
Marian Beermann
0a5d9b6f7c
cache sync: close archive chunks file before renaming
2017-05-31 18:06:28 +02:00
enkore
02689174ab
Merge pull request #2574 from edgimar/patch-1
...
Update changes.rst
2017-05-28 09:12:28 +02:00
edgimar
b57fb1a5e5
Update changes.rst
...
fix denglish
2017-05-28 00:52:56 -04:00
enkore
7548f525c8
Merge pull request #2569 from enkore/f/setup-cclean
...
setup.py clean to remove compiled files
2017-05-26 00:47:11 +02:00
enkore
7d7b2f3407
Merge pull request #2566 from enkore/issue/2560
...
Restrict key file names
2017-05-26 00:43:42 +02:00
Marian Beermann
0a295dd753
setup.py clean to remove compiled files
2017-05-25 17:58:57 +02:00
Marian Beermann
d35d388d9c
cache integrity: handle interference from old versions
2017-05-25 17:44:01 +02:00
Marian Beermann
50ac9d914d
testsuite: add ArchiverCorruptionTestCase
2017-05-25 17:44:01 +02:00
Marian Beermann
83bca02a4e
file_integrity: hash_part: mix length into state
2017-05-25 17:44:01 +02:00
Marian Beermann
d463dd89aa
hashindex: read/write: use hash_part for HashHeader
2017-05-25 17:44:01 +02:00
Marian Beermann
f59affe585
cache: fix possible printf issue with archive names in sync
2017-05-25 17:44:01 +02:00
Marian Beermann
addd7addfe
cache: chunks.archive.d: autofix corruption
2017-05-25 17:44:01 +02:00
Marian Beermann
1dfe693003
cache: integrity checking in archive.chunks.d
2017-05-25 16:28:46 +02:00
Marian Beermann
2b518b7188
cache: add integrity checking of chunks and files caches
2017-05-25 16:28:46 +02:00
Marian Beermann
06cf15cc6d
hashindex: read/write: accept file-like objects for path
2017-05-25 14:04:41 +02:00
Marian Beermann
39051ac5f1
file_integrity: split in IntegrityCheckedFile + Detached variant
2017-05-25 14:04:41 +02:00
Marian Beermann
38ed9a20af
key file names: limit to 100 characters (not bytes)
2017-05-25 12:36:45 +02:00
Marian Beermann
4b8a04b5e7
key file names: remove colons from host name
2017-05-25 12:32:42 +02:00