Commit Graph

4594 Commits

Author SHA1 Message Date
Marian Beermann 2766693706 fuse: update comments 2017-06-15 23:50:17 +02:00
enkore 7877aab198 Merge pull request #2678 from enkore/f/chunker-buzhash
chunker: fix invalid use of types
2017-06-15 09:05:44 +02:00
Marian Beermann faf2d0b537 chunker: fix invalid use of types
With the argument specified as unsigned char *, Cython emits
code in the Python wrapper to convert string-like objects to
unsigned char* (essentially PyBytes_AS_STRING).

Because the len(data) call is performed on a cdef'd string-ish type,
Cython emits a strlen() call, on the result of PyBytes_AS_STRING.

This is not correct, since embedded null bytes are entirely possible.

Incidentally, the code generated by Cython was also not correct,
since the Clang Static Analyzer found a path of execution where
passing arguments in a weird way from Python resulted in strlen(NULL).

Formulated like this, Cython emits essentially:

c_buzhash(
 PyBytes_AS_STRING(data),
 PyObject_Length(data),
 ...
)

which is correct.
2017-06-14 19:16:36 +02:00
Marian Beermann 3b928a4558 fuse: refactor ItemCache 2017-06-14 13:58:09 +02:00
Marian Beermann 9fd79a9e56 fuse: decrypted cache 2017-06-14 13:35:07 +02:00
Marian Beermann f04119c246 fuse: ItemCache on top of object cache 2017-06-14 13:35:07 +02:00
Marian Beermann ec532304d2 fuse: remove unnecessary normpaths 2017-06-14 13:34:34 +02:00
enkore 8e477414ee Merge pull request #2675 from enkore/f/fusecache
fuse: instrumentation
2017-06-14 09:31:55 +02:00
Marian Beermann ff05895b7e fuse: don't keep all Archive() instances around
they're only needed inside process_archive, and not needed in general
for pending_archives.
2017-06-13 23:16:28 +02:00
Marian Beermann 879f72f227 fuse: log process_archive timing
the easier alternative to "/bin/time stat mountpoint/<archive>/..."
2017-06-13 23:16:28 +02:00
Marian Beermann c791921951 fuse: instrument caches
note: signal processing can be arbitrarily delayed;
Python processes signals as soon as the code returns into the interpreter
loop, which doesn't happen unless libfuse returns control, i.e.
some request has been sent to the file system.
2017-06-13 23:16:28 +02:00
enkore ae162307f7 Merge pull request #2672 from enkore/f/fastinfo
info: use CacheSynchronizer & HashIndex.stats_against
2017-06-13 22:51:03 +02:00
TW b51198ebe7 Merge pull request #2673 from ThomasWaldmann/negative-ids
FUSE: fix negative uid/gid crash, fixes #2674
2017-06-13 19:53:54 +02:00
Thomas Waldmann ccd066f0af FUSE: fix negative uid/gid crash, fixes #2674
they could come into archives e.g. when backing up
external drives under cygwin.
2017-06-13 19:51:37 +02:00
TW 130e6771d0 Merge pull request #2665 from ThomasWaldmann/vagrant-openindiana
vagrant: add OpenIndiana
2017-06-13 19:08:20 +02:00
Marian Beermann e189a4d302 info: use CacheSynchronizer & HashIndex.stats_against 2017-06-13 14:34:10 +02:00
enkore fd1efbac90 Merge pull request #2671 from enkore/f/chunkerub
chunker: don't do uint32_t >> 32
2017-06-13 13:13:33 +02:00
enkore 6ce7b9505c Merge pull request #2670 from enkore/f/fcs-playnice
cache sync: don't do memcpy(..., 0, 0)
2017-06-13 13:13:06 +02:00
Marian Beermann 944a4abd58 chunker: don't do uint32_t >> 32 2017-06-13 11:42:43 +02:00
Marian Beermann 4490a8bbc3 cache sync: don't do memcpy(..., 0, 0)
!ctx->buf => ctx->tail - ctx->head == 0
2017-06-13 11:16:04 +02:00
enkore f3f3710279 Merge pull request #2669 from ThomasWaldmann/no-extract-stdout
don't write to disk with --stdout, fixes #2645
2017-06-13 09:25:13 +02:00
Thomas Waldmann 0b00c14c27 don't write to disk with --stdout, fixes #2645
if we always give stdout to extract_item(), it gets into the stdout-
processing branch which only emits data from items that have chunks
and does nothing for items which don't.
2017-06-13 01:12:14 +02:00
Marian Beermann c9c227f2ca cache sync: check Operation.READ compatibility with manifest 2017-06-12 23:46:49 +02:00
TW 2ff880ec6e Merge pull request #2668 from philippje/patch-1
changed the date of "day without backup"
2017-06-12 22:31:07 +02:00
philippje 88e937d0f9 changed the date of day without backup
Changed from 20. December to 19. December for easier comprehension (viewing the calendar.)
The missing 'd' at 20. December is hardly noticeable compared to e.g. the 19. December.
2017-06-12 22:17:29 +02:00
enkore fca40c62ed Merge pull request #2666 from enkore/f/fcs-gccwarn
cache sync: suppress GCC C90/C99 int literal warning
2017-06-12 11:28:18 +02:00
Marian Beermann 78cbf695c4 cache sync: suppress GCC C90/C99 int literal warning
warning: this decimal constant is unsigned only in ISO C90

Raised by GCC 4.9.2 on PowerPC.

The warning is bogus here due to the immediate explicit cast; newer
versions don't emit it.
2017-06-12 10:53:55 +02:00
Thomas Waldmann dff5f2041e vagrant: add OpenIndiana 2017-06-12 03:29:37 +02:00
TW 99b28a005a Merge pull request #2664 from ThomasWaldmann/vagrant-fix
Vagrantfile: backslash needs escaping
2017-06-12 03:09:29 +02:00
Thomas Waldmann a6dc6b611b Vagrantfile: backslash needs escaping 2017-06-12 03:04:31 +02:00
enkore 148a8a855b Merge pull request #2663 from enkore/f/fcs-changes
cache sync changes
2017-06-11 21:27:25 +02:00
enkore eb6deb9b05 Merge pull request #2642 from enkore/docs/deployment-local
docs: deployment: Automated backups to a local hard drive
2017-06-11 20:47:57 +02:00
Marian Beermann 501859ca33 docs: less bothersome experimental stripes 2017-06-11 20:46:53 +02:00
Marian Beermann 25bee21253 docs: deployment: Automated backups to a local hard drive 2017-06-11 20:46:53 +02:00
Marian Beermann 783a5926d6 cache sync: introduce BORG_NO_PYTHON
textshell edition
2017-06-11 20:23:17 +02:00
Marian Beermann 3c6372f841 cache sync: convert incoming integers to uint64_t 2017-06-11 20:20:00 +02:00
Marian Beermann f786211b12 RepositoryCache: truncate+unlink errored file 2017-06-11 20:20:00 +02:00
enkore 49d74e12fc Merge pull request #2661 from enkore/f/consistent-key-names
normalize authenticated key modes
2017-06-11 20:01:32 +02:00
enkore ffea54d197 Merge pull request #2658 from enkore/issue/2448
implement --glob-archives/-a
2017-06-11 19:24:10 +02:00
Marian Beermann 221dc1c4c7 normalize authenticated key modes
rename authenticated to authenticated-blake2, consistent with the other
blake2 key modes

add authenticated mode that fills the blank and is consistent with the
other "unqualified" key modes
2017-06-11 18:07:09 +02:00
enkore de00d9d822 Merge pull request #2602 from enkore/pr/2134.docs
internals: rewrite manifest & feature flags
2017-06-11 12:29:12 +02:00
Marian Beermann bffcc60f90 docs: internals: feature flags typos, clarifications 2017-06-11 12:28:37 +02:00
Marian Beermann 5f5371f0b1 implement --glob-archives/-a 2017-06-11 12:15:12 +02:00
enkore a7fb74dbb1 Merge pull request #2657 from enkore/issue/2656
cache sync: move assert() behind declarations
2017-06-11 11:43:41 +02:00
Marian Beermann 49ca3dca33 cache sync: move assert() behind declarations 2017-06-11 11:42:39 +02:00
textshell 86363dcd4b Merge pull request #2648 from textshell/feature/mandatory-features-master
Add minimal version of in repository mandatory feature flags. (master)
2017-06-10 17:50:28 +02:00
enkore 33a7331bda Merge pull request #2643 from enkore/f/experimental-patterns
mark --pattern, --patterns-from as experimental
2017-06-10 17:15:22 +02:00
enkore 13f396d5ad Merge pull request #2638 from enkore/f/fastcachesync-minify
Compact chunks.archive.d
2017-06-10 17:13:25 +02:00
enkore d06ee5648c Merge pull request #2572 from enkore/f/fastcachesync
Improve cache sync speed
2017-06-10 17:12:51 +02:00
Marian Beermann 4cd1cc6a28 docs: correct create exclude comment 2017-06-10 12:06:18 +02:00