mirror
/
restic
mirror of https://github.com/restic/restic.git
1
0
Fork 0
Code Issues Releases Wiki Activity
255 Commits 7 Branches 48 Tags 92 MiB
Commit Graph

11 Commits

Author SHA1 Message Date
Alexander Neumann 3daeee8e2c Make chunker resettable 2015-02-09 23:37:33 +01:00
Alexander Neumann bda33e612c Implement streaming chunker using io.Reader 2015-02-09 00:28:54 +01:00
Alexander Neumann a5c33d80d8 Chunker: remove pool, buf and make bufsize an option 2015-02-08 19:32:12 +01:00
Alexander Neumann bdcdcdea7d Optimize chunker 
Skip MinSize bytes at the beginning of each chunk. This increases
throughput about 30% percent.

Before:
    $ go test -v -bench . ./chunker
    === RUN TestChunker
    --- PASS: TestChunker (0.60s)
    === RUN TestChunkerReuse
    --- PASS: TestChunkerReuse (2.18s)
    PASS
    BenchmarkChunker          10     111229642 ns/op      94.27 MB/s
    --- BENCH: BenchmarkChunker
        chunker_test.go:168: 6 chunks, average chunk size: 1747626 bytes
        chunker_test.go:168: 6 chunks, average chunk size: 1747626 bytes
    ok      github.com/restic/restic/chunker    4.120s

After:
    $ go test -v -bench . ./chunker
    === RUN TestChunker
    --- PASS: TestChunker (0.48s)
    === RUN TestChunkerReuse
    --- PASS: TestChunkerReuse (1.75s)
    PASS
    BenchmarkChunker          20      81468596 ns/op     128.71 MB/s
    --- BENCH: BenchmarkChunker
        chunker_test.go:168: 6 chunks, average chunk size: 1747626 bytes
        chunker_test.go:168: 6 chunks, average chunk size: 1747626 bytes
    ok      github.com/restic/restic/chunker    4.061s
 2015-01-14 16:33:41 +01:00
Alexander Neumann e2fea0d088 Rename khepri -> restic 2014-12-05 21:45:49 +01:00
Alexander Neumann 9dc0bf6378 Fix reusing chunker 2014-12-03 23:20:45 +01:00
Alexander Neumann f95788ed90 Limit memory usage, add several sync.Pool 2014-11-23 21:26:20 +01:00
Alexander Neumann 83ea81d8c3 Fix special case with null byte input 2014-10-03 21:44:39 +02:00
Alexander Neumann 878a96e761 Fix missing byte 2014-09-18 22:34:08 +02:00
Alexander Neumann 8e24a8b811 Add more documentation and error handling 2014-09-18 22:20:12 +02:00
Alexander Neumann 18131f88bf Add code for content defined chunking (CDC) 2014-09-18 21:10:20 +02:00