mirror
/
restic
mirror of https://github.com/restic/restic.git
1
0
Fork 0
Code Issues Releases Wiki Activity
485 Commits 7 Branches 49 Tags 76 MiB
Commit Graph

73 Commits

Author SHA1 Message Date
Alexander Neumann 60a0fe8349 Major restructure, bundle blobs 
This commit also breaks the repository format.
 2015-04-29 23:27:32 +02:00
Alexander Neumann d19b23d4f1 Move Server and Key to new sub-package 2015-04-26 14:46:15 +02:00
Alexander Neumann 434c79354e Merge pull request #143 from fw42/count_errors 
Count errors separately and not as files that are "done"
 2015-04-26 12:05:09 +02:00
Florian Weingarten 61b1a5b1de Count errors separately and not as files that are "done" 2015-04-26 01:54:35 +00:00
Florian Weingarten c947623707 Fix progress off-by-1 2015-04-25 15:40:42 -04:00
Florian Weingarten 262e5a8b4a O_NOATIME on Linux 2015-04-25 00:45:20 +00:00
Alexander Neumann b2307cafa2 Seed chunker with random per-repository polynomial 2015-04-06 00:22:19 +02:00
Alexander Neumann 24b14e21cc Remove pools for nodes and IDs 
This removes the allocation pools for nodes and IDs. I feel they aren't
really needed. Benchmarks:

    benchmark                         old ns/op      new ns/op      delta
    BenchmarkChunkEncrypt             197890867      198616293      +0.37%
    BenchmarkChunkEncryptParallel     196127004      198819818      +1.37%
    BenchmarkArchiveDirectory         1098848419     1087237723     -1.06%
    BenchmarkPreload                  30464455       29910239       -1.82%
    BenchmarkLoadTree                 3265092        3088543        -5.41%
    BenchmarkEncryptWriter            37213511       37134683       -0.21%
    BenchmarkEncrypt                  36037879       36166546       +0.36%
    BenchmarkDecryptReader            38165659       38556734       +1.02%
    BenchmarkEncryptDecryptReader     77027044       77194987       +0.22%
    BenchmarkDecrypt                  36017602       35937888       -0.22%
    BenchmarkSaveJSON                 47906          50270          +4.93%
    BenchmarkSaveFrom                 49775973       50520969       +1.50%
    BenchmarkLoadJSONID               105290245      107281849      +1.89%
    BenchmarkChunkerWithSHA256        151501430      148264078      -2.14%
    BenchmarkChunkerWithMD5           93606346       94036392       +0.46%
    BenchmarkChunker                  74285431       75933882       +2.22%
    BenchmarkPipelineWalker           387689         346467         -10.63%

    benchmark                         old MB/s     new MB/s     speedup
    BenchmarkChunkEncrypt             52.99        52.79        1.00x
    BenchmarkChunkEncryptParallel     53.46        52.74        0.99x
    BenchmarkEncryptWriter            225.42       225.90       1.00x
    BenchmarkEncrypt                  232.77       231.94       1.00x
    BenchmarkDecryptReader            219.79       217.57       0.99x
    BenchmarkEncryptDecryptReader     108.90       108.67       1.00x
    BenchmarkDecrypt                  232.90       233.42       1.00x
    BenchmarkSaveFrom                 84.26        83.02        0.99x
    BenchmarkChunkerWithSHA256        69.21        70.72        1.02x
    BenchmarkChunkerWithMD5           112.02       111.51       1.00x
    BenchmarkChunker                  141.15       138.09       0.98x

    benchmark                         old allocs     new allocs     delta
    BenchmarkChunkEncrypt             110            110            +0.00%
    BenchmarkChunkEncryptParallel     100            100            +0.00%
    BenchmarkArchiveDirectory         475591         476635         +0.22%
    BenchmarkPreload                  28059          24182          -13.82%
    BenchmarkLoadTree                 3124           2889           -7.52%
    BenchmarkEncryptWriter            19             19             +0.00%
    BenchmarkEncrypt                  13             13             +0.00%
    BenchmarkDecryptReader            16             15             -6.25%
    BenchmarkEncryptDecryptReader     39             39             +0.00%
    BenchmarkDecrypt                  11             11             +0.00%
    BenchmarkSaveJSON                 74             74             +0.00%
    BenchmarkSaveFrom                 109            112            +2.75%
    BenchmarkLoadJSONID               103630         97849          -5.58%
    BenchmarkChunkerWithSHA256        13             13             +0.00%
    BenchmarkChunkerWithMD5           12             12             +0.00%
    BenchmarkChunker                  6              6              +0.00%
    BenchmarkPipelineWalker           212            165            -22.17%

    benchmark                         old bytes     new bytes     delta
    BenchmarkChunkEncrypt             64697         64697         +0.00%
    BenchmarkChunkEncryptParallel     64681         64681         +0.00%
    BenchmarkArchiveDirectory         193385504     193790864     +0.21%
    BenchmarkPreload                  4064701       3942000       -3.02%
    BenchmarkLoadTree                 344954        325396        -5.67%
    BenchmarkEncryptWriter            12793         12793         +0.00%
    BenchmarkEncrypt                  1950          1950          +0.00%
    BenchmarkDecryptReader            3120          2774          -11.09%
    BenchmarkEncryptDecryptReader     1528036       1528036       +0.00%
    BenchmarkDecrypt                  1919          1919          +0.00%
    BenchmarkSaveJSON                 5524          5524          +0.00%
    BenchmarkSaveFrom                 31353         40804         +30.14%
    BenchmarkLoadJSONID               12872020      16010968      +24.39%
    BenchmarkChunkerWithSHA256        26821         26821         +0.00%
    BenchmarkChunkerWithMD5           13554         13554         +0.00%
    BenchmarkChunker                  13458         13458         +0.00%
    BenchmarkPipelineWalker           58584         55560         -5.16%
 2015-03-28 16:51:37 +01:00
Alexander Neumann f157f775da Add checks for Server.Load(), use Blob for load 2015-03-28 15:07:08 +01:00
Alexander Neumann 5e69788eac Refactor backends 2015-03-28 14:36:50 +01:00
Alexander Neumann cfa2229bc0 Refactor cache refresh for blobs, add progress 2015-03-22 15:09:50 +01:00
Alexander Neumann aca0692ee6 Add more error reporting 2015-03-21 14:43:33 +01:00
Alexander Neumann 71a57537ef Report errors, ignore files and continue. Closes #108 2015-03-15 15:48:05 +01:00
Alexander Neumann 3d768e39ea Add more error reporting 2015-03-15 12:20:30 +01:00
Alexander Neumann 3c92c7e689 Recreate blob cache if missing (closes #104) 2015-03-14 18:15:47 +01:00
Alexander Neumann e1fc17aeb1 Implement per-repository cache 2015-03-14 12:16:28 +01:00
Alexander Neumann 7b54c21726 Store ID in Snapshot 2015-03-10 14:17:52 +01:00
Alexander Neumann 2166367039 Fix comparison for new files with '.' 2015-03-10 14:17:50 +01:00
Alexander Neumann 14298fe232 Cache blobs for each snapshot 2015-03-09 23:50:32 +01:00
Alexander Neumann ab8b97c4ba Remove code duplication 
The top-level tree is now handled by the DirWorkers
 2015-03-08 21:06:51 +01:00
Alexander Neumann b3cdd3ead9 Remove unused code 2015-03-08 14:22:35 +01:00
Alexander Neumann ba892e1ec2 Refactor backup pipeline 2015-03-08 14:16:01 +01:00
Alexander Neumann f167366be5 Add walk for trees, restructure 2015-03-02 19:58:06 +01:00
Alexander Neumann c8be54564f Add local cache 2015-03-02 10:04:01 +01:00
Alexander Neumann a59b0ec1f6 Add progress report for loading blobs 2015-02-21 15:32:48 +01:00
Alexander Neumann 58cded6b75 Refactor progress reporting 2015-02-21 14:25:06 +01:00
Alexander Neumann 97b963ef1d Report files and dirs 2015-02-21 14:05:02 +01:00
Alexander Neumann a0fea201d9 Reenable Preloading 2015-02-18 23:10:59 +01:00
Alexander Neumann f8e47181b0 Add more debug for preloading 2015-02-17 23:38:40 +01:00
Alexander Neumann 677a5038c5 Increase concurrency for loading blobs 2015-02-17 23:15:08 +01:00
Alexander Neumann 4bb7f2f2ed Add Preloader for blobs 2015-02-17 22:39:44 +01:00
Alexander Neumann 4fd8222da6 Remove unneeded buffers from Archiver.SaveFile() 2015-02-16 23:47:32 +01:00
Alexander Neumann 06ed5c12b8 Remove Compress() and Uncompress() from backend 2015-02-16 00:30:51 +01:00
Alexander Neumann f8f8107d55 wip 2015-02-15 15:17:24 +01:00
Alexander Neumann bbb3145f11 Add chunker pool 2015-02-09 23:38:50 +01:00
Alexander Neumann bda33e612c Implement streaming chunker using io.Reader 2015-02-09 00:28:54 +01:00
Alexander Neumann 25a214809b Refactor debug into debug module 2015-01-17 16:32:13 +01:00
Alexander Neumann af43fb3dda Fix handling vanished files 2015-01-14 17:48:23 +01:00
Alexander Neumann 4e54c6af2c Fix calls to debug() 2015-01-14 17:48:10 +01:00
Alexander Neumann 48751e1935 Introduce debug tags and debug breaks 
The environment variable DEBUG_TAGS can be used to control what is
printed on stderr. Example:

    DEBUG_TAGS="+all,-Archiver.*" ./restic backup foo

The variable DEBUG_BREAK can be used to set a breakpoint (implemented
using SIGSTOP):

    DEBUG_BREAK=Archiver.Snapshot ./restic backup foo
 2015-01-14 16:34:30 +01:00
Alexander Neumann 203a911de9 Refactor repository structure 
Merge Map data type into Tree.
 2015-01-14 16:34:30 +01:00
Alexander Neumann cdf3336e7a Improve error message for newer files 2015-01-08 20:49:32 +01:00
Alexander Neumann 33bcf31bae Fix incremental backup 
Copying blobs (for unchanged files and subtrees) from old BlobList to
new BlobList was missing
 2015-01-05 21:40:43 +01:00
Alexander Neumann 6eb969a492 Update modified files, store error message 2015-01-04 23:25:13 +01:00
Alexander Neumann e543f5926c Refactor Archiver and backup command 
Improve incremental backup by first scanning the tree, loading the old
tree and afterwards comparing both trees in memory.
 2015-01-04 22:39:30 +01:00
Alexander Neumann fe231af7fc Remove error from return value of NewContentHandler 2015-01-04 20:07:30 +01:00
Alexander Neumann 4b70bba588 Split Scanner from Archiver, refactor Progress 2015-01-04 18:23:00 +01:00
Alexander Neumann a93bc3c991 Rename functions, remove code for compare 
* Archiver: Rename loadTree -> scan and LoadTree -> Scan
 * Archiver: Remove code to compare against an old snapshot/tree,
   the current implementation was really slow
 2015-01-04 16:13:50 +01:00
Alexander Neumann ef41a77aff Remove explicit key handling 2014-12-21 18:10:19 +01:00
Alexander Neumann cc147c002e Introduce type Server 2014-12-21 17:37:29 +01:00