mirror/vorta
1
0
Fork 0
mirror of https://github.com/borgbase/vorta synced 2025-03-11 14:49:16 +00:00
Code Issues Releases Wiki Activity

Refresh borgmatic projects.

Dan Helfman 2024-02-06 09:57:27 -08:00
parent 5450f3e80e
commit 6287c91ea5
1 changed files with 6 additions and 27 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

33
Google-Summer-of-Code-2024-Ideas.md

@ -181,15 +181,6 @@ Other issues tagged [help wanted](https://github.com/borgbackup/borg/issues?q=is
## Borgmatic
### Restore a database backup to a different database or server
**Difficulty**: Easy<br />
**Length**: 30 hours<br />
**Skills required**: Python, Linux, relational database administration basics (PostgreSQL and/or MySQL/MariaDB)<br />
**Description**: Today, borgmatic supports creating database backups and restoring those backups to the same database. However, this doesn't support use cases likes: 1. Running a test restore to another database or server without impacting the production database, or 2. In the event of a database loss, restoring a database to a newly created replacement server with a different hostname and/or credentials.<br />
**Task outline**: Add command-line support to borgmatic's restore action to override the restored database name, hostname, port username, password, etc. and plumb those values through to the restore commands. Additionally, manually test on all database types borgmatic supports, update the existing documentation, and add/update tests for these features.<br />
**Additional details**: See [the ticket](https://projects.torsion.org/borgmatic-collective/borgmatic/issues/326)<br />
**Possible mentors**: [@witten][witten], [@real-yfprojects][real-yfprojects] (backup mentor)
### Wrap all Borg sub-commands with borgmatic actions
**Difficulty**: Easy<br />
**Length**: 90 hours<br />
@ -197,25 +188,13 @@ Other issues tagged [help wanted](https://github.com/borgbackup/borg/issues?q=is
**Description**: borgmatic is effectively a wrapper around Borg backup, providing additional features like a configuration file, database integration, etc. But borgmatic only wraps a fraction of the sub-commands that Borg provides. And for those that it does wrap, it doesn't necessarily support all command-line flags as borgmatic options. Users can always drop back down to running Borg directly for those missing sub-commands (or use the `borgmatic borg` action), but that doesn't provide all the conveniences of borgmatic and its configuration file.<br />
**Task outline**: Implement borgmatic actions for all Borg sub-commands that are not yet implemented. For each Borg flag within those sub-commands, decide whether it makes sense to add a new borgmatic configuration option for it—or whether it would be more appropriate as a borgmatic action command-line flag. Also as part of this work, consider implementing missing flags/options on existing borgmatic actions.<br />
**Additional details**: Not all Borg sub-commands make sense to wrap. For instance, Borg invokes `borg serve` internally, and there's likely not a good use case for running it via borgmatic. Similarly, some Borg flags like `--info` and `--debug` shouldn't be exposed directly via borgmatic configuration options or command-line flags, because borgmatic uses them implicitly (e.g. via `--verbosity`) without exposing them to the end-user.<br />
**Possible mentors**: [@witten][witten], [@real-yfprojects][real-yfprojects] (backup mentor)
### Bootstrap a borgmatic restore from nothing
**Difficulty**: Medium<br />
**Length**: 80 hours<br />
**Skills required**: Python, Linux, relational database administration basics (PostgreSQL and/or MySQL/MariaDB)<br />
**Description**: borgmatic is great for restoring the occasional accidentally deleted file or database, but it's not as streamlined for the use case of spinning up an entire replacement system from nothing in the case of catastrophic server loss. That's because in order to function, borgmatic needs a configuration file with a list of Borg repositories to restore from, databases to restore, credentials, etc. <br />
**Task outline**: Design and implement a better, more streamlined workflow for bootstrapping the restore of backed up files and databases to a totally blank system. This could include approaches like: 1. Automatically storing borgmatic configuration in a canonical location in any backup archive such that it can be restored or used upon restore, 2. Automatically storing borgmatic database configuration along with each backed up database dump so as to facilitate restore, and/or 3. Implementing a new borgmatic action like `borgmatic bootstrap` to provide the user with an entry point to some of these features. Additionally, update the documentation and tests accordingly.<br />
**Additional details**: See a [related ticket](https://projects.torsion.org/borgmatic-collective/borgmatic/issues/345)<br />
**Possible mentors**: [@witten][witten], [@real-yfprojects][real-yfprojects] (backup mentor)
### Perform ZFS filesystem snapshots
**Difficulty**: Hard<br />
**Length**: 150 hours<br />
**Skills required**: Python, Linux, relational database administration basics (PostgreSQL and/or MySQL/MariaDB)<br />
**Description**: borgmatic creates backup archives by reading a collection of files on a filesystem. However, this can result in inconsistent backups if the files change as borgmatic/Borg are reading them, which may pose problems for some users / use cases. A common solution today is to take advantage of the snapshotting capabilities of various filesystems like ZFS to produce a consistent view of the filesystem from a particular point in time, but those snapshots are not natively integrated with borgmatic.<br />
**Task outline**: Implement native ZFS support in borgmatic such that a snapshot is created before backups occur and then the snapshot is removed / cleaned up afterwards. This should ideally be integrated with borgmatic via a new general-purpose filesystem snapshotting hook internal API (analogous to the existing database hook internal API already in place) so that subsequent projects can support other filesystems. Additionally, update the documentation and tests accordingly.<br />
**Additional details**: See [the ticket](https://projects.torsion.org/borgmatic-collective/borgmatic/issues/261)<br />
**Possible mentors**: [@witten][witten], [@real-yfprojects][real-yfprojects] (backup mentor)
### MySQL/MariaDB database directory format support
**Length**: 40 hours<br />
**Skills required**: Python, Linux, MySQL/MariaDB<br />
**Description**: Today borgmatic supports dumping MySQL/MariaDB databases directly to Borg for backup purposes and also restoring them directly from Borg. However, borgmatic does not support the MySQL/MariaDB "directory" format for database dumps currently.
**Task outline**: Implement a new "directory" value for the existing MySQL/MariaDB "format" configuration options. When that's set, pass the relevant option to `mysqldump` so that database dumps are dumped into a directory instead of as a single file. Note that this will have to bypass the existing streaming to named pipe logic since a directory can't be streamed that way. Also note that there are separate MySQL and MariaDB hooks, which would both need similar updates to support this feature.
**Additional details**: Look at the existing PostgreSQL hook for an example of a database hook that supports both streaming database dumps and non-streaming directory format database dumps. You can probably take a similar approach with this MySQL/MariaDB work.
Also see [good first issues](https://projects.torsion.org/borgmatic-collective/borgmatic/issues?q=&type=all&state=open&labels=52&milestone=0&assignee=0&poster=0).