psw/borgmatic
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
1780 commits 2 branches 0 tags 3.6 MiB
Commit graph

369 commits

Author SHA1 Message Date
Dan Helfman
317dc7fbce Add "before_actions" and "after_actions" command hooks that run before/after all the actions for each repository, update docs to cover per-repository configurations (#463). 2022-08-21 21:48:37 -07:00
Dan Helfman
462326406e Drop only-style actions like "--create", rename "prune --files" to "prune --list", and add "--list" alias to "create" and "export-tar" (#571). 2022-08-21 14:25:16 -07:00
Dan Helfman
540f9f6b72 Add missing test for "transfer" action (#557). 2022-08-19 09:40:29 -07:00
Dan Helfman
f9d7faf884 Fix mount action to work without archive again (#557). 2022-08-18 23:33:05 -07:00
Dan Helfman
7dee6194a2 Add new "transfer" action for Borg 2 (#557). 2022-08-18 23:06:51 -07:00
Dan Helfman
43d711463c Add additional command-line flags to rcreate action (#557). 2022-08-18 14:28:12 -07:00
Dan Helfman
b40e9b7da2 Ignore archive filter parameters passed to list action when --archive is given (#557). 2022-08-18 09:59:48 -07:00
Dan Helfman
89d201c8ff Fleshing out NEWS for the Borg 2 changes. 2022-08-17 21:54:00 -07:00
Dan Helfman
f47c98c4a5 Rename several configuration options to match Borg 2 (#557). 2022-08-17 21:14:58 -07:00
Dan Helfman
3b6ed06686 Add --other-repo flag to rcreate action (#557). 2022-08-17 17:33:09 -07:00
Dan Helfman
57009e22b5 Use flag-related utility functions in info action (#557). 2022-08-17 17:11:02 -07:00
Dan Helfman
3ab7a3b64a Replace use of --prefix with --glob-archives in info action (#557). 2022-08-17 15:36:19 -07:00
Dan Helfman
596dd49cf5 Use --glob-archives instead of --prefix on rlist command (#557). 2022-08-17 14:26:35 -07:00
Dan Helfman
28d847b8b1 Warn and tranform on non-ssh://-style repositories (#557). 2022-08-17 10:13:11 -07:00
Dan Helfman
30abd0e3de Update borg action for Borg 2 support (#557). 2022-08-16 09:30:00 -07:00
Dan Helfman
f36e38ec20 Update mount action for Borg 2 support (#557). 2022-08-15 19:32:37 -07:00
Dan Helfman
d807ce095e Update export-tar action for Borg 2 support (#557). 2022-08-15 17:34:12 -07:00
Dan Helfman
7626fe1189 Disallow borg list --json with --archive or --find (#557). 2022-08-15 15:40:28 -07:00
Dan Helfman
cc04bf57df Update list action for Borg 2 support, add rinfo action, and update extract consistency check for Borg 2. 2022-08-15 15:04:40 -07:00
Dan Helfman
cce6d56661 Update extract action for Borg 2 support (#557). 2022-08-13 23:07:29 -07:00
Dan Helfman
a05d0f378e Factor out repository/archive flags formatting code from create action (#557). 2022-08-13 22:50:14 -07:00
Dan Helfman
94321aec7a Update compact action for Borg 2 support (#557). 2022-08-13 22:07:15 -07:00
Dan Helfman
4a55749bd2 Update prune action for Borg 2 support (#557). 2022-08-13 17:26:51 -07:00
Dan Helfman
2898e63166 Update create action for Borg 2 support (#557). 2022-08-12 23:54:13 -07:00
Dan Helfman
c7176bd00a Add rinfo action for Borg 2 support (#557). 2022-08-12 23:06:56 -07:00
Dan Helfman
647ecdac29 Borg 2 support in borgmatic check action (#557). 2022-08-12 15:46:33 -07:00
Dan Helfman
e7a8acfb96 Add missing rinfo action source files (#557). 2022-08-12 14:59:03 -07:00
Dan Helfman
622caa0c21 Support for Borg 2's rcreate and rinfo sub-commands (#557). 2022-08-12 14:53:20 -07:00
Dan Helfman
9aece3936a Modify "mount" and "extract" actions to require the "--repository" flag when multiple repositories are configured (#566). 2022-07-25 11:30:02 -07:00
Dan Helfman
bcad0de1a4
Add verify_tls option for Healthchecks to optionally disable TLS verification. 2022-07-23 23:11:41 -07:00
Uli
5c6407047f feat: add verify_tls flag for Healthchecks 2022-07-24 07:37:00 +02:00
Dan Helfman
6ddae20fa1 Fix handling of "repository" and "data" consistency checks to prevent invalid Borg flags (#565). 2022-07-23 21:02:21 -07:00
Dan Helfman
16066942e3 Fix traceback with "create" action and "--json" flag when a database hook is configured (#563). 2022-07-19 10:25:10 -07:00
Dan Helfman
e85d551eac Fix all database hooks to error when the requested database to restore isn't present in the Borg archive (#560). 2022-07-06 23:21:24 -07:00
Dan Helfman
457ed80744 Fix environment variable plumbing so options in one configuration file aren't used for others (#555). 2022-06-30 13:42:17 -07:00
Dan Helfman
10723efc68 Fix all monitoring hooks to warn if the server returns an HTTP 4xx error (#554). 2022-06-29 21:19:40 -07:00
Dan Helfman
d372a86fe6 Code formatting. 2022-06-23 10:41:04 -07:00
Sébastien MB
b63c854509 Fix escaped environment variable in configuration 
- when an env variable is escaped in the configuration file, we expect
  not to resolve it and remove the escape char `\`
 2022-06-17 09:50:56 +02:00
Dan Helfman
aecb6fcd74 Code style, rename command-line flag, and move new code into its own file (#546) 2022-06-16 11:35:24 -07:00
Sébastien MB
97b5cd089d Allow environment variable resolution in configuration file 
- all string fields containing an environment variable like ${FOO} will
  be resolved
- supported format ${FOO}, ${FOO:-bar} and ${FOO-bar} to allow default
  values if variable is not present in environment
- add --no-env argument for CLI to disable the feature which is enabled
  by default

Resolves: #546
 2022-06-16 18:52:54 +02:00
Gavin Chappell
a8b8d507b6
add a hook for sending push notifications via ntfy.sh 2022-06-09 21:10:38 +01:00
Dan Helfman
3561c93d74 Fix Healthchecks tests that leak global state, breaking downstream tests (discovered in #543). 2022-06-09 11:05:44 -07:00
Dan Helfman
9aefb5179f Fix None find paths (#541). 2022-06-03 15:20:05 -07:00
Dan Helfman
d14f22e121 Add "borgmatic list --find" flag for searching for files across multiple archives (#541). 2022-06-03 15:12:14 -07:00
Dan Helfman
80ec3e7d97 Deprecate "borgmatic list --successful" flag, as listing only non-checkpoint (successful) archives is now the default in newer versions of Borg. 2022-06-02 20:35:39 -07:00
Dan Helfman
5b615d51a4 Add support for "borgmatic borg debug" command (#538). 2022-05-29 15:43:03 -07:00
Dan Helfman
8fa90053cf Add "borgmatic check --force" flag to ignore configured check frequencies (#523). 2022-05-28 19:29:33 -07:00
Dan Helfman
ad0e2e0d7c Tweak default check frequency to 1 month (#523). 2022-05-28 15:49:50 -07:00
Dan Helfman
e76bfa555f Reduce the default consistency check frequency and support configuring the frequency independently for each check (#523). 2022-05-28 14:42:19 -07:00
Dan Helfman
8ddb7268eb Reuse "borg info" function. 2022-05-27 13:51:11 -07:00
Dan Helfman
54933ebef5 Change connection failures for monitoring hooks to be warnings instead of errors (#439). 2022-05-24 15:50:04 -07:00
Dan Helfman
157e59ac88 Add Healthchecks monitoring hook "send_logs" option to enable/disable sending borgmatic logs to the Healthchecks server (#460). 2022-05-24 14:44:33 -07:00
Dan Helfman
8b179e4647 Reverse logic of Healtchecks "skip_states" option to just "states" (#525). 2022-05-24 14:09:42 -07:00
Dan Helfman
865eff7d98 Add Healthchecks monitoring hook "skip_states" option to disable pinging for particular monitoring states (#525). 2022-05-24 13:59:28 -07:00
Dan Helfman
b9741f4d0b Add Healthchecks monitoring hook "ping_body_limit" option to configure how many bytes of logs to send to the Healthchecks server (#294). 2022-05-24 12:23:38 -07:00
Dan Helfman
02781662f8 Change monitoring hooks to specify the ping URL / integration key as a named option. 2022-05-23 20:02:10 -07:00
Dan Helfman
32a1043468 Remove the error when "archive_name_format" is specified but a retention prefix isn't (#402). 2022-05-23 16:11:24 -07:00
Dan Helfman
3e4aeec649 Warn when an unsupported variable is used in a hook command (#420). 2022-05-23 15:27:54 -07:00
Dan Helfman
0e6b425ac5 Fix "borgmatic borg key ..." to pass parameters to Borg in correct order (#515). 2022-04-23 14:03:15 -07:00
Dan Helfman
ed7fe5c6d0 Instead of executing "before" command hooks before all borgmatic actions run (and "after" hooks after), execute these hooks right before/after the corresponding action (#473). 2022-04-21 22:08:25 -07:00
Dan Helfman
0c027a3050 Fix handling of TERM signal to exit borgmatic, not just forward the signal to Borg (#516). 2022-04-03 13:12:48 -07:00
Dan Helfman
9f44bbad65 Fix borgmatic exit code (so it's zero) when initial Borg calls fail but later retries succeed (#517). 2022-04-02 22:28:41 -07:00
Dan Helfman
976516d0e1 When loading a configuration file that is unreadable due to file permissions, warn instead of erroring (#444). 2022-03-08 10:19:36 -08:00
Dan Helfman
574eb91921 Fix Borg usage error in the "compact" action when running "borgmatic --dry-run". Now, skip "compact" entirely during a dry run (#507). 2022-03-07 21:46:12 -08:00
Dan Helfman
28fef3264b Fix handling of "patterns_from" and "exclude_from" options to error instead of warning when referencing unreadable files and running "create" action (#486). 2022-03-07 15:32:07 -08:00
Dan Helfman
9161dbcb7d Removing unnecessary leading underscores from functions. 2022-03-07 11:58:29 -08:00
Dan Helfman
4b3027e4fc Add test for new working_directory option (#431). 2022-03-03 11:48:18 -08:00
Dan Helfman
0eb2634f9b Working directory option to support source directories with relative paths (#431). 
Reviewed-on: https://projects.torsion.org/borgmatic-collective/borgmatic/pulls/477
 2022-03-03 19:28:17 +00:00
Dan Helfman
9317cbaaf0 Code formatting. 2022-02-10 10:23:34 -08:00
Dan Helfman
1b5f04b79f When using the "remote_rate_limit" option, tailor the flags passed to Borg depending on the Borg version (#394). 2022-02-10 10:16:09 -08:00
Dan Helfman
948c86f62c When using the "numeric_owner" option with the "extract" action, tailor the flags passed to Borg depending on the Borg version (#394). 2022-02-10 10:09:18 -08:00
Dan Helfman
7e7209322a When using the "numeric_owner" option, tailor the flags passed to Borg depending on the Borg version (#394). 2022-02-10 09:51:13 -08:00
Dan Helfman
00a57fd947 Code formatting. 2022-02-09 21:20:28 -08:00
Dan Helfman
6bf6ac310b When using the "bsd_flags" option, tailor the flags passed to Borg depending on the Borg version (#394). 2022-02-09 21:11:00 -08:00
Dan Helfman
4b5af2770d When the "atime" option is used, tailor the flags passed to Borg depending on version (#394). 2022-02-09 16:54:35 -08:00
Dan Helfman
b525e70e1c Run "compact" action by default when no actions are specified (#394). 2022-02-09 14:33:12 -08:00
Dan Helfman
48f44d2f3d Add tests for compact action (#394). 2022-02-08 12:05:02 -08:00
Dan Helfman
bec73245e9 Fix traceback when a YAML validation error occurs (#480, #482). 2022-01-19 20:39:03 -08:00
fabianschilling
b39f08694d Merge branch 'master' into pr-working-directory 2022-01-05 09:30:27 +00:00
Dan Helfman
2ee75546f5 Add MongoDB database hook documentation. 2022-01-04 16:26:38 -08:00
Andrea Ghensi
7c6ce9399c fix integration tests and mongodb auth 2021-12-29 22:18:50 +01:00
Andrea Ghensi
6b7653484b Add mongodb dump hook 2021-12-26 01:00:58 +01:00
Fabian Schilling
85e0334826 Add missing working_directory arg to pass tests 2021-12-10 18:24:41 +01:00
Dan Helfman
a1673d1fa1 Fix unicode error when restoring particular MySQL databases (#476). 2021-12-08 16:40:25 -08:00
Dan Helfman
7a086d8430 Fix import ordering. 2021-11-29 14:00:14 -08:00
Dan Helfman
0e8e9ced64 When command-line configuration override produces a parse error, error cleanly (#471). 2021-11-29 12:49:21 -08:00
nebulon42
c3e76585fc
move mysqldump options to the beginning of the command due to MySQL bug 30994. 2021-11-26 17:16:03 +01:00
nebulon42
3729ba5ca3
add list_options setting, fixes #306 2021-11-20 15:43:58 +01:00
Dan Helfman
38ebfd2969 Rename retry_timeout to retry_wait and standardize log formatting (#28). 2021-11-15 11:51:17 -08:00
cadamswaite
976a877a25 Formatting 2021-11-14 22:37:42 +00:00
cadamswaite
b4117916b8 Add timeout and tests 2021-11-14 22:15:22 +00:00
cadamswaite
19cad89978 Add some tests for retry logic 2021-11-14 21:35:23 +00:00
Dan Helfman
449896f661 Fix error when configured source directories are not present on the filesystem at the time of backup (#387). 2021-10-11 10:40:10 -07:00
Dan Helfman
54d57e1349 Add test for #407: Fix syslog logging on FreeBSD. 2021-06-23 10:21:45 -07:00
Dan Helfman
27d37b606b Better error messages! Switch the library used for validating configuration files (from pykwalify to jsonschema). 2021-06-22 13:27:59 -07:00
Dan Helfman
cf8882f2bc Run arbitrary Borg commands with new "borgmatic borg" action (#425). 2021-06-17 20:41:44 -07:00
Dan Helfman
9fd28d2eed Fix error handling to error loudly when Borg gets killed due to running out of memory (#423)! 2021-06-08 11:43:55 -07:00
Dan Helfman
ea6cd53067 Update versions of test dependencies (test_requirements.txt and test containers). 2020-11-22 14:48:07 -08:00
François Poulain
ebac02f118 Mention placeholders for repositories in documentation; fix #369 2020-11-18 20:22:41 +01:00
Dan Helfman
0e978299cf Fix traceback when a database hook value is null in a configuration file (#355). 2020-10-07 15:41:45 -07:00
networkjanitor
3c723e8d99 Merge branch 'master' into master 2020-08-12 21:11:45 +00:00
Dan Helfman
72bd96c656 Fix traceback when a configuration directory is non-readable due to directory permissions (#350). 2020-08-12 11:32:00 -07:00
networkjanitor
893fca2816 Add before_extract and after_extract hooks 2020-08-01 16:08:32 +02:00
Dan Helfman
b3fd1be5f6 Add "borgmatic export-tar" action to export an archive to a tar-formatted file or stream (#300). 2020-07-22 21:53:55 -07:00
Dan Helfman
17c2d109e5 Add tests for pass-through of BORG_* environment variables. 2020-06-21 14:41:22 -07:00
Dan Helfman
32e15dc905 Add a few more mocks to PostgreSQL SSL tests. 2020-06-20 14:39:16 -07:00
Dan Helfman
f5ebca4907 Add SSL support to PostgreSQL database configuration (#331). 
Reviewed-on: https://projects.torsion.org/witten/borgmatic/pulls/331
 2020-06-20 21:24:14 +00:00
Edward Shornock
d2d92b1f1a Add tests for the PostgreSQL SSL options 2020-06-20 23:32:24 +03:00
Dan Helfman
27cbe9dfc0 Fix for potential data loss (data not getting backed up) when borgmatic omitted configured source directories in certain situations (#333). 2020-06-19 20:16:38 -07:00
Edward Shornock
abd47fc14e Add SSL support to PostgreSQL hooks 2020-06-19 02:19:17 +03:00
Dan Helfman
aa14449857 Add "borgmatic extract --strip-components" flag to remove leading path components when extracting an archive (#324). 2020-06-06 14:57:14 -07:00
Dan Helfman
a155eefa23 Fix for certain configuration options like ssh_command impacting Borg invocations for separate configuration files (#323). 2020-06-06 14:30:04 -07:00
Dan Helfman
d7277893fb Fix hang when a stale database dump named pipe from an aborted borgmatic run remains on disk (#316). 2020-06-02 12:40:32 -07:00
Dan Helfman
097a09578a Fix enabled database hooks to implicitly set one_file_system configuration option to true to prevent Borg hang. (#315). 2020-05-26 09:20:14 -07:00
Dan Helfman
65472c8de2 Fix error message when there are no MySQL databases to dump for "all" databases (#319). 2020-05-26 08:59:04 -07:00
Dan Helfman
96df52ec50 Fix hang when streaming a database dump to Borg with implicit duplicate source directories by deduplicating them first (#316). 2020-05-20 13:33:53 -07:00
Dan Helfman
89cb5eb76d Fix regression in support for PostgreSQL's "directory" dump format (#314). 2020-05-18 11:31:29 -07:00
Dan Helfman
3e5a19d95a Add missing test coverage. 2020-05-14 23:21:43 -07:00
Dan Helfman
d88f321cef Fix legitimate database dump command errors (exit code 1) not being treated as errors by borgmatic (#310). 2020-05-14 22:38:38 -07:00
Dan Helfman
15ea70a71b Flesh out missing test coverage for execute.py. 2020-05-12 15:49:49 -07:00
Dan Helfman
8b91c01a4c Add some missing test coverage. 2020-05-11 11:34:14 -07:00
Dan Helfman
695c764a01 Merge log output functions into one. 2020-05-11 10:55:50 -07:00
Dan Helfman
1ea047dd94 Remove "borgmatic restore --progress" flag, as it now conflicts with streaming database restores. 2020-05-09 21:53:16 -07:00
Dan Helfman
4b523f9e2c Make database restore output only show at verbosity 2. 2020-05-08 19:38:33 -07:00
Dan Helfman
6a61070d85 Use shell redirection rather than the --file flag to sidestep synchronization issues when pg_dump/pg_dumpall tries to write to a named pipe. 2020-05-08 19:11:26 -07:00
Dan Helfman
f36082938e Additional test coverage. 2020-05-08 09:48:04 -07:00
Dan Helfman
1ba996ad93 Additional test coverage. 2020-05-07 12:14:27 -07:00
Dan Helfman
a23fdf946d Stream database dumps and restores directly to/from Borg without using any additional filesystem space (#258). 2020-05-07 11:44:04 -07:00
Dan Helfman
12cf6913ef Remove unused function parameter. 2020-05-03 14:07:34 -07:00
Dan Helfman
e4eff0e3dc Bump version for release. 2020-04-24 15:56:56 -07:00
Dan Helfman
dce1928dc4
Fix PostgreSQL restore error on "all" database dump. 2020-04-24 15:50:33 -07:00
Nathan Beals
3c8dc4929f Added test_restore_all_database_dump unit test. 
Updated the other unit tests, as I had to re-arrange argument order
Added an 'all' test for the postgres end-to-end test.

Ran black formatter on it all.
 2020-04-24 18:32:53 -04:00
Dan Helfman
e511014a28 Fix MySQL restore error on "all" database dump by excluding system tables (#301). 2020-04-22 12:17:22 -07:00
Dan Helfman
55141bda67 Specify "--archive latest" to all actions that accept an archive (#289). 2020-01-29 16:59:02 -08:00
Dan Helfman
bc02c123e6 Monitor backups with PagerDuty hook integration (#245). 2020-01-27 15:32:09 -08:00
Dan Helfman
e76d5ad988 Fix tests. 2020-01-27 12:56:12 -08:00
Dan Helfman
2405e97c38 Backup to a removable drive or intermittent server via "soft failure" feature (#284). 2020-01-24 20:52:48 -08:00
Dan Helfman
fdbb2ee905 View consistency check progress via "--progress" flag for "check" action (#287). 2020-01-24 11:27:16 -08:00
Dan Helfman
94b9ef56be Change "exclude_if_present" option to support multiple filenames, rather than just a single filename (#280). 2020-01-23 13:41:37 -08:00
Dan Helfman
53e6ff9524 No longer list files or show stats by default at verbosity 2. 2020-01-22 15:23:49 -08:00
Dan Helfman
f66fd1caaa Customize Healthchecks log level via borgmatic "--monitoring-verbosity" flag (#277). 2020-01-22 15:10:47 -08:00
Dan Helfman
d93fdbc5ad Support "--files" and "--stats" flags at verbosity level 0. 2020-01-22 13:28:24 -08:00
palto42
75b5e7254e changes as per comments in PR #283 2020-01-22 19:03:26 +01:00
palto42
5f0c084bee Merge 'upstream/master' into list-files 2020-01-22 18:12:26 +01:00
Dan Helfman
7824a034ca Add test for database dump directory removal. 2020-01-21 10:34:46 -08:00
palto42
e27ba0d08a less detail at v1 + option "--files" for details 2020-01-11 16:38:07 +01:00
Dan Helfman
d0c533555e In "borgmatic --help", don't expand $HOME in listing of default "--config" paths. 2020-01-02 10:37:31 -08:00
Dan Helfman
911668f0c8 Only trigger "on_error" hooks and monitoring failures for "prune", "create", and "check" actions, and not for other actions (#270). 2019-12-20 13:58:02 -08:00
Dan Helfman
d64bcd5e83 When pruning with verbosity level 1, list pruned and kept archives. 2019-12-17 20:12:41 -08:00
Dan Helfman
f787dfe809 Override particular configuration options from the command-line via "--override" flag (#268). 2019-12-17 11:46:27 -08:00