2019-10-24 00:35:37 +02:00
|
|
|
---
|
|
|
|
title: How to backup your databases
|
2020-08-21 23:27:47 +02:00
|
|
|
eleventyNavigation:
|
|
|
|
key: Backup your databases
|
|
|
|
parent: How-to guides
|
|
|
|
order: 7
|
2019-10-24 00:35:37 +02:00
|
|
|
---
|
|
|
|
## Database dump hooks
|
|
|
|
|
|
|
|
If you want to backup a database, it's best practice with most database
|
|
|
|
systems to backup an exported database dump, rather than backing up your
|
|
|
|
database's internal file storage. That's because the internal storage can
|
|
|
|
change while you're reading from it. In contrast, a database dump creates a
|
|
|
|
consistent snapshot that is more suited for backups.
|
|
|
|
|
|
|
|
Fortunately, borgmatic includes built-in support for creating database dumps
|
|
|
|
prior to running backups. For example, here is everything you need to dump and
|
2019-11-12 20:59:01 +01:00
|
|
|
backup a couple of local PostgreSQL databases and a MySQL/MariaDB database:
|
2019-10-24 00:35:37 +02:00
|
|
|
|
|
|
|
```yaml
|
|
|
|
hooks:
|
|
|
|
postgresql_databases:
|
|
|
|
- name: users
|
|
|
|
- name: orders
|
2019-11-12 20:59:01 +01:00
|
|
|
mysql_databases:
|
|
|
|
- name: posts
|
2019-10-24 00:35:37 +02:00
|
|
|
```
|
|
|
|
|
2020-05-07 20:44:04 +02:00
|
|
|
As part of each backup, borgmatic streams a database dump for each configured
|
|
|
|
database directly to Borg, so it's included in the backup without consuming
|
2020-05-18 20:31:29 +02:00
|
|
|
additional disk space. (The one exception is PostgreSQL's "directory" dump
|
|
|
|
format, which can't stream and therefore does consume temporary disk space.)
|
2019-12-11 01:04:34 +01:00
|
|
|
|
2020-05-07 20:44:04 +02:00
|
|
|
To support this, borgmatic creates temporary named pipes in `~/.borgmatic` by
|
|
|
|
default. To customize this path, set the `borgmatic_source_directory` option
|
|
|
|
in the `location` section of borgmatic's configuration.
|
2019-10-24 00:35:37 +02:00
|
|
|
|
2020-05-26 18:20:14 +02:00
|
|
|
Also note that using a database hook implicitly enables both the
|
|
|
|
`read_special` and `one_file_system` configuration settings (even if they're
|
|
|
|
disabled in your configuration) to support this dump and restore streaming.
|
2020-07-13 19:45:58 +02:00
|
|
|
See Limitations below for more on this.
|
2020-05-26 18:20:14 +02:00
|
|
|
|
2019-11-12 20:59:01 +01:00
|
|
|
Here's a more involved example that connects to remote databases:
|
2019-10-24 00:35:37 +02:00
|
|
|
|
|
|
|
```yaml
|
|
|
|
hooks:
|
|
|
|
postgresql_databases:
|
|
|
|
- name: users
|
2019-11-12 20:59:01 +01:00
|
|
|
hostname: database1.example.org
|
2019-10-24 00:35:37 +02:00
|
|
|
port: 5433
|
2019-11-12 20:59:01 +01:00
|
|
|
username: postgres
|
2019-10-24 00:35:37 +02:00
|
|
|
password: trustsome1
|
|
|
|
format: tar
|
|
|
|
options: "--role=someone"
|
2019-11-12 20:59:01 +01:00
|
|
|
mysql_databases:
|
|
|
|
- name: posts
|
|
|
|
hostname: database2.example.org
|
|
|
|
port: 3307
|
|
|
|
username: root
|
|
|
|
password: trustsome1
|
|
|
|
options: "--skip-comments"
|
2019-10-24 00:35:37 +02:00
|
|
|
```
|
|
|
|
|
|
|
|
If you want to dump all databases on a host, use `all` for the database name:
|
|
|
|
|
|
|
|
```yaml
|
|
|
|
hooks:
|
|
|
|
postgresql_databases:
|
|
|
|
- name: all
|
2019-11-12 20:59:01 +01:00
|
|
|
mysql_databases:
|
|
|
|
- name: all
|
2019-10-24 00:35:37 +02:00
|
|
|
```
|
|
|
|
|
|
|
|
Note that you may need to use a `username` of the `postgres` superuser for
|
2019-11-12 20:59:01 +01:00
|
|
|
this to work with PostgreSQL.
|
2019-10-24 00:35:37 +02:00
|
|
|
|
2020-09-21 19:00:38 +02:00
|
|
|
If you would like to backup databases only and not source directories, you can
|
|
|
|
specify an empty `source_directories` value because it is a mandatory field:
|
2020-09-21 17:35:26 +02:00
|
|
|
|
|
|
|
```yaml
|
|
|
|
location:
|
|
|
|
source_directories: []
|
|
|
|
hooks:
|
|
|
|
mysql_databases:
|
|
|
|
- name: all
|
|
|
|
```
|
|
|
|
|
2019-11-01 05:45:47 +01:00
|
|
|
|
|
|
|
### Configuration backups
|
|
|
|
|
|
|
|
An important note about this database configuration: You'll need the
|
|
|
|
configuration to be present in order for borgmatic to restore a database. So
|
|
|
|
to prepare for this situation, it's a good idea to include borgmatic's own
|
|
|
|
configuration files as part of your regular backups. That way, you can always
|
|
|
|
bring back any missing configuration files in order to restore a database.
|
|
|
|
|
|
|
|
|
2019-10-24 00:35:37 +02:00
|
|
|
## Supported databases
|
|
|
|
|
2019-11-12 20:59:01 +01:00
|
|
|
As of now, borgmatic supports PostgreSQL and MySQL/MariaDB databases
|
|
|
|
directly. But see below about general-purpose preparation and cleanup hooks as
|
|
|
|
a work-around with other database systems. Also, please [file a
|
2019-10-24 00:35:37 +02:00
|
|
|
ticket](https://torsion.org/borgmatic/#issues) for additional database systems
|
|
|
|
that you'd like supported.
|
|
|
|
|
2019-11-01 05:45:47 +01:00
|
|
|
|
2019-10-24 00:35:37 +02:00
|
|
|
## Database restoration
|
|
|
|
|
2019-11-01 05:45:47 +01:00
|
|
|
To restore a database dump from an archive, use the `borgmatic restore`
|
|
|
|
action. But the first step is to figure out which archive to restore from. A
|
|
|
|
good way to do that is to use the `list` action:
|
|
|
|
|
|
|
|
```bash
|
|
|
|
borgmatic list
|
|
|
|
```
|
|
|
|
|
|
|
|
(No borgmatic `list` action? Try the old-style `--list`, or upgrade
|
|
|
|
borgmatic!)
|
|
|
|
|
|
|
|
That should yield output looking something like:
|
|
|
|
|
|
|
|
```text
|
|
|
|
host-2019-01-01T04:05:06.070809 Tue, 2019-01-01 04:05:06 [...]
|
|
|
|
host-2019-01-02T04:06:07.080910 Wed, 2019-01-02 04:06:07 [...]
|
|
|
|
```
|
|
|
|
|
|
|
|
Assuming that you want to restore all database dumps from the archive with the
|
|
|
|
most up-to-date files and therefore the latest timestamp, run a command like:
|
|
|
|
|
|
|
|
```bash
|
|
|
|
borgmatic restore --archive host-2019-01-02T04:06:07.080910
|
|
|
|
```
|
|
|
|
|
|
|
|
(No borgmatic `restore` action? Upgrade borgmatic!)
|
|
|
|
|
2020-01-30 02:08:03 +01:00
|
|
|
With newer versions of borgmatic, you can simplify this to:
|
|
|
|
|
|
|
|
```bash
|
|
|
|
borgmatic restore --archive latest
|
|
|
|
```
|
|
|
|
|
2019-11-01 05:45:47 +01:00
|
|
|
The `--archive` value is the name of the archive to restore from. This
|
|
|
|
restores all databases dumps that borgmatic originally backed up to that
|
|
|
|
archive.
|
|
|
|
|
|
|
|
This is a destructive action! `borgmatic restore` replaces live databases by
|
|
|
|
restoring dumps from the selected archive. So be very careful when and where
|
|
|
|
you run it.
|
|
|
|
|
|
|
|
|
|
|
|
### Repository selection
|
|
|
|
|
|
|
|
If you have a single repository in your borgmatic configuration file(s), no
|
|
|
|
problem: the `restore` action figures out which repository to use.
|
|
|
|
|
|
|
|
But if you have multiple repositories configured, then you'll need to specify
|
|
|
|
the repository path containing the archive to restore. Here's an example:
|
|
|
|
|
|
|
|
```bash
|
|
|
|
borgmatic restore --repository repo.borg --archive host-2019-...
|
|
|
|
```
|
|
|
|
|
|
|
|
### Restore particular databases
|
|
|
|
|
|
|
|
If you've backed up multiple databases into an archive, and you'd only like to
|
|
|
|
restore one of them, use the `--database` flag to select one or more
|
|
|
|
databases. For instance:
|
|
|
|
|
|
|
|
```bash
|
|
|
|
borgmatic restore --archive host-2019-... --database users
|
|
|
|
```
|
|
|
|
|
|
|
|
### Limitations
|
|
|
|
|
|
|
|
There are a few important limitations with borgmatic's current database
|
|
|
|
restoration feature that you should know about:
|
|
|
|
|
|
|
|
1. You must restore as the same Unix user that created the archive containing
|
|
|
|
the database dump. That's because the user's home directory path is encoded
|
|
|
|
into the path of the database dump within the archive.
|
|
|
|
2. As mentioned above, borgmatic can only restore a database that's defined in
|
|
|
|
borgmatic's own configuration file. So include your configuration file in
|
|
|
|
backups to avoid getting caught without a way to restore a database.
|
|
|
|
3. borgmatic does not currently support backing up or restoring multiple
|
|
|
|
databases that share the exact same name on different hosts.
|
2020-07-13 19:45:58 +02:00
|
|
|
4. Because database hooks implicitly enable the `read_special` configuration
|
|
|
|
setting to support dump and restore streaming, you'll need to ensure that any
|
|
|
|
special files are excluded from backups (named pipes, block devices, and
|
2020-07-13 19:56:59 +02:00
|
|
|
character devices) to prevent hanging. Try a command like `find / -type c,b,p`
|
|
|
|
to find such files. Common directories to exclude are `/dev` and `/run`, but
|
2020-07-13 19:45:58 +02:00
|
|
|
that may not be exhaustive.
|
2019-11-01 05:45:47 +01:00
|
|
|
|
|
|
|
|
|
|
|
### Manual restoration
|
|
|
|
|
|
|
|
If you prefer to restore a database without the help of borgmatic, first
|
|
|
|
[extract](https://torsion.org/borgmatic/docs/how-to/extract-a-backup/) an
|
|
|
|
archive containing a database dump, and then manually restore the dump file
|
2019-11-12 20:59:01 +01:00
|
|
|
found within the extracted `~/.borgmatic/` path (e.g. with `pg_restore` or
|
|
|
|
`mysql` commands).
|
2019-11-01 05:45:47 +01:00
|
|
|
|
2019-10-24 00:35:37 +02:00
|
|
|
|
|
|
|
## Preparation and cleanup hooks
|
|
|
|
|
|
|
|
If this database integration is too limited for needs, borgmatic also supports
|
|
|
|
general-purpose [preparation and cleanup
|
2019-10-29 17:53:45 +01:00
|
|
|
hooks](https://torsion.org/borgmatic/docs/how-to/add-preparation-and-cleanup-steps-to-backups/).
|
|
|
|
These hooks allows you to trigger arbitrary commands or scripts before and
|
|
|
|
after backups. So if necessary, you can use these hooks to create database
|
|
|
|
dumps with any database system.
|
2019-10-24 00:35:37 +02:00
|
|
|
|
2019-11-01 05:45:47 +01:00
|
|
|
|
2019-11-14 19:33:47 +01:00
|
|
|
## Troubleshooting
|
|
|
|
|
|
|
|
### MySQL table lock errors
|
|
|
|
|
|
|
|
If you encounter table lock errors during a database dump with MySQL/MariaDB,
|
|
|
|
you may need to [use a
|
|
|
|
transaction](https://dev.mysql.com/doc/refman/8.0/en/mysqldump.html#option_mysqldump_single-transaction).
|
|
|
|
You can add any additional flags to the `options:` in your database
|
|
|
|
configuration. Here's an example:
|
|
|
|
|
|
|
|
```yaml
|
|
|
|
hooks:
|
|
|
|
mysql_databases:
|
|
|
|
- name: posts
|
|
|
|
options: "--single-transaction --quick"
|
|
|
|
```
|
|
|
|
|
2020-07-13 19:45:58 +02:00
|
|
|
### borgmatic hangs during backup
|
|
|
|
|
|
|
|
See Limitations above about `read_special`. You may need to exclude certain
|
2020-07-13 19:56:59 +02:00
|
|
|
paths with named pipes, block devices, or character devices on which borgmatic
|
|
|
|
is hanging.
|