ansible/ansible_role_restic_archiver
1
0
Fork 0
mirror of https://github.com/roles-ansible/ansible_role_restic_archiver.git synced 2024-08-16 10:09:49 +02:00
Code Issues Releases Activity

make template usable

Browse source
This commit is contained in:
L3D 2020-12-09 17:26:16 +01:00 committed by Lilian Roller
parent 1c423fb742
commit 0163419f00
No known key found for this signature in database
GPG key ID: 5215C3441797AB2B
7 changed files with 150 additions and 294 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

188
README.md
View file

@ -12,191 +12,3 @@
--> -->
Description
----------------
[Restic](https://github.com/restic/restic) is a versatile Go based backup
solution which supports multiple backends, deduplication and incremental
backups.
This role is **only intended to clean up a restick backup**. So for example to keep only backups of the last 7 days and monthly backups. And if necessary to copy the backups again to an external storage.
----------------------
This role installs restic on a client, configures the backup repositories
and optionally sets up cronjobs to run said backups.
Aditionally, it will setup executable scripts to run a Backup manually.
> This Project borrows heavily from the
> [donat-b/ansible-restic](https://github.com/donat-b/ansible-restic)
> ansible role. We try to make this role more semver deployment friendly
> by allowing to use version tags and keep these snapshots and adapting the
> automated backup definition for use with windows systems.
### Backup Scripts
This role will create a backup script and a file with credentials usable with the `source` command on linux for each backup in the `restic_script_dir`.
These executable scripts can be used to manually trigger a backup action, but
are also used for automated backups if you have set `restic_create_cron` to true.
make sure to not change the files manually, as this can interfere with your
backups quite a bit.
on Linux, if you want to take a manual snapshot, you can run the backup like this:
```bash
$ /path/to/backup/script/backup-example.sh
```
by default, such a snapshot will be given the tag `manual`, so you can distinguish
them from automatically created snapshots. You can also append more tags by
simply appending them:
```bash
$ /path/to/backup/script/backup-example.sh --tag deployment
```
### CRON / Scheduled Tasks
In order to make use of defined backups, they can be automatically setup as
scheduled tasks. You have to be aware of the fact that (on linux systems at
least) you need to have administrator permissions for configuring such an action.
If you cannot use the automatic creation of the tasks, you can still make use
of the generated scripts. If you are for example on a shared hosting server
and can define a cronjob via a webinterface, simply add each backup file to
be executed. Make sure to prefix the command with `CRON=true` to imply that the
snapshot was created via a scheduled task:
```bash
CRON=true /path/to/backup/script/backup-example.sh
```
## Installation
```bash
ansible-galaxy install arillso.restic
```
## Requirements
* bzip2
## Role Variables
| Name | Default | Description |
| ---------------------- | -------------------- | --------------------------------------------------------------------------- |
| `restic_url` | `undefined` | The URL to download restic from. Use this variable to overwrite the default |
| `restic_version` | `'0.11.0'` | The version of Restic to install |
| `restic_download_path` | `'/opt/restic'` | Download location for the restic binary |
| `restic_install_path` | `'/usr/local/bin'` | Install location for the restic binary |
| `restic_script_dir` | `'~/restic'` | Location of the generated backup scripts |
| `restic_repos` | `{}` | A dictionary of repositories where snapshots are stored |
| `restic_archiver__backups` | `{}` (or `[]`) | A list of dictionaries specifying the files and directories to be backed up |
| `restic_create_cron` | `false` | Should a cronjob be created for each backup |
| `restic_dir_owner` | `'{{ansible_user}}'` | The owner of all created dirs |
| `restic_dir_group` | `'{{ansible_user}}'` | The group of all created dirs |
### Repos
Restic stores data in repositories. You have to specify at least one repository
to be able to use this role. A repository can be local or remote (see the
official [documentation](https://restic.readthedocs.io/en/stable/030_preparing_a_new_repo.html)).
> **Using an SFTP repository**
>
> For using an SFTP backend, the user needs passwordless access to the host.
> Please make sure to distribute ssh keys accordingly, as this is outside of
> the scope of this role.
Available variables:
| Name | Required | Description |
| ----------------------- |:--------:| ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `location` | yes | The location of the Backend. Currently, [Local](https://restic.readthedocs.io/en/stable/030_preparing_a_new_repo.html#local), [SFTP](https://restic.readthedocs.io/en/stable/030_preparing_a_new_repo.html#sftp), [S3](https://restic.readthedocs.io/en/stable/030_preparing_a_new_repo.html#amazon-s3) and [B2](https://restic.readthedocs.io/en/stable/030_preparing_a_new_repo.html#backblaze-b2) are supported |
| `password` | yes | The password used to secure this repository |
| `init` | no | Describes if the repository should be initialized or not. Use `false` if you are backuping to an already existing repo. |
| `aws_access_key` | no | The access key for the S3 backend |
| `aws_secret_access_key` | no | The secret access key for the S3 backend |
| `aws_default_region` | no | The desired region for the S3 backend |
| `b2_account_id` | no | The account ID for Backblaze B2 backend |
| `b2_account_key` | no | The account key for Backblaze B2 backend |
Example:
```yaml
restic_repos:
local:
location: /srv/restic-repo
password: securepassword1
init: true
remote:
location: sftp:user@host:/srv/restic-repo
password: securepassword2
init: true
```
### Backups
A backup specifies a directory or file to be backuped. A backup is written to a
Repository defined in `restic_repos`.
Available variables:
| Name | Required (Default) | Description |
| ------------------ |:-----------------------------:| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `name` | yes | The name of this backup. Used together with pruning and scheduling and needs to be unique. |
| `repo` | yes | The name of the repository to backup to. |
| `src` | yes | The source directory or file |
| `stdin` | no | Is this backup created from a [stdin](https://restic.readthedocs.io/en/stable/040_backup.html#reading-data-from-stdin)? |
| `stdin_cmd` | no (yes if `stdin` == `true`) | The command to produce the stdin. |
| `stdin_filename` | no | The filename used in the repository. |
| `tags` | no | Array of default tags |
| `keep_last` | no | If set, only keeps the last n snapshots. |
| `keep_hourly` | no | If set, only keeps the last n hourly snapshots. |
| `keep_daily` | no | If set, only keeps the last n daily snapshots. |
| `keep_weekly ` | no | If set, only keeps the last n weekly snapshots. |
| `keep_monthly` | no | If set, only keeps the last n monthly snapshots. |
| `keep_yearly ` | no | If set, only keeps the last n yearly snapshots. |
| `keep_within` | no | If set, only keeps snapshots in this time period. |
| `keep_tag` | no | If set, keep snapshots with this tags. Make sure to specify a list. |
| `prune` | no (`false`) | If `true`, the `restic forget` command in the script has the [`--prune` option](https://restic.readthedocs.io/en/stable/060_forget.html#removing-backup-snapshots) appended. |
| `scheduled` | no (`false`) | If `restic_create_cron` is set to `true`, this backup is scheduled. |
| `schedule_minute` | no (`*`) | Minute when the job is run. ( 0-59, *, */2, etc ) |
| `schedule_hour` | no (`*`) | Hour when the job is run. ( 0-23, *, */2, etc ) |
| `schedule_weekday` | no (`*`) | Weekday when the job is run. ( 0-6 for Sunday-Saturday, *, etc ) |
| `schedule_month` | no (`*`) | Month when the job is run. ( 1-12, *, */2, etc ) |
| `exclude` | no (`{}`) | Allows you to specify files to exclude. See [Exclude](#exclude) for reference. |
Example:
```yaml
restic_archiver__backups:
data:
name: data
repo: remove
src: /path/to/data
scheduled: true
schedule_hour: 3
```
> You can also specify restic_archiver__backups as an array, which is a legacy feature and
> might be deprecated in the future. currently, the name key is used for
> namint the access and backup files
#### Exclude
the `exclude` key on a backup allows you to specify multiple files to exclude or
files to look for filenames to be excluded. You can specify the following keys:
```yaml
exclude:
exclude_cache: true
exclude:
- /path/to/file
iexclude:
- /path/to/file
exclude_file:
- /path/to/file
exclude_if_present:
- /path/to/file
```
Please refer to the use of the specific keys to the
[documentation](https://restic.readthedocs.io/en/latest/040_backup.html#excluding-files).
## Dependencies
none
## Example Playbook
```yml
- hosts: all
roles:
- restic
```
## Author
- Matthias Leutenegger

18
defaults/main.yml
View file

@ -1,4 +1,20 @@
--- ---
# which repos should we cleanup by default
restic_archiver__repos: {}
# example_server:
# location: /srv/restic/example_server_repo
# password: securepassword4eXaMpleSserver
# how long should we store all backups by default
restic_archiver__keep: 9
restic_archiver__keep_hourly: 28
restic_archiver__keep_daily: 26
restic_archiver__keep_weekly: 8
restic_archiver__keep_monthly: 13
restic_archiver__keep_yearly: 12
# defaults file for skeleton # defaults file for skeleton
restic_install_path: '/usr/bin' restic_install_path: '/usr/bin'
restic_script_dir: '~/restic' restic_script_dir: '~/restic'
@ -10,4 +26,4 @@ restic_dir_owner: '{{ ansible_user | default(ansible_user_id) }}'
restic_dir_group: '{{ ansible_user | default(ansible_user_id) }}' restic_dir_group: '{{ ansible_user | default(ansible_user_id) }}'
# version check for this playbook # version check for this playbook
submodules_versioncheck: true submodules_versioncheck: false

1
meta/main.yml
View file

@ -3,7 +3,6 @@ galaxy_info:
role_name: restic role_name: restic
author: L3D author: L3D
description: Role to deploy restic and setup backups. description: Role to deploy restic and setup backups.
# company:
license: MIT license: MIT
min_ansible_version: 2.8 min_ansible_version: 2.8
platforms: platforms:

23
tasks/main.yml
View file

@ -2,7 +2,12 @@
- include_tasks: versioncheck.yml - include_tasks: versioncheck.yml
when: submodules_versioncheck|bool when: submodules_versioncheck|bool
- include_tasks: template.yml - name: use template
template:
src: templates/restic_forget_snapshots.j2
dest: /tmp/test
#- include_tasks: template.yml
#- name: add OS specific variables #- name: add OS specific variables
# include_vars: '{{ loop_vars }}' # include_vars: '{{ loop_vars }}'
@ -26,11 +31,11 @@
# - configuration # - configuration
# - packages # - packages
- name: Ensure restic directories exist #- name: Ensure restic directories exist
file: # file:
state: 'directory' # state: 'directory'
path: '{{ item }}' # path: '{{ item }}'
mode: '0755' # mode: '0755'
owner: '{{ restic_dir_owner }}' # owner: '{{ restic_dir_owner }}'
group: '{{ restic_dir_group }}' # group: '{{ restic_dir_group }}'
with_items: '{{ restic_create_paths }}' # with_items: '{{ restic_create_paths }}'

100
tasks/template.yml
View file

@ -1,54 +1,48 @@
--- ---
- name: reformat dict if necessary #- name: Create backup credentials
set_fact: # template:
restic_archiver__backups: "{{ restic_archiver__backups|dict2items|json_query('[*].value') }}" # src: restic_access_Linux.j2
when: # dest: '{{ restic_script_dir }}/access-{{ item.name }}.sh'
- restic_archiver__backups | type_debug == "dict" # mode: '0700'
# owner: '{{ restic_dir_owner }}'
- name: Create backup credentials # group: '{{ restic_dir_group }}'
template: # no_log: true
src: restic_access_Linux.j2 # with_items: '{{ restic_archiver__backups }}'
dest: '{{ restic_script_dir }}/access-{{ item.name }}.sh' # when:
mode: '0700' # - item.name is defined
owner: '{{ restic_dir_owner }}' # - item.src is defined or item.stdin is defined
group: '{{ restic_dir_group }}' # - item.src is defined or item.stdin and item.stdin_cmd is defined
no_log: true # - item.repo in restic_repos
with_items: '{{ restic_archiver__backups }}' #
when: #- name: Create backup script
- item.name is defined # template:
- item.src is defined or item.stdin is defined # src: restic_script_Linux.j2
- item.src is defined or item.stdin and item.stdin_cmd is defined # dest: '{{ restic_script_dir }}/backup-{{ item.name }}.sh'
- item.repo in restic_repos # mode: '0700'
# owner: '{{ restic_dir_owner }}'
- name: Create backup script # group: '{{ restic_dir_group }}'
template: # no_log: true
src: restic_script_Linux.j2 # with_items: '{{ restic_archiver__backups }}'
dest: '{{ restic_script_dir }}/backup-{{ item.name }}.sh' # when:
mode: '0700' # - item.name is defined
owner: '{{ restic_dir_owner }}' ## - item.src is defined or item.stdin is defined
group: '{{ restic_dir_group }}' # - item.src is defined or item.stdin and item.stdin_cmd is defined
no_log: true # - item.repo in restic_repos
with_items: '{{ restic_archiver__backups }}' #
when: #- name: Setup CRON jobs
- item.name is defined # cron:
- item.src is defined or item.stdin is defined # name: 'do1jlr.restic_archiver {{ item.name }}'
- item.src is defined or item.stdin and item.stdin_cmd is defined ## job: 'CRON=true {{ restic_script_dir }}/backup-{{ item.name }}.sh'
- item.repo in restic_repos # minute: '{{ item.schedule_minute | default("*") }}'
# hour: '{{ item.schedule_hour | default("*") }}'
- name: Setup CRON jobs # weekday: '{{ item.schedule_weekday | default("*") }}'
cron: # month: '{{ item.schedule_month | default("*") }}'
name: 'do1jlr.restic_archiver {{ item.name }}' # cron_file: '/etc/crontab'
job: 'CRON=true {{ restic_script_dir }}/backup-{{ item.name }}.sh' # state: present
minute: '{{ item.schedule_minute | default("*") }}' # become: true
hour: '{{ item.schedule_hour | default("*") }}' # no_log: true
weekday: '{{ item.schedule_weekday | default("*") }}' ## with_items: '{{ restic_archiver__backups }}'
month: '{{ item.schedule_month | default("*") }}' # when:
cron_file: '/etc/crontab' # - restic_create_cron
state: present # - item.name is defined
become: true # - item.scheduled | default(false)
no_log: true
with_items: '{{ restic_archiver__backups }}'
when:
- restic_create_cron
- item.name is defined
- item.scheduled | default(false)

101
templates/restic_script_Linux.j2 → templates/restic_forget_snapshots.j2
View file

@ -1,55 +1,94 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# {{ ansible_managed }} # {{ ansible_managed }}
# Backup script for {{ item.src|default('stdin') }}
# This file is to cleanup your backup archive and move some snapshots to a external storage. # This file is to cleanup your backup archive and move some snapshots to a external storage.
set -euxo pipefail
{% for repo in restic_archiver__repos %}
{#
define macro:
how fast should we delete snapshots?
Variables are defined via defaults!
#}
{% macro retention_pattern(repo) -%}
{% if repo.keep_last is defined and repo.keep_last != None -%}
--keep-last {{ repo.keep_last }}
{%- else -%}
--keep-last {{ restic_archiver__keep }}
{%- endif %} \
{% if repo.keep_hourly is defined and repo.keep_hourly != None -%}
--keep-hourly {{ repo.keep_hourly }}
{%- else -%}
--keep-hourly {{ restic_archiver__keep_hourly }}
{%- endif %} \
{% if repo.keep_daily is defined and repo.keep_daily != None -%}
--keep-daily {{ repo.keep_daily }}
{%- else -%}
--keep-daily {{ restic_archiver__keep_daily }}
{%- endif %} \
{% if repo.keep_weekly is defined and repo.keep_weekly != None -%}
--keep-weekly {{ repo.keep_weekly }}
{%- else -%}
--keep-weekly {{ restic_archiver__keep_weekly }}
{%- endif %} \
{% if repo.keep_monthly is defined and repo.keep_monthly != None -%}
--keep-monthly {{ repo.keep_monthly }}
{%- else -%}
--keep-monthly {{ restic_archiver__keep_monthly }}
{%- endif %} \
{% if repo.keep_yearly is defined and repo.keep_yearly != None -%}
--keep-yearly {{ repo.keep_yearly }}
{%- else -%}
--keep-yearly {{ restic_archiver__keep_yearly }}
{%- endif -%}
{% if repo.keep_within is defined and repo.keep_within != None %} \
--keep-within {{ repo.keep_within }} {% endif -%}
{%- endmacro %}
export RESTIC_REPOSITORY={{ restic_repos[item.repo].location }} # Settings for Server {{ repo['name'] | string }}
export RESTIC_PASSWORD='{{ restic_repos[item.repo].password | regex_replace('\'', '\'\\\'\'') }}' export RESTIC_REPOSITORY="{{ repo['location'] }}"
BACKUP_NAME={{ item.name }} export RESTIC_PASSWORD='''{{ repo['password'] | regex_replace('\'', '\'\\\'\'') }}'''
BACKUP_NAME="{{ repo.name }}"
{{ restic_install_path }}/restic forget {{ retention_pattern(repo) }} {% if repo.prune is defined and repo.prune == true %}--prune{% endif %}
{% endfor %}
{#
export RESTIC_REPOSITORY="{{ restic_repos[item.repo].location }}"
export RESTIC_PASSWORD="{{ restic_repos[item.repo].password | regex_replace('\'', '\'\\\'\'') }}"
BACKUP_NAME="{{ item.name }}"
{% if item.src is defined -%} {% if item.src is defined -%}
# BACKUP_SOURCE={{ item.src }} # BACKUP_SOURCE={{ item.src }}
{%- endif %} {%- endif %}
set -euxo pipefail
{# {xxx
Define Tags
#}
{% macro tags(tags) -%}
{% if tags is defined and (tags|length>0) %}{% for tag in tags %} --tag {{ tag }}{% endfor %}{% endif %}
{%- endmacro %}
{#
Define Keeped Tags
#}
{% macro keep_tags(tags) -%}
{% if tags is defined and (tags|length>0) %}{% for tag in tags %} --keep-tag {{ tag }}{% endfor %}{% endif %}
{%- endmacro %}
{#
Define Hostname Define Hostname
#} xxx}
{% macro hostname(h) -%} {% macro hostname(h) -%}
{% if h is defined %} --hostname {{ h }}{% endif %} {% if h is defined %} --hostname {{ h }}{% endif %}
{%- endmacro %} {%- endmacro %}
{# {xxx
Define stdin filename Define stdin filename
#} xxx}
{% macro stdin_filename(n) -%} {% macro stdin_filename(n) -%}
{% if n is defined %} --stdin-filename {{ n }}{% endif %} {% if n is defined %} --stdin-filename {{ n }}{% endif %}
{%- endmacro %} {%- endmacro %}
{# {xxx
Define path Define path
#} xxx}
{% macro path(repo) -%} {% macro path(repo) -%}
{% if repo.src is defined and repo.src != None and (repo.src|length>0) %}{{ repo.src }}{% else %}{{ repo.stdin_filename }}{% endif %} {% if repo.src is defined and repo.src != None and (repo.src|length>0) %}{{ repo.src }}{% else %}{{ repo.stdin_filename }}{% endif %}
{%- endmacro %} {%- endmacro %}
{# {xxx
Define retention pattern Define retention pattern
#} xxx}
{% macro retention_pattern(repo) -%} {% macro retention_pattern(repo) -%}
{% if repo.keep_last is defined and repo.keep_last != None %}--keep-last {{ item.keep_last }}{% endif %} \ {% if repo.keep_last is defined and repo.keep_last != None %}--keep-last {{ item.keep_last }}{% endif %} \
{% if repo.keep_hourly is defined and repo.keep_hourly != None %}--keep-hourly {{ item.keep_hourly }}{% endif %} \ {% if repo.keep_hourly is defined and repo.keep_hourly != None %}--keep-hourly {{ item.keep_hourly }}{% endif %} \
@ -86,7 +125,11 @@ set -euxo pipefail
{% endfor %} {% endfor %}
{% endif %} {% endif %}
{% endmacro %} {% endmacro %}
{#
{xxx
Define backup commands Define backup commands
#} xxx}
{{ restic_install_path }}/restic forget {{ retention_pattern(item) }} {% if item.prune is defined and item.prune == true %}--prune{% endif %} {{ restic_install_path }}/restic forget {{ retention_pattern(item) }} {% if item.prune is defined and item.prune == true %}--prune{% endif %}
#}

13
vars/main.yml
View file

@ -1,16 +1,3 @@
--- ---
# we use a platform map to decipher system version
_platform_map:
i386: 386
x86_64: amd64
aarch64: arm64
armv7l: arm
armv6l: arm
restic_create_paths:
- '{{ restic_download_path }}/bin'
- '{{ restic_script_dir }}'
---
playbook_version_number: 20 # should be over ninethousand playbook_version_number: 20 # should be over ninethousand
playbook_version_path: 'role-restic_archiver_roles-ansible_github.com.version' playbook_version_path: 'role-restic_archiver_roles-ansible_github.com.version'