diff --git a/README.md b/README.md index 8b0033e..7d79832 100644 --- a/README.md +++ b/README.md @@ -12,191 +12,3 @@ --> - - Description ----------------- -[Restic](https://github.com/restic/restic) is a versatile Go based backup -solution which supports multiple backends, deduplication and incremental -backups. - -This role is **only intended to clean up a restick backup**. So for example to keep only backups of the last 7 days and monthly backups. And if necessary to copy the backups again to an external storage. - ----------------------- - -This role installs restic on a client, configures the backup repositories -and optionally sets up cronjobs to run said backups. -Aditionally, it will setup executable scripts to run a Backup manually. - -> This Project borrows heavily from the -> [donat-b/ansible-restic](https://github.com/donat-b/ansible-restic) -> ansible role. We try to make this role more semver deployment friendly -> by allowing to use version tags and keep these snapshots and adapting the -> automated backup definition for use with windows systems. - -### Backup Scripts -This role will create a backup script and a file with credentials usable with the `source` command on linux for each backup in the `restic_script_dir`. -These executable scripts can be used to manually trigger a backup action, but -are also used for automated backups if you have set `restic_create_cron` to true. -make sure to not change the files manually, as this can interfere with your -backups quite a bit. - -on Linux, if you want to take a manual snapshot, you can run the backup like this: -```bash -$ /path/to/backup/script/backup-example.sh -``` -by default, such a snapshot will be given the tag `manual`, so you can distinguish -them from automatically created snapshots. You can also append more tags by -simply appending them: -```bash -$ /path/to/backup/script/backup-example.sh --tag deployment -``` - -### CRON / Scheduled Tasks -In order to make use of defined backups, they can be automatically setup as -scheduled tasks. You have to be aware of the fact that (on linux systems at -least) you need to have administrator permissions for configuring such an action. - -If you cannot use the automatic creation of the tasks, you can still make use -of the generated scripts. If you are for example on a shared hosting server -and can define a cronjob via a webinterface, simply add each backup file to -be executed. Make sure to prefix the command with `CRON=true` to imply that the -snapshot was created via a scheduled task: -```bash -CRON=true /path/to/backup/script/backup-example.sh -``` -## Installation - -```bash -ansible-galaxy install arillso.restic -``` -## Requirements -* bzip2 -## Role Variables - -| Name | Default | Description | -| ---------------------- | -------------------- | --------------------------------------------------------------------------- | -| `restic_url` | `undefined` | The URL to download restic from. Use this variable to overwrite the default | -| `restic_version` | `'0.11.0'` | The version of Restic to install | -| `restic_download_path` | `'/opt/restic'` | Download location for the restic binary | -| `restic_install_path` | `'/usr/local/bin'` | Install location for the restic binary | -| `restic_script_dir` | `'~/restic'` | Location of the generated backup scripts | -| `restic_repos` | `{}` | A dictionary of repositories where snapshots are stored | -| `restic_archiver__backups` | `{}` (or `[]`) | A list of dictionaries specifying the files and directories to be backed up | -| `restic_create_cron` | `false` | Should a cronjob be created for each backup | -| `restic_dir_owner` | `'{{ansible_user}}'` | The owner of all created dirs | -| `restic_dir_group` | `'{{ansible_user}}'` | The group of all created dirs | - -### Repos -Restic stores data in repositories. You have to specify at least one repository -to be able to use this role. A repository can be local or remote (see the -official [documentation](https://restic.readthedocs.io/en/stable/030_preparing_a_new_repo.html)). - -> **Using an SFTP repository** -> -> For using an SFTP backend, the user needs passwordless access to the host. -> Please make sure to distribute ssh keys accordingly, as this is outside of -> the scope of this role. - -Available variables: - -| Name | Required | Description | -| ----------------------- |:--------:| ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `location` | yes | The location of the Backend. Currently, [Local](https://restic.readthedocs.io/en/stable/030_preparing_a_new_repo.html#local), [SFTP](https://restic.readthedocs.io/en/stable/030_preparing_a_new_repo.html#sftp), [S3](https://restic.readthedocs.io/en/stable/030_preparing_a_new_repo.html#amazon-s3) and [B2](https://restic.readthedocs.io/en/stable/030_preparing_a_new_repo.html#backblaze-b2) are supported | -| `password` | yes | The password used to secure this repository | -| `init` | no | Describes if the repository should be initialized or not. Use `false` if you are backuping to an already existing repo. | -| `aws_access_key` | no | The access key for the S3 backend | -| `aws_secret_access_key` | no | The secret access key for the S3 backend | -| `aws_default_region` | no | The desired region for the S3 backend | -| `b2_account_id` | no | The account ID for Backblaze B2 backend | -| `b2_account_key` | no | The account key for Backblaze B2 backend | - -Example: -```yaml -restic_repos: - local: - location: /srv/restic-repo - password: securepassword1 - init: true - remote: - location: sftp:user@host:/srv/restic-repo - password: securepassword2 - init: true -``` - -### Backups -A backup specifies a directory or file to be backuped. A backup is written to a -Repository defined in `restic_repos`. - -Available variables: - -| Name | Required (Default) | Description | -| ------------------ |:-----------------------------:| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `name` | yes | The name of this backup. Used together with pruning and scheduling and needs to be unique. | -| `repo` | yes | The name of the repository to backup to. | -| `src` | yes | The source directory or file | -| `stdin` | no | Is this backup created from a [stdin](https://restic.readthedocs.io/en/stable/040_backup.html#reading-data-from-stdin)? | -| `stdin_cmd` | no (yes if `stdin` == `true`) | The command to produce the stdin. | -| `stdin_filename` | no | The filename used in the repository. | -| `tags` | no | Array of default tags | -| `keep_last` | no | If set, only keeps the last n snapshots. | -| `keep_hourly` | no | If set, only keeps the last n hourly snapshots. | -| `keep_daily` | no | If set, only keeps the last n daily snapshots. | -| `keep_weekly ` | no | If set, only keeps the last n weekly snapshots. | -| `keep_monthly` | no | If set, only keeps the last n monthly snapshots. | -| `keep_yearly ` | no | If set, only keeps the last n yearly snapshots. | -| `keep_within` | no | If set, only keeps snapshots in this time period. | -| `keep_tag` | no | If set, keep snapshots with this tags. Make sure to specify a list. | -| `prune` | no (`false`) | If `true`, the `restic forget` command in the script has the [`--prune` option](https://restic.readthedocs.io/en/stable/060_forget.html#removing-backup-snapshots) appended. | -| `scheduled` | no (`false`) | If `restic_create_cron` is set to `true`, this backup is scheduled. | -| `schedule_minute` | no (`*`) | Minute when the job is run. ( 0-59, *, */2, etc ) | -| `schedule_hour` | no (`*`) | Hour when the job is run. ( 0-23, *, */2, etc ) | -| `schedule_weekday` | no (`*`) | Weekday when the job is run. ( 0-6 for Sunday-Saturday, *, etc ) | -| `schedule_month` | no (`*`) | Month when the job is run. ( 1-12, *, */2, etc ) | -| `exclude` | no (`{}`) | Allows you to specify files to exclude. See [Exclude](#exclude) for reference. | - -Example: -```yaml -restic_archiver__backups: - data: - name: data - repo: remove - src: /path/to/data - scheduled: true - schedule_hour: 3 -``` - -> You can also specify restic_archiver__backups as an array, which is a legacy feature and -> might be deprecated in the future. currently, the name key is used for -> namint the access and backup files - -#### Exclude -the `exclude` key on a backup allows you to specify multiple files to exclude or -files to look for filenames to be excluded. You can specify the following keys: -```yaml -exclude: - exclude_cache: true - exclude: - - /path/to/file - iexclude: - - /path/to/file - exclude_file: - - /path/to/file - exclude_if_present: - - /path/to/file -``` -Please refer to the use of the specific keys to the -[documentation](https://restic.readthedocs.io/en/latest/040_backup.html#excluding-files). - -## Dependencies -none -## Example Playbook - -```yml -- hosts: all - roles: - - restic -``` - -## Author - -- Matthias Leutenegger diff --git a/defaults/main.yml b/defaults/main.yml index 118290f..0949cf4 100644 --- a/defaults/main.yml +++ b/defaults/main.yml @@ -1,4 +1,20 @@ --- +# which repos should we cleanup by default +restic_archiver__repos: {} +# example_server: +# location: /srv/restic/example_server_repo +# password: securepassword4eXaMpleSserver + +# how long should we store all backups by default +restic_archiver__keep: 9 +restic_archiver__keep_hourly: 28 +restic_archiver__keep_daily: 26 +restic_archiver__keep_weekly: 8 +restic_archiver__keep_monthly: 13 +restic_archiver__keep_yearly: 12 + + + # defaults file for skeleton restic_install_path: '/usr/bin' restic_script_dir: '~/restic' @@ -10,4 +26,4 @@ restic_dir_owner: '{{ ansible_user | default(ansible_user_id) }}' restic_dir_group: '{{ ansible_user | default(ansible_user_id) }}' # version check for this playbook -submodules_versioncheck: true +submodules_versioncheck: false diff --git a/meta/main.yml b/meta/main.yml index 1b45739..95d7d59 100644 --- a/meta/main.yml +++ b/meta/main.yml @@ -3,7 +3,6 @@ galaxy_info: role_name: restic author: L3D description: Role to deploy restic and setup backups. - # company: license: MIT min_ansible_version: 2.8 platforms: diff --git a/tasks/main.yml b/tasks/main.yml index 61f1f96..e2a705e 100644 --- a/tasks/main.yml +++ b/tasks/main.yml @@ -2,7 +2,12 @@ - include_tasks: versioncheck.yml when: submodules_versioncheck|bool -- include_tasks: template.yml +- name: use template + template: + src: templates/restic_forget_snapshots.j2 + dest: /tmp/test + +#- include_tasks: template.yml #- name: add OS specific variables # include_vars: '{{ loop_vars }}' @@ -26,11 +31,11 @@ # - configuration # - packages -- name: Ensure restic directories exist - file: - state: 'directory' - path: '{{ item }}' - mode: '0755' - owner: '{{ restic_dir_owner }}' - group: '{{ restic_dir_group }}' - with_items: '{{ restic_create_paths }}' +#- name: Ensure restic directories exist +# file: +# state: 'directory' +# path: '{{ item }}' +# mode: '0755' +# owner: '{{ restic_dir_owner }}' +# group: '{{ restic_dir_group }}' +# with_items: '{{ restic_create_paths }}' diff --git a/tasks/template.yml b/tasks/template.yml index 9dd78d1..3e4d706 100644 --- a/tasks/template.yml +++ b/tasks/template.yml @@ -1,54 +1,48 @@ --- -- name: reformat dict if necessary - set_fact: - restic_archiver__backups: "{{ restic_archiver__backups|dict2items|json_query('[*].value') }}" - when: - - restic_archiver__backups | type_debug == "dict" - -- name: Create backup credentials - template: - src: restic_access_Linux.j2 - dest: '{{ restic_script_dir }}/access-{{ item.name }}.sh' - mode: '0700' - owner: '{{ restic_dir_owner }}' - group: '{{ restic_dir_group }}' - no_log: true - with_items: '{{ restic_archiver__backups }}' - when: - - item.name is defined - - item.src is defined or item.stdin is defined - - item.src is defined or item.stdin and item.stdin_cmd is defined - - item.repo in restic_repos - -- name: Create backup script - template: - src: restic_script_Linux.j2 - dest: '{{ restic_script_dir }}/backup-{{ item.name }}.sh' - mode: '0700' - owner: '{{ restic_dir_owner }}' - group: '{{ restic_dir_group }}' - no_log: true - with_items: '{{ restic_archiver__backups }}' - when: - - item.name is defined - - item.src is defined or item.stdin is defined - - item.src is defined or item.stdin and item.stdin_cmd is defined - - item.repo in restic_repos - -- name: Setup CRON jobs - cron: - name: 'do1jlr.restic_archiver {{ item.name }}' - job: 'CRON=true {{ restic_script_dir }}/backup-{{ item.name }}.sh' - minute: '{{ item.schedule_minute | default("*") }}' - hour: '{{ item.schedule_hour | default("*") }}' - weekday: '{{ item.schedule_weekday | default("*") }}' - month: '{{ item.schedule_month | default("*") }}' - cron_file: '/etc/crontab' - state: present - become: true - no_log: true - with_items: '{{ restic_archiver__backups }}' - when: - - restic_create_cron - - item.name is defined - - item.scheduled | default(false) +#- name: Create backup credentials +# template: +# src: restic_access_Linux.j2 +# dest: '{{ restic_script_dir }}/access-{{ item.name }}.sh' +# mode: '0700' +# owner: '{{ restic_dir_owner }}' +# group: '{{ restic_dir_group }}' +# no_log: true +# with_items: '{{ restic_archiver__backups }}' +# when: +# - item.name is defined +# - item.src is defined or item.stdin is defined +# - item.src is defined or item.stdin and item.stdin_cmd is defined +# - item.repo in restic_repos +# +#- name: Create backup script +# template: +# src: restic_script_Linux.j2 +# dest: '{{ restic_script_dir }}/backup-{{ item.name }}.sh' +# mode: '0700' +# owner: '{{ restic_dir_owner }}' +# group: '{{ restic_dir_group }}' +# no_log: true +# with_items: '{{ restic_archiver__backups }}' +# when: +# - item.name is defined +## - item.src is defined or item.stdin is defined + # - item.src is defined or item.stdin and item.stdin_cmd is defined + # - item.repo in restic_repos +# +#- name: Setup CRON jobs +# cron: +# name: 'do1jlr.restic_archiver {{ item.name }}' + ## job: 'CRON=true {{ restic_script_dir }}/backup-{{ item.name }}.sh' + # minute: '{{ item.schedule_minute | default("*") }}' + # hour: '{{ item.schedule_hour | default("*") }}' + # weekday: '{{ item.schedule_weekday | default("*") }}' +# month: '{{ item.schedule_month | default("*") }}' +# cron_file: '/etc/crontab' +# state: present +# become: true +# no_log: true +## with_items: '{{ restic_archiver__backups }}' + # when: + # - restic_create_cron + # - item.name is defined + # - item.scheduled | default(false) diff --git a/templates/restic_script_Linux.j2 b/templates/restic_forget_snapshots.j2 similarity index 56% rename from templates/restic_script_Linux.j2 rename to templates/restic_forget_snapshots.j2 index e890522..1e647a9 100644 --- a/templates/restic_script_Linux.j2 +++ b/templates/restic_forget_snapshots.j2 @@ -1,55 +1,94 @@ #!/usr/bin/env bash # {{ ansible_managed }} -# Backup script for {{ item.src|default('stdin') }} # This file is to cleanup your backup archive and move some snapshots to a external storage. +set -euxo pipefail + +{% for repo in restic_archiver__repos %} + +{# + define macro: + how fast should we delete snapshots? + + Variables are defined via defaults! +#} +{% macro retention_pattern(repo) -%} + {% if repo.keep_last is defined and repo.keep_last != None -%} + --keep-last {{ repo.keep_last }} + {%- else -%} + --keep-last {{ restic_archiver__keep }} + {%- endif %} \ + {% if repo.keep_hourly is defined and repo.keep_hourly != None -%} + --keep-hourly {{ repo.keep_hourly }} + {%- else -%} + --keep-hourly {{ restic_archiver__keep_hourly }} + {%- endif %} \ + {% if repo.keep_daily is defined and repo.keep_daily != None -%} + --keep-daily {{ repo.keep_daily }} + {%- else -%} + --keep-daily {{ restic_archiver__keep_daily }} + {%- endif %} \ + {% if repo.keep_weekly is defined and repo.keep_weekly != None -%} + --keep-weekly {{ repo.keep_weekly }} + {%- else -%} + --keep-weekly {{ restic_archiver__keep_weekly }} + {%- endif %} \ + {% if repo.keep_monthly is defined and repo.keep_monthly != None -%} + --keep-monthly {{ repo.keep_monthly }} + {%- else -%} + --keep-monthly {{ restic_archiver__keep_monthly }} + {%- endif %} \ + {% if repo.keep_yearly is defined and repo.keep_yearly != None -%} + --keep-yearly {{ repo.keep_yearly }} + {%- else -%} + --keep-yearly {{ restic_archiver__keep_yearly }} + {%- endif -%} + {% if repo.keep_within is defined and repo.keep_within != None %} \ + --keep-within {{ repo.keep_within }} {% endif -%} +{%- endmacro %} -export RESTIC_REPOSITORY={{ restic_repos[item.repo].location }} -export RESTIC_PASSWORD='{{ restic_repos[item.repo].password | regex_replace('\'', '\'\\\'\'') }}' -BACKUP_NAME={{ item.name }} +# Settings for Server {{ repo['name'] | string }} +export RESTIC_REPOSITORY="{{ repo['location'] }}" +export RESTIC_PASSWORD='''{{ repo['password'] | regex_replace('\'', '\'\\\'\'') }}''' +BACKUP_NAME="{{ repo.name }}" + + +{{ restic_install_path }}/restic forget {{ retention_pattern(repo) }} {% if repo.prune is defined and repo.prune == true %}--prune{% endif %} + + +{% endfor %} +{# +export RESTIC_REPOSITORY="{{ restic_repos[item.repo].location }}" +export RESTIC_PASSWORD="{{ restic_repos[item.repo].password | regex_replace('\'', '\'\\\'\'') }}" +BACKUP_NAME="{{ item.name }}" {% if item.src is defined -%} # BACKUP_SOURCE={{ item.src }} {%- endif %} -set -euxo pipefail -{# - Define Tags -#} -{% macro tags(tags) -%} - {% if tags is defined and (tags|length>0) %}{% for tag in tags %} --tag {{ tag }}{% endfor %}{% endif %} -{%- endmacro %} -{# - Define Keeped Tags -#} -{% macro keep_tags(tags) -%} - {% if tags is defined and (tags|length>0) %}{% for tag in tags %} --keep-tag {{ tag }}{% endfor %}{% endif %} -{%- endmacro %} - - -{# +{xxx Define Hostname -#} +xxx} {% macro hostname(h) -%} {% if h is defined %} --hostname {{ h }}{% endif %} {%- endmacro %} -{# +{xxx Define stdin filename -#} +xxx} {% macro stdin_filename(n) -%} {% if n is defined %} --stdin-filename {{ n }}{% endif %} {%- endmacro %} -{# +{xxx Define path -#} +xxx} {% macro path(repo) -%} {% if repo.src is defined and repo.src != None and (repo.src|length>0) %}{{ repo.src }}{% else %}{{ repo.stdin_filename }}{% endif %} {%- endmacro %} -{# +{xxx Define retention pattern -#} +xxx} {% macro retention_pattern(repo) -%} {% if repo.keep_last is defined and repo.keep_last != None %}--keep-last {{ item.keep_last }}{% endif %} \ {% if repo.keep_hourly is defined and repo.keep_hourly != None %}--keep-hourly {{ item.keep_hourly }}{% endif %} \ @@ -86,7 +125,11 @@ set -euxo pipefail {% endfor %} {% endif %} {% endmacro %} -{# + + + +{xxx Define backup commands -#} +xxx} {{ restic_install_path }}/restic forget {{ retention_pattern(item) }} {% if item.prune is defined and item.prune == true %}--prune{% endif %} +#} diff --git a/vars/main.yml b/vars/main.yml index cf6567a..4d7db3a 100644 --- a/vars/main.yml +++ b/vars/main.yml @@ -1,16 +1,3 @@ --- -# we use a platform map to decipher system version -_platform_map: - i386: 386 - x86_64: amd64 - aarch64: arm64 - armv7l: arm - armv6l: arm - -restic_create_paths: - - '{{ restic_download_path }}/bin' - - '{{ restic_script_dir }}' - - --- playbook_version_number: 20 # should be over ninethousand playbook_version_path: 'role-restic_archiver_roles-ansible_github.com.version'