ansible/community.general
1
0
Fork 0
mirror of https://github.com/ansible-collections/community.general.git synced 2024-09-14 20:13:21 +02:00
Code Releases Activity
community.general/test/integration/targets/vault/runme.sh

507 lines
25 KiB
Bash
Raw Normal View History

Split integration tests out from Makefile. (#17976)
2016-10-12 23:57:53 +02:00
#!/usr/bin/env bash
Support using vault password files that are themselves vault encrypted (#27668) Extract vault related bits of DataLoader._get_file_contents to DataLoader._decrypt_if_vault_data When loading vault password files, detect if they are vault encrypted, and if so, try to decrypt with any already known vault secrets. This implements the 'Allow vault password files to be vault encrypted' (#31002) feature card from the 2.5.0 project at https://github.com/ansible/ansible/projects/9 Fixes #31002
2018-01-20 20:56:18 +01:00
set -euvx
Split integration tests out from Makefile. (#17976)
2016-10-12 23:57:53 +02:00
Add smoke/intg tests for vault cli (#21679) Add smoke/intg tests for vault cli
2017-02-24 16:30:54 +01:00
MYTMPDIR=$(mktemp -d 2>/dev/null || mktemp -d -t 'mytmpdir')
trap 'rm -rf "${MYTMPDIR}"' EXIT
add a intg test for vault encrypted inventory (#18550)
2016-11-22 16:56:37 +01:00
Add smoke/intg tests for vault cli (#21679) Add smoke/intg tests for vault cli
2017-02-24 16:30:54 +01:00
# create a test file
TEST_FILE="${MYTMPDIR}/test_file"
echo "This is a test file" > "${TEST_FILE}"
Support multiple vault passwords (#22756) Fixes #13243 ** Add --vault-id to name/identify multiple vault passwords Use --vault-id to indicate id and path/type --vault-id=prompt # prompt for default vault id password --vault-id=myorg@prompt # prompt for a vault_id named 'myorg' --vault-id=a_password_file # load ./a_password_file for default id --vault-id=myorg@a_password_file # load file for 'myorg' vault id vault_id's are created implicitly for existing --vault-password-file and --ask-vault-pass options. Vault ids are just for UX purposes and bookkeeping. Only the vault payload and the password bytestring is needed to decrypt a vault blob. Replace passing password around everywhere with a VaultSecrets object. If we specify a vault_id, mention that in password prompts Specifying multiple -vault-password-files will now try each until one works ** Rev vault format in a backwards compatible way The 1.2 vault format adds the vault_id to the header line of the vault text. This is backwards compatible with older versions of ansible. Old versions will just ignore it and treat it as the default (and only) vault id. Note: only 2.4+ supports multiple vault passwords, so while earlier ansible versions can read the vault-1.2 format, it does not make them magically support multiple vault passwords. use 1.1 format for 'default' vault_id Vaulted items that need to include a vault_id will be written in 1.2 format. If we set a new DEFAULT_VAULT_IDENTITY, then the default will use version 1.2 vault will only use a vault_id if one is specified. So if none is specified and C.DEFAULT_VAULT_IDENTITY is 'default' we use the old format. ** Changes/refactors needed to implement multiple vault passwords raise exceptions on decrypt fail, check vault id early split out parsing the vault plaintext envelope (with the sha/original plaintext) to _split_plaintext_envelope() some cli fixups for specifying multiple paths in the unfrack_paths optparse callback fix py3 dict.keys() 'dict_keys object is not indexable' error pluralize cli.options.vault_password_file -> vault_password_files pluralize cli.options.new_vault_password_file -> new_vault_password_files pluralize cli.options.vault_id -> cli.options.vault_ids ** Add a config option (vault_id_match) to force vault id matching. With 'vault_id_match=True' and an ansible vault that provides a vault_id, then decryption will require that a matching vault_id is required. (via --vault-id=my_vault_id@password_file, for ex). In other words, if the config option is true, then only the vault secrets with matching vault ids are candidates for decrypting a vault. If option is false (the default), then all of the provided vault secrets will be selected. If a user doesn't want all vault secrets to be tried to decrypt any vault content, they can enable this option. Note: The vault id used for the match is not encrypted or cryptographically signed. It is just a label/id/nickname used for referencing a specific vault secret.
2017-07-28 21:20:58 +02:00
TEST_FILE_1_2="${MYTMPDIR}/test_file_1_2"
echo "This is a test file for format 1.2" > "${TEST_FILE_1_2}"
Support using vault password files that are themselves vault encrypted (#27668) Extract vault related bits of DataLoader._get_file_contents to DataLoader._decrypt_if_vault_data When loading vault password files, detect if they are vault encrypted, and if so, try to decrypt with any already known vault secrets. This implements the 'Allow vault password files to be vault encrypted' (#31002) feature card from the 2.5.0 project at https://github.com/ansible/ansible/projects/9 Fixes #31002
2018-01-20 20:56:18 +01:00
TEST_FILE_ENC_PASSWORD="${MYTMPDIR}/test_file_enc_password"
echo "This is a test file for encrypted with a vault password that is itself vault encrypted" > "${TEST_FILE_ENC_PASSWORD}"
TEST_FILE_ENC_PASSWORD_DEFAULT="${MYTMPDIR}/test_file_enc_password_default"
echo "This is a test file for encrypted with a vault password that is itself vault encrypted using --encrypted-vault-id default" > "${TEST_FILE_ENC_PASSWORD_DEFAULT}"
Fix vault reading from stdin (avoid realpath() on non-links) (#23583) * Fix vault reading from stdin (avoid realpath() on non-links) os.path.realpath() is used to find the target of file paths that are symlinks so vault operations happen directly on the target. However, in addition to resolving symlinks, realpath() also returns a full path. when reading from stdin, vault cli uses '-' as a special file path so VaultEditor() will replace with stdin. realpath() was expanding '-' with the CWD to something like '/home/user/playbooks/-' causing errors like: ERROR! [Errno 2] No such file or directory: u'/home/user/ansible/-' Fix is to specialcase '-' to not use realpath() Fixes #23567 * to_text decrypt output when writing to stdout
2017-04-18 19:09:02 +02:00
TEST_FILE_OUTPUT="${MYTMPDIR}/test_file_output"
Use vault_id when encrypted via vault-edit (#30772) * Use vault_id when encrypted via vault-edit On the encryption stage of 'ansible-vault edit --vault-id=someid@passfile somefile', the vault id was not being passed to encrypt() so the files were always saved with the default vault id in the 1.1 version format. When trying to edit that file a second time, also with a --vault-id, the file would be decrypted with the secret associated with the provided vault-id, but since the encrypted file had no vault id in the envelope there would be no match for 'default' secrets. (Only the --vault-id was included in the potential matches, so the vault id actually used to decrypt was not). If that list was empty, there would be an IndexError when trying to encrypted the changed file. This would result in the displayed error: ERROR! Unexpected Exception, this is probably a bug: list index out of range Fix is two parts: 1) use the vault id when encrypting from edit 2) when matching the secret to use for encrypting after edit, include the vault id that was used for decryption and not just the vault id (or lack of vault id) from the envelope. add unit tests for #30575 and intg tests for 'ansible-vault edit' Fixes #30575
2017-09-26 18:28:31 +02:00
TEST_FILE_EDIT="${MYTMPDIR}/test_file_edit"
echo "This is a test file for edit" > "${TEST_FILE_EDIT}"
Vault secrets empty password (#28186) * Better handling of empty/invalid passwords empty password files are global error and cause an exit. A warning is also emitted with more detail. ie, if any of the password/secret sources provide a bogus password (ie, empty) or fail (exception, ctrl-d, EOFError), we stop at the first error and exit. This makes behavior when entering empty password at prompt match 2.3 (ie, an error)
2017-08-15 17:01:46 +02:00
Use vault_id when encrypted via vault-edit (#30772) * Use vault_id when encrypted via vault-edit On the encryption stage of 'ansible-vault edit --vault-id=someid@passfile somefile', the vault id was not being passed to encrypt() so the files were always saved with the default vault id in the 1.1 version format. When trying to edit that file a second time, also with a --vault-id, the file would be decrypted with the secret associated with the provided vault-id, but since the encrypted file had no vault id in the envelope there would be no match for 'default' secrets. (Only the --vault-id was included in the potential matches, so the vault id actually used to decrypt was not). If that list was empty, there would be an IndexError when trying to encrypted the changed file. This would result in the displayed error: ERROR! Unexpected Exception, this is probably a bug: list index out of range Fix is two parts: 1) use the vault id when encrypting from edit 2) when matching the secret to use for encrypting after edit, include the vault id that was used for decryption and not just the vault id (or lack of vault id) from the envelope. add unit tests for #30575 and intg tests for 'ansible-vault edit' Fixes #30575
2017-09-26 18:28:31 +02:00
TEST_FILE_EDIT2="${MYTMPDIR}/test_file_edit2"
echo "This is a test file for edit2" > "${TEST_FILE_EDIT2}"
Fix 'New Vault password' on vault 'edit' (#35923) * Fix 'New Vault password' on vault 'edit' ffe0ddea96bbe8ac27af816e58667c212e74688e introduce a change on 'ansible-vault edit' that tried to check for --encrypt-vault-id in that mode. But '--encrypt-vault-id' is not intended for 'edit' since the 'edit' should always reuse the vault secret that was used to decrypt the text. Change cli to not check for --encrypt-vault-id on 'edit'. VaultLib.decrypt_and_get_vault_id() was change to return the vault secret used to decrypt (in addition to vault_id and the plaintext). VaultEditor.edit_file() will now use 'vault_secret_used' as returned from decrypt_and_get_vault_id() so that an edited file always gets reencrypted with the same secret, regardless of any vault id configuration or cli options. Fixes #35834
2018-03-27 20:12:21 +02:00
# test case for https://github.com/ansible/ansible/issues/35834
# (being prompted for new password on vault-edit with no configured passwords)
TEST_FILE_EDIT3="${MYTMPDIR}/test_file_edit3"
echo "This is a test file for edit3" > "${TEST_FILE_EDIT3}"
# ansible-config view
ansible-config view
# ansisle-config
ansible-config dump --only-changed
ansible-vault encrypt "$@" --vault-id vault-password "${TEST_FILE_EDIT3}"
# EDITOR=./faux-editor.py ansible-vault edit "$@" "${TEST_FILE_EDIT3}"
EDITOR=./faux-editor.py ansible-vault edit --vault-id vault-password -vvvvv "${TEST_FILE_EDIT3}"
echo $?
Support using vault password files that are themselves vault encrypted (#27668) Extract vault related bits of DataLoader._get_file_contents to DataLoader._decrypt_if_vault_data When loading vault password files, detect if they are vault encrypted, and if so, try to decrypt with any already known vault secrets. This implements the 'Allow vault password files to be vault encrypted' (#31002) feature card from the 2.5.0 project at https://github.com/ansible/ansible/projects/9 Fixes #31002
2018-01-20 20:56:18 +01:00
# view the vault encrypted password file
ansible-vault view "$@" --vault-id vault-password encrypted-vault-password
# encrypt with a password from a vault encrypted password file and multiple vault-ids
# should fail because we dont know which vault id to use to encrypt with
ansible-vault encrypt "$@" --vault-id vault-password --vault-id encrypted-vault-password "${TEST_FILE_ENC_PASSWORD}" && :
WRONG_RC=$?
echo "rc was $WRONG_RC (5 is expected)"
[ $WRONG_RC -eq 5 ]
# try to view the file encrypted with the vault-password we didnt specify
# to verify we didnt choose the wrong vault-id
ansible-vault view "$@" --vault-id vault-password encrypted-vault-password
Use vault_id when encrypted via vault-edit (#30772) * Use vault_id when encrypted via vault-edit On the encryption stage of 'ansible-vault edit --vault-id=someid@passfile somefile', the vault id was not being passed to encrypt() so the files were always saved with the default vault id in the 1.1 version format. When trying to edit that file a second time, also with a --vault-id, the file would be decrypted with the secret associated with the provided vault-id, but since the encrypted file had no vault id in the envelope there would be no match for 'default' secrets. (Only the --vault-id was included in the potential matches, so the vault id actually used to decrypt was not). If that list was empty, there would be an IndexError when trying to encrypted the changed file. This would result in the displayed error: ERROR! Unexpected Exception, this is probably a bug: list index out of range Fix is two parts: 1) use the vault id when encrypting from edit 2) when matching the secret to use for encrypting after edit, include the vault id that was used for decryption and not just the vault id (or lack of vault id) from the envelope. add unit tests for #30575 and intg tests for 'ansible-vault edit' Fixes #30575
2017-09-26 18:28:31 +02:00
FORMAT_1_1_HEADER="\$ANSIBLE_VAULT;1.1;AES256"
FORMAT_1_2_HEADER="\$ANSIBLE_VAULT;1.2;AES256"
Fix config value type for VAULT_IDENTITY_LIST (#28678) Was using the 'value_type' key, but didn't get updated to the new 'type' key in merge. Fix playbooks cli so it uses VAULT_IDENTITY_LIST as well.
2017-08-28 16:13:14 +02:00
Better handling of malformed vault data envelope (#32515) * Better handling of malformed vault data envelope If an embedded vaulted variable ('!vault' in yaml) had an invalid format, it would eventually cause an error for seemingly unrelated reasons. "Invalid" meaning not valid hexlify (extra chars, non-hex chars, etc). For ex, if a host_vars file had invalid vault format variables, on py2, it would cause an error like: 'ansible.vars.hostvars.HostVars object' has no attribute u'broken.example.com' Depending on where the invalid vault is, it could also cause "VARIABLE IS NOT DEFINED!". The behavior can also change if ansible-playbook is py2 or py3. Root cause is errors from binascii.unhexlify() not being handled consistently. Fix is to add a AnsibleVaultFormatError exception and raise it on any unhexlify() errors and to handle it properly elsewhere. Add a _unhexlify() that try/excepts around a binascii.unhexlify() and raises an AnsibleVaultFormatError on invalid vault data. This is so the same exception type is always raised for this case. Previous it was different between py2 and py3. binascii.unhexlify() raises a binascii.Error if the hexlified blobs in a vault data blob are invalid. On py2, binascii.Error is a subclass of Exception. On py3, binascii.Error is a subclass of TypeError When decrypting content of vault encrypted variables, if a binascii.Error is raised it propagates up to playbook.base.Base.post_validate(). post_validate() handles exceptions for TypeErrors but not for base Exception subclasses (like py2 binascii.Error). * Add a display.warning on vault format errors * Unit tests for _unhexlify, parse_vaulttext* * Add intg test cases for invalid vault formats Fixes #28038
2017-11-10 20:24:56 +01:00
Handle vault decrypt --output=- (#31066) In cli.CLI.unfrack_path callback, special case if the value of '--output' is '-', and avoid expanding it to a full path. vault cli already has special cases for '-', so it just needs to get the original value to work. Fixes #30550
2017-10-03 18:02:16 +02:00
VAULT_PASSWORD_FILE=vault-password
Vault secrets script client inc new 'keyring' client (#27669) This adds a new type of vault-password script (a 'client') that takes advantage of and enhances the multiple vault password support. If a vault password script basename ends with the name '-client', consider it a vault password script client. A vault password script 'client' just means that the script will take a '--vault-id' command line arg. The previous vault password script (as invoked by --vault-password-file pointing to an executable) takes no args and returns the password on stdout. But it doesnt know anything about --vault-id or multiple vault passwords. The new 'protocol' of the vault password script takes a cli arg ('--vault-id') so that it can lookup that specific vault-id and return it's password. Since existing vault password scripts don't know the new 'protocol', a way to distinguish password scripts that do understand the protocol was needed. The convention now is to consider password scripts that are named like 'something-client.py' (and executable) to be vault password client scripts. The new client scripts get invoked with the '--vault-id' they were requested for. An example: ansible-playbook --vault-id my_vault_id@contrib/vault/vault-keyring-client.py some_playbook.yml That will cause the 'contrib/vault/vault-keyring-client.py' script to be invoked as: contrib/vault/vault-keyring-client.py --vault-id my_vault_id The previous vault-keyring.py password script was extended to become vault-keyring-client.py. It uses the python 'keyring' module to request secrets from various backends. The plain 'vault-keyring.py' script would determine which key id and keyring name to use based on values that had to be set in ansible.cfg. So it was also limited to one keyring name. The new vault-keyring-client.py will request the secret for the vault id provided via the '--vault-id' option. The script can be used without config and can be used for multiple keyring ids (and keyrings). On success, a vault password client script will print the password to stdout and exit with a return code of 0. If the 'client' script can't find a secret for the --vault-id, the script will exit with return code of 2 and print an error to stderr.
2017-10-13 21:23:08 +02:00
# new format, view, using password client script
ansible-vault view "$@" --vault-id vault-password@test-vault-client.py format_1_1_AES256.yml
# view, using password client script, unknown vault/keyname
ansible-vault view "$@" --vault-id some_unknown_vault_id@test-vault-client.py format_1_1_AES256.yml && :
Handle vault decrypt --output=- (#31066) In cli.CLI.unfrack_path callback, special case if the value of '--output' is '-', and avoid expanding it to a full path. vault cli already has special cases for '-', so it just needs to get the original value to work. Fixes #30550
2017-10-03 18:02:16 +02:00
Fix vault --ask-vault-pass with no tty (#31493) * Fix vault --ask-vault-pass with no tty 2.4.0 added a check for isatty() that would skip setting up interactive vault password prompts if not running on a tty. But... getpass.getpass() will fallback to reading from stdin if it gets that far without a tty. Since 2.4.0 skipped the interactive prompts / getpass.getpass() in that case, it would never get a chance to fall back to stdin. So if 'echo $VAULT_PASSWORD| ansible-playbook --ask-vault-pass site.yml' was ran without a tty (ie, from a jenkins job or via the vagrant ansible provisioner) the 2.4 behavior was different than 2.3. 2.4 would never read the password from stdin, resulting in a vault password error like: ERROR! Attempting to decrypt but no vault secrets found Fix is just to always call the interactive password prompts based on getpass.getpass() on --ask-vault-pass or --vault-id @prompt and let getpass sort it out. * up test_prompt_no_tty to expect prompt with no tty We do call the PromptSecret class if there is no tty, but we are back to expecting it to read from stdin in that case. * Fix logic for when to auto-prompt vault pass If --ask-vault-pass is used, then pretty much always prompt. If it is not used, then prompt if there are no other vault ids provided and 'auto_prompt==True'. Fixes vagrant bug https://github.com/hashicorp/vagrant/issues/9033 Fixes #30993
2017-11-15 20:01:32 +01:00
# Use linux setsid to test without a tty. No setsid if osx/bsd though...
if [ -x "$(command -v setsid)" ]; then
# tests related to https://github.com/ansible/ansible/issues/30993
CMD='ansible-playbook -vvvvv --ask-vault-pass test_vault.yml'
setsid sh -c "echo test-vault-password|${CMD}" < /dev/null > log 2>&1 && :
WRONG_RC=$?
cat log
echo "rc was $WRONG_RC (0 is expected)"
[ $WRONG_RC -eq 0 ]
setsid sh -c 'tty; ansible-vault --ask-vault-pass -vvvvv view test_vault.yml' < /dev/null > log 2>&1 && :
WRONG_RC=$?
echo "rc was $WRONG_RC (1 is expected)"
[ $WRONG_RC -eq 1 ]
cat log
setsid sh -c 'tty; echo passbhkjhword|ansible-playbook -vvvvv --ask-vault-pass test_vault.yml' < /dev/null > log 2>&1 && :
WRONG_RC=$?
echo "rc was $WRONG_RC (1 is expected)"
[ $WRONG_RC -eq 1 ]
cat log
setsid sh -c 'tty; echo test-vault-password |ansible-playbook -vvvvv --ask-vault-pass test_vault.yml' < /dev/null > log 2>&1
echo $?
cat log
setsid sh -c 'tty; echo test-vault-password|ansible-playbook -vvvvv --ask-vault-pass test_vault.yml' < /dev/null > log 2>&1
echo $?
cat log
setsid sh -c 'tty; echo test-vault-password |ansible-playbook -vvvvv --ask-vault-pass test_vault.yml' < /dev/null > log 2>&1
echo $?
cat log
setsid sh -c 'tty; echo test-vault-password|ansible-vault --ask-vault-pass -vvvvv view vaulted.inventory' < /dev/null > log 2>&1
echo $?
cat log
fi
Cyptography pr 20566 rebase (#25560) Make pyca/cryptography the preferred backend for cryptographic needs (mainly vault) falling back to pycrypto pyca/cryptography is already implicitly a dependency in many cases through paramiko (2.0+) as well as the new openssl_publickey module, which requires pyOpenSSL 16.0+. Additionally, pyca/cryptography is an optional dep for better performance with vault already. This commit leverages cryptography's padding, constant time comparisons, and CBC/CTR modes to reduce the amount of code ansible needs to maintain. * Handle wrong password given for VaultAES format * Do not display deprecation warning for cryptography on python-2.6 * Namespace all of the pycrypto imports and always import them Makes unittests better and the code less likely to get stupid mistakes (like using HMAC from cryptogrpahy when the one from pycrypto is needed) * Add back in atfork since we need pycrypto to reinitialize its RNG just in case we're being used with old paramiko * contrib/inventory/gce: Remove spurious require on pycrypto (cherry picked from commit 9e16b9db275263b3ea8d1b124966fdebfc9ab271) * Add cryptography to ec2_win_password module requirements * Fix python3 bug which would pass text strings to a function which requires byte strings. * Attempt to add pycrypto version to setup deps * Change hacking README for dual pycrypto/cryptography * update dependencies for various CI scripts * additional CI dockerfile/script updates * add paramiko to the windows and sanity requirement set This is needed because ansible lists it as a requirement. Previously the missing dep wasn't enforced, but cryptography imports pkg_resources so you can't ignore a requirement any more * Add integration test cases for old vault and for wrong passwords * helper script for manual testing of pycrypto/cryptography * Skip the pycrypto tests so that users without it installed can still run the unittests * Run unittests for vault with both cryptography and pycrypto backend
2017-06-27 15:00:15 +02:00
# old format
ansible-vault view "$@" --vault-password-file vault-password-ansible format_1_0_AES.yml
ansible-vault view "$@" --vault-password-file vault-password-ansible format_1_1_AES.yml
# old format, wrong password
echo "The wrong password tests are expected to return 1"
ansible-vault view "$@" --vault-password-file vault-password-wrong format_1_0_AES.yml && :
WRONG_RC=$?
echo "rc was $WRONG_RC (1 is expected)"
[ $WRONG_RC -eq 1 ]
ansible-vault view "$@" --vault-password-file vault-password-wrong format_1_1_AES.yml && :
WRONG_RC=$?
echo "rc was $WRONG_RC (1 is expected)"
[ $WRONG_RC -eq 1 ]
ansible-vault view "$@" --vault-password-file vault-password-wrong format_1_1_AES256.yml && :
WRONG_RC=$?
echo "rc was $WRONG_RC (1 is expected)"
[ $WRONG_RC -eq 1 ]
set -eux
Vault secrets empty password (#28186) * Better handling of empty/invalid passwords empty password files are global error and cause an exit. A warning is also emitted with more detail. ie, if any of the password/secret sources provide a bogus password (ie, empty) or fail (exception, ctrl-d, EOFError), we stop at the first error and exit. This makes behavior when entering empty password at prompt match 2.3 (ie, an error)
2017-08-15 17:01:46 +02:00
Cyptography pr 20566 rebase (#25560) Make pyca/cryptography the preferred backend for cryptographic needs (mainly vault) falling back to pycrypto pyca/cryptography is already implicitly a dependency in many cases through paramiko (2.0+) as well as the new openssl_publickey module, which requires pyOpenSSL 16.0+. Additionally, pyca/cryptography is an optional dep for better performance with vault already. This commit leverages cryptography's padding, constant time comparisons, and CBC/CTR modes to reduce the amount of code ansible needs to maintain. * Handle wrong password given for VaultAES format * Do not display deprecation warning for cryptography on python-2.6 * Namespace all of the pycrypto imports and always import them Makes unittests better and the code less likely to get stupid mistakes (like using HMAC from cryptogrpahy when the one from pycrypto is needed) * Add back in atfork since we need pycrypto to reinitialize its RNG just in case we're being used with old paramiko * contrib/inventory/gce: Remove spurious require on pycrypto (cherry picked from commit 9e16b9db275263b3ea8d1b124966fdebfc9ab271) * Add cryptography to ec2_win_password module requirements * Fix python3 bug which would pass text strings to a function which requires byte strings. * Attempt to add pycrypto version to setup deps * Change hacking README for dual pycrypto/cryptography * update dependencies for various CI scripts * additional CI dockerfile/script updates * add paramiko to the windows and sanity requirement set This is needed because ansible lists it as a requirement. Previously the missing dep wasn't enforced, but cryptography imports pkg_resources so you can't ignore a requirement any more * Add integration test cases for old vault and for wrong passwords * helper script for manual testing of pycrypto/cryptography * Skip the pycrypto tests so that users without it installed can still run the unittests * Run unittests for vault with both cryptography and pycrypto backend
2017-06-27 15:00:15 +02:00
# new format, view
ansible-vault view "$@" --vault-password-file vault-password format_1_1_AES256.yml
Support multiple vault passwords (#22756) Fixes #13243 ** Add --vault-id to name/identify multiple vault passwords Use --vault-id to indicate id and path/type --vault-id=prompt # prompt for default vault id password --vault-id=myorg@prompt # prompt for a vault_id named 'myorg' --vault-id=a_password_file # load ./a_password_file for default id --vault-id=myorg@a_password_file # load file for 'myorg' vault id vault_id's are created implicitly for existing --vault-password-file and --ask-vault-pass options. Vault ids are just for UX purposes and bookkeeping. Only the vault payload and the password bytestring is needed to decrypt a vault blob. Replace passing password around everywhere with a VaultSecrets object. If we specify a vault_id, mention that in password prompts Specifying multiple -vault-password-files will now try each until one works ** Rev vault format in a backwards compatible way The 1.2 vault format adds the vault_id to the header line of the vault text. This is backwards compatible with older versions of ansible. Old versions will just ignore it and treat it as the default (and only) vault id. Note: only 2.4+ supports multiple vault passwords, so while earlier ansible versions can read the vault-1.2 format, it does not make them magically support multiple vault passwords. use 1.1 format for 'default' vault_id Vaulted items that need to include a vault_id will be written in 1.2 format. If we set a new DEFAULT_VAULT_IDENTITY, then the default will use version 1.2 vault will only use a vault_id if one is specified. So if none is specified and C.DEFAULT_VAULT_IDENTITY is 'default' we use the old format. ** Changes/refactors needed to implement multiple vault passwords raise exceptions on decrypt fail, check vault id early split out parsing the vault plaintext envelope (with the sha/original plaintext) to _split_plaintext_envelope() some cli fixups for specifying multiple paths in the unfrack_paths optparse callback fix py3 dict.keys() 'dict_keys object is not indexable' error pluralize cli.options.vault_password_file -> vault_password_files pluralize cli.options.new_vault_password_file -> new_vault_password_files pluralize cli.options.vault_id -> cli.options.vault_ids ** Add a config option (vault_id_match) to force vault id matching. With 'vault_id_match=True' and an ansible vault that provides a vault_id, then decryption will require that a matching vault_id is required. (via --vault-id=my_vault_id@password_file, for ex). In other words, if the config option is true, then only the vault secrets with matching vault ids are candidates for decrypting a vault. If option is false (the default), then all of the provided vault secrets will be selected. If a user doesn't want all vault secrets to be tried to decrypt any vault content, they can enable this option. Note: The vault id used for the match is not encrypted or cryptographically signed. It is just a label/id/nickname used for referencing a specific vault secret.
2017-07-28 21:20:58 +02:00
# new format, view with vault-id
ansible-vault view "$@" --vault-id=vault-password format_1_1_AES256.yml
# new format, view, using password script
ansible-vault view "$@" --vault-password-file password-script.py format_1_1_AES256.yml
# new format, view, using password script with vault-id
ansible-vault view "$@" --vault-id password-script.py format_1_1_AES256.yml
# new 1.2 format, view
ansible-vault view "$@" --vault-password-file vault-password format_1_2_AES256.yml
# new 1.2 format, view with vault-id
ansible-vault view "$@" --vault-id=test_vault_id@vault-password format_1_2_AES256.yml
# new 1,2 format, view, using password script
ansible-vault view "$@" --vault-password-file password-script.py format_1_2_AES256.yml
# new 1.2 format, view, using password script with vault-id
ansible-vault view "$@" --vault-id password-script.py format_1_2_AES256.yml
Fix config value type for VAULT_IDENTITY_LIST (#28678) Was using the 'value_type' key, but didn't get updated to the new 'type' key in merge. Fix playbooks cli so it uses VAULT_IDENTITY_LIST as well.
2017-08-28 16:13:14 +02:00
# newish 1.1 format, view, using a vault-id list from config env var
ANSIBLE_VAULT_IDENTITY_LIST='wrong-password@vault-password-wrong,default@vault-password' ansible-vault view "$@" --vault-id password-script.py format_1_1_AES256.yml
Support multiple vault passwords (#22756) Fixes #13243 ** Add --vault-id to name/identify multiple vault passwords Use --vault-id to indicate id and path/type --vault-id=prompt # prompt for default vault id password --vault-id=myorg@prompt # prompt for a vault_id named 'myorg' --vault-id=a_password_file # load ./a_password_file for default id --vault-id=myorg@a_password_file # load file for 'myorg' vault id vault_id's are created implicitly for existing --vault-password-file and --ask-vault-pass options. Vault ids are just for UX purposes and bookkeeping. Only the vault payload and the password bytestring is needed to decrypt a vault blob. Replace passing password around everywhere with a VaultSecrets object. If we specify a vault_id, mention that in password prompts Specifying multiple -vault-password-files will now try each until one works ** Rev vault format in a backwards compatible way The 1.2 vault format adds the vault_id to the header line of the vault text. This is backwards compatible with older versions of ansible. Old versions will just ignore it and treat it as the default (and only) vault id. Note: only 2.4+ supports multiple vault passwords, so while earlier ansible versions can read the vault-1.2 format, it does not make them magically support multiple vault passwords. use 1.1 format for 'default' vault_id Vaulted items that need to include a vault_id will be written in 1.2 format. If we set a new DEFAULT_VAULT_IDENTITY, then the default will use version 1.2 vault will only use a vault_id if one is specified. So if none is specified and C.DEFAULT_VAULT_IDENTITY is 'default' we use the old format. ** Changes/refactors needed to implement multiple vault passwords raise exceptions on decrypt fail, check vault id early split out parsing the vault plaintext envelope (with the sha/original plaintext) to _split_plaintext_envelope() some cli fixups for specifying multiple paths in the unfrack_paths optparse callback fix py3 dict.keys() 'dict_keys object is not indexable' error pluralize cli.options.vault_password_file -> vault_password_files pluralize cli.options.new_vault_password_file -> new_vault_password_files pluralize cli.options.vault_id -> cli.options.vault_ids ** Add a config option (vault_id_match) to force vault id matching. With 'vault_id_match=True' and an ansible vault that provides a vault_id, then decryption will require that a matching vault_id is required. (via --vault-id=my_vault_id@password_file, for ex). In other words, if the config option is true, then only the vault secrets with matching vault ids are candidates for decrypting a vault. If option is false (the default), then all of the provided vault secrets will be selected. If a user doesn't want all vault secrets to be tried to decrypt any vault content, they can enable this option. Note: The vault id used for the match is not encrypted or cryptographically signed. It is just a label/id/nickname used for referencing a specific vault secret.
2017-07-28 21:20:58 +02:00
# new 1.2 format, view, ENFORCE_IDENTITY_MATCH=true, should fail, no 'test_vault_id' vault_id
ANSIBLE_VAULT_ID_MATCH=1 ansible-vault view "$@" --vault-password-file vault-password format_1_2_AES256.yml && :
WRONG_RC=$?
echo "rc was $WRONG_RC (1 is expected)"
[ $WRONG_RC -eq 1 ]
# new 1.2 format, view with vault-id, ENFORCE_IDENTITY_MATCH=true, should work, 'test_vault_id' is provided
ANSIBLE_VAULT_ID_MATCH=1 ansible-vault view "$@" --vault-id=test_vault_id@vault-password format_1_2_AES256.yml
# new 1,2 format, view, using password script, ENFORCE_IDENTITY_MATCH=true, should fail, no 'test_vault_id'
ANSIBLE_VAULT_ID_MATCH=1 ansible-vault view "$@" --vault-password-file password-script.py format_1_2_AES256.yml && :
WRONG_RC=$?
echo "rc was $WRONG_RC (1 is expected)"
[ $WRONG_RC -eq 1 ]
# new 1.2 format, view, using password script with vault-id, ENFORCE_IDENTITY_MATCH=true, should fail
ANSIBLE_VAULT_ID_MATCH=1 ansible-vault view "$@" --vault-id password-script.py format_1_2_AES256.yml && :
WRONG_RC=$?
echo "rc was $WRONG_RC (1 is expected)"
[ $WRONG_RC -eq 1 ]
# new 1.2 format, view, using password script with vault-id, ENFORCE_IDENTITY_MATCH=true, 'test_vault_id' provided should work
ANSIBLE_VAULT_ID_MATCH=1 ansible-vault view "$@" --vault-id=test_vault_id@password-script.py format_1_2_AES256.yml
Add back support for vault_password_file config var (#27597) Got removed in arg parsing updates. Now added back in setup_vault_secrets(). The default value for DEFAULT_VAULT_PASSWORD_FILE was also set to '~' for some reason, change to to no default. Add integration tests.
2017-08-02 00:07:33 +02:00
# test with a default vault password set via config/env, right password
ANSIBLE_VAULT_PASSWORD_FILE=vault-password ansible-vault view "$@" format_1_1_AES256.yml
# test with a default vault password set via config/env, wrong password
ANSIBLE_VAULT_PASSWORD_FILE=vault-password-wrong ansible-vault view "$@" format_1_1_AES.yml && :
WRONG_RC=$?
echo "rc was $WRONG_RC (1 is expected)"
[ $WRONG_RC -eq 1 ]
Vault secrets default vault ids list (#28190) * Add config option for a default list of vault-ids This is the vault-id equilivent of ANSIBLE_DEFAULT_PASSWORD_FILE except ANSIBLE_DEFAULT_VAULT_IDENTITY_LIST is a list.
2017-08-15 17:56:17 +02:00
# test with a default vault-id list set via config/env, right password
ANSIBLE_VAULT_PASSWORD_FILE=wrong@vault-password-wrong,correct@vault-password ansible-vault view "$@" format_1_1_AES.yml && :
# test with a default vault-id list set via config/env,wrong passwords
ANSIBLE_VAULT_PASSWORD_FILE=wrong@vault-password-wrong,alsowrong@vault-password-wrong ansible-vault view "$@" format_1_1_AES.yml && :
WRONG_RC=$?
echo "rc was $WRONG_RC (1 is expected)"
[ $WRONG_RC -eq 1 ]
add a vault --encrypt-vault-to specify vault id to use for encrypt (#31067) Enforce that there can be only one --new-vault-id or --new-vault-password-file and use this instead of --encrypt-vault-id * Add a config option for default vault encrypt id
2018-01-22 23:12:10 +01:00
# try specifying a --encrypt-vault-id that doesnt exist, should exit with an error indicating
# that --encrypt-vault-id and the known vault-ids
ansible-vault encrypt "$@" --vault-password-file vault-password --encrypt-vault-id doesnt_exist "${TEST_FILE}" && :
WRONG_RC=$?
echo "rc was $WRONG_RC (1 is expected)"
[ $WRONG_RC -eq 1 ]
Add smoke/intg tests for vault cli (#21679) Add smoke/intg tests for vault cli
2017-02-24 16:30:54 +01:00
# encrypt it
ansible-vault encrypt "$@" --vault-password-file vault-password "${TEST_FILE}"
ansible-vault view "$@" --vault-password-file vault-password "${TEST_FILE}"
Support multiple vault passwords (#22756) Fixes #13243 ** Add --vault-id to name/identify multiple vault passwords Use --vault-id to indicate id and path/type --vault-id=prompt # prompt for default vault id password --vault-id=myorg@prompt # prompt for a vault_id named 'myorg' --vault-id=a_password_file # load ./a_password_file for default id --vault-id=myorg@a_password_file # load file for 'myorg' vault id vault_id's are created implicitly for existing --vault-password-file and --ask-vault-pass options. Vault ids are just for UX purposes and bookkeeping. Only the vault payload and the password bytestring is needed to decrypt a vault blob. Replace passing password around everywhere with a VaultSecrets object. If we specify a vault_id, mention that in password prompts Specifying multiple -vault-password-files will now try each until one works ** Rev vault format in a backwards compatible way The 1.2 vault format adds the vault_id to the header line of the vault text. This is backwards compatible with older versions of ansible. Old versions will just ignore it and treat it as the default (and only) vault id. Note: only 2.4+ supports multiple vault passwords, so while earlier ansible versions can read the vault-1.2 format, it does not make them magically support multiple vault passwords. use 1.1 format for 'default' vault_id Vaulted items that need to include a vault_id will be written in 1.2 format. If we set a new DEFAULT_VAULT_IDENTITY, then the default will use version 1.2 vault will only use a vault_id if one is specified. So if none is specified and C.DEFAULT_VAULT_IDENTITY is 'default' we use the old format. ** Changes/refactors needed to implement multiple vault passwords raise exceptions on decrypt fail, check vault id early split out parsing the vault plaintext envelope (with the sha/original plaintext) to _split_plaintext_envelope() some cli fixups for specifying multiple paths in the unfrack_paths optparse callback fix py3 dict.keys() 'dict_keys object is not indexable' error pluralize cli.options.vault_password_file -> vault_password_files pluralize cli.options.new_vault_password_file -> new_vault_password_files pluralize cli.options.vault_id -> cli.options.vault_ids ** Add a config option (vault_id_match) to force vault id matching. With 'vault_id_match=True' and an ansible vault that provides a vault_id, then decryption will require that a matching vault_id is required. (via --vault-id=my_vault_id@password_file, for ex). In other words, if the config option is true, then only the vault secrets with matching vault ids are candidates for decrypting a vault. If option is false (the default), then all of the provided vault secrets will be selected. If a user doesn't want all vault secrets to be tried to decrypt any vault content, they can enable this option. Note: The vault id used for the match is not encrypted or cryptographically signed. It is just a label/id/nickname used for referencing a specific vault secret.
2017-07-28 21:20:58 +02:00
# view with multiple vault-password files, including a wrong one
ansible-vault view "$@" --vault-password-file vault-password --vault-password-file vault-password-wrong "${TEST_FILE}"
# view with multiple vault-password files, including a wrong one, using vault-id
ansible-vault view "$@" --vault-id vault-password --vault-id vault-password-wrong "${TEST_FILE}"
# And with the password files specified in a different order
ansible-vault view "$@" --vault-password-file vault-password-wrong --vault-password-file vault-password "${TEST_FILE}"
# And with the password files specified in a different order, using vault-id
ansible-vault view "$@" --vault-id vault-password-wrong --vault-id vault-password "${TEST_FILE}"
# And with the password files specified in a different order, using --vault-id and non default vault_ids
ansible-vault view "$@" --vault-id test_vault_id@vault-password-wrong --vault-id test_vault_id@vault-password "${TEST_FILE}"
Add smoke/intg tests for vault cli (#21679) Add smoke/intg tests for vault cli
2017-02-24 16:30:54 +01:00
ansible-vault decrypt "$@" --vault-password-file vault-password "${TEST_FILE}"
Support multiple vault passwords (#22756) Fixes #13243 ** Add --vault-id to name/identify multiple vault passwords Use --vault-id to indicate id and path/type --vault-id=prompt # prompt for default vault id password --vault-id=myorg@prompt # prompt for a vault_id named 'myorg' --vault-id=a_password_file # load ./a_password_file for default id --vault-id=myorg@a_password_file # load file for 'myorg' vault id vault_id's are created implicitly for existing --vault-password-file and --ask-vault-pass options. Vault ids are just for UX purposes and bookkeeping. Only the vault payload and the password bytestring is needed to decrypt a vault blob. Replace passing password around everywhere with a VaultSecrets object. If we specify a vault_id, mention that in password prompts Specifying multiple -vault-password-files will now try each until one works ** Rev vault format in a backwards compatible way The 1.2 vault format adds the vault_id to the header line of the vault text. This is backwards compatible with older versions of ansible. Old versions will just ignore it and treat it as the default (and only) vault id. Note: only 2.4+ supports multiple vault passwords, so while earlier ansible versions can read the vault-1.2 format, it does not make them magically support multiple vault passwords. use 1.1 format for 'default' vault_id Vaulted items that need to include a vault_id will be written in 1.2 format. If we set a new DEFAULT_VAULT_IDENTITY, then the default will use version 1.2 vault will only use a vault_id if one is specified. So if none is specified and C.DEFAULT_VAULT_IDENTITY is 'default' we use the old format. ** Changes/refactors needed to implement multiple vault passwords raise exceptions on decrypt fail, check vault id early split out parsing the vault plaintext envelope (with the sha/original plaintext) to _split_plaintext_envelope() some cli fixups for specifying multiple paths in the unfrack_paths optparse callback fix py3 dict.keys() 'dict_keys object is not indexable' error pluralize cli.options.vault_password_file -> vault_password_files pluralize cli.options.new_vault_password_file -> new_vault_password_files pluralize cli.options.vault_id -> cli.options.vault_ids ** Add a config option (vault_id_match) to force vault id matching. With 'vault_id_match=True' and an ansible vault that provides a vault_id, then decryption will require that a matching vault_id is required. (via --vault-id=my_vault_id@password_file, for ex). In other words, if the config option is true, then only the vault secrets with matching vault ids are candidates for decrypting a vault. If option is false (the default), then all of the provided vault secrets will be selected. If a user doesn't want all vault secrets to be tried to decrypt any vault content, they can enable this option. Note: The vault id used for the match is not encrypted or cryptographically signed. It is just a label/id/nickname used for referencing a specific vault secret.
2017-07-28 21:20:58 +02:00
# encrypt it, using a vault_id so we write a 1.2 format file
ansible-vault encrypt "$@" --vault-id test_vault_1_2@vault-password "${TEST_FILE_1_2}"
ansible-vault view "$@" --vault-id vault-password "${TEST_FILE_1_2}"
ansible-vault view "$@" --vault-id test_vault_1_2@vault-password "${TEST_FILE_1_2}"
# view with multiple vault-password files, including a wrong one
ansible-vault view "$@" --vault-id vault-password --vault-id wrong_password@vault-password-wrong "${TEST_FILE_1_2}"
# And with the password files specified in a different order, using vault-id
ansible-vault view "$@" --vault-id vault-password-wrong --vault-id vault-password "${TEST_FILE_1_2}"
# And with the password files specified in a different order, using --vault-id and non default vault_ids
ansible-vault view "$@" --vault-id test_vault_id@vault-password-wrong --vault-id test_vault_id@vault-password "${TEST_FILE_1_2}"
ansible-vault decrypt "$@" --vault-id test_vault_1_2@vault-password "${TEST_FILE_1_2}"
# multiple vault passwords
ansible-vault view "$@" --vault-password-file vault-password --vault-password-file vault-password-wrong format_1_1_AES256.yml
# multiple vault passwords, --vault-id
ansible-vault view "$@" --vault-id test_vault_id@vault-password --vault-id test_vault_id@vault-password-wrong format_1_1_AES256.yml
# encrypt it, with password from password script
ansible-vault encrypt "$@" --vault-password-file password-script.py "${TEST_FILE}"
ansible-vault view "$@" --vault-password-file password-script.py "${TEST_FILE}"
ansible-vault decrypt "$@" --vault-password-file password-script.py "${TEST_FILE}"
# encrypt it, with password from password script
ansible-vault encrypt "$@" --vault-id test_vault_id@password-script.py "${TEST_FILE}"
ansible-vault view "$@" --vault-id test_vault_id@password-script.py "${TEST_FILE}"
ansible-vault decrypt "$@" --vault-id test_vault_id@password-script.py "${TEST_FILE}"
Add smoke/intg tests for vault cli (#21679) Add smoke/intg tests for vault cli
2017-02-24 16:30:54 +01:00
# new password file for rekeyed file
NEW_VAULT_PASSWORD="${MYTMPDIR}/new-vault-password"
echo "newpassword" > "${NEW_VAULT_PASSWORD}"
ansible-vault encrypt "$@" --vault-password-file vault-password "${TEST_FILE}"
ansible-vault rekey "$@" --vault-password-file vault-password --new-vault-password-file "${NEW_VAULT_PASSWORD}" "${TEST_FILE}"
add a vault --encrypt-vault-to specify vault id to use for encrypt (#31067) Enforce that there can be only one --new-vault-id or --new-vault-password-file and use this instead of --encrypt-vault-id * Add a config option for default vault encrypt id
2018-01-22 23:12:10 +01:00
# --new-vault-password-file and --new-vault-id should cause options error
ansible-vault rekey "$@" --vault-password-file vault-password --new-vault-id=foobar --new-vault-password-file "${NEW_VAULT_PASSWORD}" "${TEST_FILE}" && :
WRONG_RC=$?
echo "rc was $WRONG_RC (2 is expected)"
[ $WRONG_RC -eq 2 ]
Add smoke/intg tests for vault cli (#21679) Add smoke/intg tests for vault cli
2017-02-24 16:30:54 +01:00
ansible-vault view "$@" --vault-password-file "${NEW_VAULT_PASSWORD}" "${TEST_FILE}"
Support multiple vault passwords (#22756) Fixes #13243 ** Add --vault-id to name/identify multiple vault passwords Use --vault-id to indicate id and path/type --vault-id=prompt # prompt for default vault id password --vault-id=myorg@prompt # prompt for a vault_id named 'myorg' --vault-id=a_password_file # load ./a_password_file for default id --vault-id=myorg@a_password_file # load file for 'myorg' vault id vault_id's are created implicitly for existing --vault-password-file and --ask-vault-pass options. Vault ids are just for UX purposes and bookkeeping. Only the vault payload and the password bytestring is needed to decrypt a vault blob. Replace passing password around everywhere with a VaultSecrets object. If we specify a vault_id, mention that in password prompts Specifying multiple -vault-password-files will now try each until one works ** Rev vault format in a backwards compatible way The 1.2 vault format adds the vault_id to the header line of the vault text. This is backwards compatible with older versions of ansible. Old versions will just ignore it and treat it as the default (and only) vault id. Note: only 2.4+ supports multiple vault passwords, so while earlier ansible versions can read the vault-1.2 format, it does not make them magically support multiple vault passwords. use 1.1 format for 'default' vault_id Vaulted items that need to include a vault_id will be written in 1.2 format. If we set a new DEFAULT_VAULT_IDENTITY, then the default will use version 1.2 vault will only use a vault_id if one is specified. So if none is specified and C.DEFAULT_VAULT_IDENTITY is 'default' we use the old format. ** Changes/refactors needed to implement multiple vault passwords raise exceptions on decrypt fail, check vault id early split out parsing the vault plaintext envelope (with the sha/original plaintext) to _split_plaintext_envelope() some cli fixups for specifying multiple paths in the unfrack_paths optparse callback fix py3 dict.keys() 'dict_keys object is not indexable' error pluralize cli.options.vault_password_file -> vault_password_files pluralize cli.options.new_vault_password_file -> new_vault_password_files pluralize cli.options.vault_id -> cli.options.vault_ids ** Add a config option (vault_id_match) to force vault id matching. With 'vault_id_match=True' and an ansible vault that provides a vault_id, then decryption will require that a matching vault_id is required. (via --vault-id=my_vault_id@password_file, for ex). In other words, if the config option is true, then only the vault secrets with matching vault ids are candidates for decrypting a vault. If option is false (the default), then all of the provided vault secrets will be selected. If a user doesn't want all vault secrets to be tried to decrypt any vault content, they can enable this option. Note: The vault id used for the match is not encrypted or cryptographically signed. It is just a label/id/nickname used for referencing a specific vault secret.
2017-07-28 21:20:58 +02:00
# view with old password file and new password file
ansible-vault view "$@" --vault-password-file "${NEW_VAULT_PASSWORD}" --vault-password-file vault-password "${TEST_FILE}"
# view with old password file and new password file, different order
ansible-vault view "$@" --vault-password-file vault-password --vault-password-file "${NEW_VAULT_PASSWORD}" "${TEST_FILE}"
# view with old password file and new password file and another wrong
ansible-vault view "$@" --vault-password-file "${NEW_VAULT_PASSWORD}" --vault-password-file vault-password-wrong --vault-password-file vault-password "${TEST_FILE}"
# view with old password file and new password file and another wrong, using --vault-id
ansible-vault view "$@" --vault-id "tmp_new_password@${NEW_VAULT_PASSWORD}" --vault-id wrong_password@vault-password-wrong --vault-id myorg@vault-password "${TEST_FILE}"
Add smoke/intg tests for vault cli (#21679) Add smoke/intg tests for vault cli
2017-02-24 16:30:54 +01:00
ansible-vault decrypt "$@" --vault-password-file "${NEW_VAULT_PASSWORD}" "${TEST_FILE}"
Fix vault reading from stdin (avoid realpath() on non-links) (#23583) * Fix vault reading from stdin (avoid realpath() on non-links) os.path.realpath() is used to find the target of file paths that are symlinks so vault operations happen directly on the target. However, in addition to resolving symlinks, realpath() also returns a full path. when reading from stdin, vault cli uses '-' as a special file path so VaultEditor() will replace with stdin. realpath() was expanding '-' with the CWD to something like '/home/user/playbooks/-' causing errors like: ERROR! [Errno 2] No such file or directory: u'/home/user/ansible/-' Fix is to specialcase '-' to not use realpath() Fixes #23567 * to_text decrypt output when writing to stdout
2017-04-18 19:09:02 +02:00
# reading/writing to/from stdin/stdin (See https://github.com/ansible/ansible/issues/23567)
Handle vault decrypt --output=- (#31066) In cli.CLI.unfrack_path callback, special case if the value of '--output' is '-', and avoid expanding it to a full path. vault cli already has special cases for '-', so it just needs to get the original value to work. Fixes #30550
2017-10-03 18:02:16 +02:00
ansible-vault encrypt "$@" --vault-password-file "${VAULT_PASSWORD_FILE}" --output="${TEST_FILE_OUTPUT}" < "${TEST_FILE}"
OUTPUT=$(ansible-vault decrypt "$@" --vault-password-file "${VAULT_PASSWORD_FILE}" --output=- < "${TEST_FILE_OUTPUT}")
echo "${OUTPUT}" | grep 'This is a test file'
OUTPUT_DASH=$(ansible-vault decrypt "$@" --vault-password-file "${VAULT_PASSWORD_FILE}" --output=- "${TEST_FILE_OUTPUT}")
echo "${OUTPUT_DASH}" | grep 'This is a test file'
OUTPUT_DASH_SPACE=$(ansible-vault decrypt "$@" --vault-password-file "${VAULT_PASSWORD_FILE}" --output - "${TEST_FILE_OUTPUT}")
echo "${OUTPUT_DASH_SPACE}" | grep 'This is a test file'
Fix vault reading from stdin (avoid realpath() on non-links) (#23583) * Fix vault reading from stdin (avoid realpath() on non-links) os.path.realpath() is used to find the target of file paths that are symlinks so vault operations happen directly on the target. However, in addition to resolving symlinks, realpath() also returns a full path. when reading from stdin, vault cli uses '-' as a special file path so VaultEditor() will replace with stdin. realpath() was expanding '-' with the CWD to something like '/home/user/playbooks/-' causing errors like: ERROR! [Errno 2] No such file or directory: u'/home/user/ansible/-' Fix is to specialcase '-' to not use realpath() Fixes #23567 * to_text decrypt output when writing to stdout
2017-04-18 19:09:02 +02:00
Vault secrets empty password (#28186) * Better handling of empty/invalid passwords empty password files are global error and cause an exit. A warning is also emitted with more detail. ie, if any of the password/secret sources provide a bogus password (ie, empty) or fail (exception, ctrl-d, EOFError), we stop at the first error and exit. This makes behavior when entering empty password at prompt match 2.3 (ie, an error)
2017-08-15 17:01:46 +02:00
# test using an empty vault password file
ansible-vault view "$@" --vault-password-file empty-password format_1_1_AES256.yml && :
WRONG_RC=$?
echo "rc was $WRONG_RC (1 is expected)"
[ $WRONG_RC -eq 1 ]
ansible-vault view "$@" --vault-id=empty@empty-password --vault-password-file empty-password format_1_1_AES256.yml && :
WRONG_RC=$?
echo "rc was $WRONG_RC (1 is expected)"
[ $WRONG_RC -eq 1 ]
echo 'foo' > some_file.txt
ansible-vault encrypt "$@" --vault-password-file empty-password some_file.txt && :
WRONG_RC=$?
echo "rc was $WRONG_RC (1 is expected)"
[ $WRONG_RC -eq 1 ]
Add smoke/intg tests for vault cli (#21679) Add smoke/intg tests for vault cli
2017-02-24 16:30:54 +01:00
ansible-vault encrypt_string "$@" --vault-password-file "${NEW_VAULT_PASSWORD}" "a test string"
ansible-vault encrypt_string "$@" --vault-password-file "${NEW_VAULT_PASSWORD}" --name "blippy" "a test string names blippy"
Support multiple vault passwords (#22756) Fixes #13243 ** Add --vault-id to name/identify multiple vault passwords Use --vault-id to indicate id and path/type --vault-id=prompt # prompt for default vault id password --vault-id=myorg@prompt # prompt for a vault_id named 'myorg' --vault-id=a_password_file # load ./a_password_file for default id --vault-id=myorg@a_password_file # load file for 'myorg' vault id vault_id's are created implicitly for existing --vault-password-file and --ask-vault-pass options. Vault ids are just for UX purposes and bookkeeping. Only the vault payload and the password bytestring is needed to decrypt a vault blob. Replace passing password around everywhere with a VaultSecrets object. If we specify a vault_id, mention that in password prompts Specifying multiple -vault-password-files will now try each until one works ** Rev vault format in a backwards compatible way The 1.2 vault format adds the vault_id to the header line of the vault text. This is backwards compatible with older versions of ansible. Old versions will just ignore it and treat it as the default (and only) vault id. Note: only 2.4+ supports multiple vault passwords, so while earlier ansible versions can read the vault-1.2 format, it does not make them magically support multiple vault passwords. use 1.1 format for 'default' vault_id Vaulted items that need to include a vault_id will be written in 1.2 format. If we set a new DEFAULT_VAULT_IDENTITY, then the default will use version 1.2 vault will only use a vault_id if one is specified. So if none is specified and C.DEFAULT_VAULT_IDENTITY is 'default' we use the old format. ** Changes/refactors needed to implement multiple vault passwords raise exceptions on decrypt fail, check vault id early split out parsing the vault plaintext envelope (with the sha/original plaintext) to _split_plaintext_envelope() some cli fixups for specifying multiple paths in the unfrack_paths optparse callback fix py3 dict.keys() 'dict_keys object is not indexable' error pluralize cli.options.vault_password_file -> vault_password_files pluralize cli.options.new_vault_password_file -> new_vault_password_files pluralize cli.options.vault_id -> cli.options.vault_ids ** Add a config option (vault_id_match) to force vault id matching. With 'vault_id_match=True' and an ansible vault that provides a vault_id, then decryption will require that a matching vault_id is required. (via --vault-id=my_vault_id@password_file, for ex). In other words, if the config option is true, then only the vault secrets with matching vault ids are candidates for decrypting a vault. If option is false (the default), then all of the provided vault secrets will be selected. If a user doesn't want all vault secrets to be tried to decrypt any vault content, they can enable this option. Note: The vault id used for the match is not encrypted or cryptographically signed. It is just a label/id/nickname used for referencing a specific vault secret.
2017-07-28 21:20:58 +02:00
ansible-vault encrypt_string "$@" --vault-id "${NEW_VAULT_PASSWORD}" "a test string"
ansible-vault encrypt_string "$@" --vault-id "${NEW_VAULT_PASSWORD}" --name "blippy" "a test string names blippy"
Add smoke/intg tests for vault cli (#21679) Add smoke/intg tests for vault cli
2017-02-24 16:30:54 +01:00
# from stdin
ansible-vault encrypt_string "$@" --vault-password-file "${NEW_VAULT_PASSWORD}" < "${TEST_FILE}"
ansible-vault encrypt_string "$@" --vault-password-file "${NEW_VAULT_PASSWORD}" --stdin-name "the_var_from_stdin" < "${TEST_FILE}"
# write to file
ansible-vault encrypt_string "$@" --vault-password-file "${NEW_VAULT_PASSWORD}" --name "blippy" "a test string names blippy" --output "${MYTMPDIR}/enc_string_test_file"
Use vault_id when encrypted via vault-edit (#30772) * Use vault_id when encrypted via vault-edit On the encryption stage of 'ansible-vault edit --vault-id=someid@passfile somefile', the vault id was not being passed to encrypt() so the files were always saved with the default vault id in the 1.1 version format. When trying to edit that file a second time, also with a --vault-id, the file would be decrypted with the secret associated with the provided vault-id, but since the encrypted file had no vault id in the envelope there would be no match for 'default' secrets. (Only the --vault-id was included in the potential matches, so the vault id actually used to decrypt was not). If that list was empty, there would be an IndexError when trying to encrypted the changed file. This would result in the displayed error: ERROR! Unexpected Exception, this is probably a bug: list index out of range Fix is two parts: 1) use the vault id when encrypting from edit 2) when matching the secret to use for encrypting after edit, include the vault id that was used for decryption and not just the vault id (or lack of vault id) from the envelope. add unit tests for #30575 and intg tests for 'ansible-vault edit' Fixes #30575
2017-09-26 18:28:31 +02:00
# test ansible-vault edit with a faux editor
ansible-vault encrypt "$@" --vault-password-file vault-password "${TEST_FILE_EDIT}"
# edit a 1.1 format with no vault-id, should stay 1.1
EDITOR=./faux-editor.py ansible-vault edit "$@" --vault-password-file vault-password "${TEST_FILE_EDIT}"
head -1 "${TEST_FILE_EDIT}" | grep "${FORMAT_1_1_HEADER}"
# edit a 1.1 format with vault-id, should stay 1.1
Fix 'New Vault password' on vault 'edit' (#35923) * Fix 'New Vault password' on vault 'edit' ffe0ddea96bbe8ac27af816e58667c212e74688e introduce a change on 'ansible-vault edit' that tried to check for --encrypt-vault-id in that mode. But '--encrypt-vault-id' is not intended for 'edit' since the 'edit' should always reuse the vault secret that was used to decrypt the text. Change cli to not check for --encrypt-vault-id on 'edit'. VaultLib.decrypt_and_get_vault_id() was change to return the vault secret used to decrypt (in addition to vault_id and the plaintext). VaultEditor.edit_file() will now use 'vault_secret_used' as returned from decrypt_and_get_vault_id() so that an edited file always gets reencrypted with the same secret, regardless of any vault id configuration or cli options. Fixes #35834
2018-03-27 20:12:21 +02:00
cat "${TEST_FILE_EDIT}"
Use vault_id when encrypted via vault-edit (#30772) * Use vault_id when encrypted via vault-edit On the encryption stage of 'ansible-vault edit --vault-id=someid@passfile somefile', the vault id was not being passed to encrypt() so the files were always saved with the default vault id in the 1.1 version format. When trying to edit that file a second time, also with a --vault-id, the file would be decrypted with the secret associated with the provided vault-id, but since the encrypted file had no vault id in the envelope there would be no match for 'default' secrets. (Only the --vault-id was included in the potential matches, so the vault id actually used to decrypt was not). If that list was empty, there would be an IndexError when trying to encrypted the changed file. This would result in the displayed error: ERROR! Unexpected Exception, this is probably a bug: list index out of range Fix is two parts: 1) use the vault id when encrypting from edit 2) when matching the secret to use for encrypting after edit, include the vault id that was used for decryption and not just the vault id (or lack of vault id) from the envelope. add unit tests for #30575 and intg tests for 'ansible-vault edit' Fixes #30575
2017-09-26 18:28:31 +02:00
EDITOR=./faux-editor.py ansible-vault edit "$@" --vault-id vault_password@vault-password "${TEST_FILE_EDIT}"
Fix 'New Vault password' on vault 'edit' (#35923) * Fix 'New Vault password' on vault 'edit' ffe0ddea96bbe8ac27af816e58667c212e74688e introduce a change on 'ansible-vault edit' that tried to check for --encrypt-vault-id in that mode. But '--encrypt-vault-id' is not intended for 'edit' since the 'edit' should always reuse the vault secret that was used to decrypt the text. Change cli to not check for --encrypt-vault-id on 'edit'. VaultLib.decrypt_and_get_vault_id() was change to return the vault secret used to decrypt (in addition to vault_id and the plaintext). VaultEditor.edit_file() will now use 'vault_secret_used' as returned from decrypt_and_get_vault_id() so that an edited file always gets reencrypted with the same secret, regardless of any vault id configuration or cli options. Fixes #35834
2018-03-27 20:12:21 +02:00
cat "${TEST_FILE_EDIT}"
Use vault_id when encrypted via vault-edit (#30772) * Use vault_id when encrypted via vault-edit On the encryption stage of 'ansible-vault edit --vault-id=someid@passfile somefile', the vault id was not being passed to encrypt() so the files were always saved with the default vault id in the 1.1 version format. When trying to edit that file a second time, also with a --vault-id, the file would be decrypted with the secret associated with the provided vault-id, but since the encrypted file had no vault id in the envelope there would be no match for 'default' secrets. (Only the --vault-id was included in the potential matches, so the vault id actually used to decrypt was not). If that list was empty, there would be an IndexError when trying to encrypted the changed file. This would result in the displayed error: ERROR! Unexpected Exception, this is probably a bug: list index out of range Fix is two parts: 1) use the vault id when encrypting from edit 2) when matching the secret to use for encrypting after edit, include the vault id that was used for decryption and not just the vault id (or lack of vault id) from the envelope. add unit tests for #30575 and intg tests for 'ansible-vault edit' Fixes #30575
2017-09-26 18:28:31 +02:00
head -1 "${TEST_FILE_EDIT}" | grep "${FORMAT_1_1_HEADER}"
ansible-vault encrypt "$@" --vault-id vault_password@vault-password "${TEST_FILE_EDIT2}"
Fix 'New Vault password' on vault 'edit' (#35923) * Fix 'New Vault password' on vault 'edit' ffe0ddea96bbe8ac27af816e58667c212e74688e introduce a change on 'ansible-vault edit' that tried to check for --encrypt-vault-id in that mode. But '--encrypt-vault-id' is not intended for 'edit' since the 'edit' should always reuse the vault secret that was used to decrypt the text. Change cli to not check for --encrypt-vault-id on 'edit'. VaultLib.decrypt_and_get_vault_id() was change to return the vault secret used to decrypt (in addition to vault_id and the plaintext). VaultEditor.edit_file() will now use 'vault_secret_used' as returned from decrypt_and_get_vault_id() so that an edited file always gets reencrypted with the same secret, regardless of any vault id configuration or cli options. Fixes #35834
2018-03-27 20:12:21 +02:00
# verify that we aren't prompted for a new vault password on edit if we are running interactively (ie, with prompts)
# have to use setsid nd --ask-vault-pass to force a prompt to simulate.
# See https://github.com/ansible/ansible/issues/35834
setsid sh -c 'tty; echo password |ansible-vault edit --ask-vault-pass vault_test.yml' < /dev/null > log 2>&1 && :
grep 'New Vault password' log && :
WRONG_RC=$?
echo "The stdout log had 'New Vault password' in it and it is not supposed to. rc of grep was $WRONG_RC (1 is expected)"
[ $WRONG_RC -eq 1 ]
Use vault_id when encrypted via vault-edit (#30772) * Use vault_id when encrypted via vault-edit On the encryption stage of 'ansible-vault edit --vault-id=someid@passfile somefile', the vault id was not being passed to encrypt() so the files were always saved with the default vault id in the 1.1 version format. When trying to edit that file a second time, also with a --vault-id, the file would be decrypted with the secret associated with the provided vault-id, but since the encrypted file had no vault id in the envelope there would be no match for 'default' secrets. (Only the --vault-id was included in the potential matches, so the vault id actually used to decrypt was not). If that list was empty, there would be an IndexError when trying to encrypted the changed file. This would result in the displayed error: ERROR! Unexpected Exception, this is probably a bug: list index out of range Fix is two parts: 1) use the vault id when encrypting from edit 2) when matching the secret to use for encrypting after edit, include the vault id that was used for decryption and not just the vault id (or lack of vault id) from the envelope. add unit tests for #30575 and intg tests for 'ansible-vault edit' Fixes #30575
2017-09-26 18:28:31 +02:00
# edit a 1.2 format with vault id, should keep vault id and 1.2 format
EDITOR=./faux-editor.py ansible-vault edit "$@" --vault-id vault_password@vault-password "${TEST_FILE_EDIT2}"
head -1 "${TEST_FILE_EDIT2}" | grep "${FORMAT_1_2_HEADER};vault_password"
# edit a 1.2 file with no vault-id, should keep vault id and 1.2 format
EDITOR=./faux-editor.py ansible-vault edit "$@" --vault-password-file vault-password "${TEST_FILE_EDIT2}"
head -1 "${TEST_FILE_EDIT2}" | grep "${FORMAT_1_2_HEADER};vault_password"
Support using vault password files that are themselves vault encrypted (#27668) Extract vault related bits of DataLoader._get_file_contents to DataLoader._decrypt_if_vault_data When loading vault password files, detect if they are vault encrypted, and if so, try to decrypt with any already known vault secrets. This implements the 'Allow vault password files to be vault encrypted' (#31002) feature card from the 2.5.0 project at https://github.com/ansible/ansible/projects/9 Fixes #31002
2018-01-20 20:56:18 +01:00
# encrypt with a password from a vault encrypted password file and multiple vault-ids
# should fail because we dont know which vault id to use to encrypt with
ansible-vault encrypt "$@" --vault-id vault-password --vault-id encrypted-vault-password "${TEST_FILE_ENC_PASSWORD}" && :
WRONG_RC=$?
echo "rc was $WRONG_RC (5 is expected)"
[ $WRONG_RC -eq 5 ]
# encrypt with a password from a vault encrypted password file and multiple vault-ids
# but this time specify with --encrypt-vault-id, but specifying vault-id names (instead of default)
fix decrypted vault utf8 values (#37539) * Fix errors decrypted non-ascii vault vars AnsibleVaultEncryptedUnicode was just using b"".decode() instead of to_text() on the bytestrings returned from vault.decrypt() and could cause errors on python2 if non-ascii since decode() defaults to ascii. Use to_text() to default to decoding utf-8. add intg and unit tests for value of vaulted vars being non-ascii utf8 based on https://github.com/ansible/ansible/issues/37258 Fixes #37258 * yamllint fixups
2018-05-25 05:22:46 +02:00
# ansible-vault encrypt "$@" --vault-id from_vault_password@vault-password --vault-id from_encrypted_vault_password@encrypted-vault-password --encrypt-vault-id from_encrypted_vault_password "${TEST_FILE(_ENC_PASSWORD}"
Support using vault password files that are themselves vault encrypted (#27668) Extract vault related bits of DataLoader._get_file_contents to DataLoader._decrypt_if_vault_data When loading vault password files, detect if they are vault encrypted, and if so, try to decrypt with any already known vault secrets. This implements the 'Allow vault password files to be vault encrypted' (#31002) feature card from the 2.5.0 project at https://github.com/ansible/ansible/projects/9 Fixes #31002
2018-01-20 20:56:18 +01:00
# try to view the file encrypted with the vault-password we didnt specify
# to verify we didnt choose the wrong vault-id
# ansible-vault view "$@" --vault-id vault-password "${TEST_FILE_ENC_PASSWORD}" && :
# WRONG_RC=$?
# echo "rc was $WRONG_RC (1 is expected)"
# [ $WRONG_RC -eq 1 ]
ansible-vault encrypt "$@" --vault-id vault-password "${TEST_FILE_ENC_PASSWORD}"
# view the file encrypted with a password from a vault encrypted password file
ansible-vault view "$@" --vault-id vault-password --vault-id encrypted-vault-password "${TEST_FILE_ENC_PASSWORD}"
# try to view the file encrypted with a password from a vault encrypted password file but without the password to the password file.
# This should fail with an
ansible-vault view "$@" --vault-id encrypted-vault-password "${TEST_FILE_ENC_PASSWORD}" && :
WRONG_RC=$?
echo "rc was $WRONG_RC (1 is expected)"
[ $WRONG_RC -eq 1 ]
Add smoke/intg tests for vault cli (#21679) Add smoke/intg tests for vault cli
2017-02-24 16:30:54 +01:00
# test playbooks using vaulted files
Split integration tests out from Makefile. (#17976)
2016-10-12 23:57:53 +02:00
ansible-playbook test_vault.yml -i ../../inventory -v "$@" --vault-password-file vault-password --list-tasks
ansible-playbook test_vault.yml -i ../../inventory -v "$@" --vault-password-file vault-password --list-hosts
ansible-playbook test_vault.yml -i ../../inventory -v "$@" --vault-password-file vault-password --syntax-check
ansible-playbook test_vault.yml -i ../../inventory -v "$@" --vault-password-file vault-password
ansible-playbook test_vault_embedded.yml -i ../../inventory -v "$@" --vault-password-file vault-password --syntax-check
ansible-playbook test_vault_embedded.yml -i ../../inventory -v "$@" --vault-password-file vault-password
add a intg test for vault encrypted inventory (#18550)
2016-11-22 16:56:37 +01:00
ansible-playbook test_vaulted_inventory.yml -i vaulted.inventory -v "$@" --vault-password-file vault-password
Allow template files to be vaulted (#22951) * Allow template files to be vaulted * Make sure to import exceptions we need * get_real_file can't take bytes, since it looks specifically for string_types * Now that we aren't using open() we don't need b_source * Expand playbooks_vault docs to include modules that support vaulted src files * Add vaulted template test
2017-06-07 20:16:03 +02:00
ansible-playbook test_vaulted_template.yml -i ../../inventory -v "$@" --vault-password-file vault-password
Support multiple vault passwords (#22756) Fixes #13243 ** Add --vault-id to name/identify multiple vault passwords Use --vault-id to indicate id and path/type --vault-id=prompt # prompt for default vault id password --vault-id=myorg@prompt # prompt for a vault_id named 'myorg' --vault-id=a_password_file # load ./a_password_file for default id --vault-id=myorg@a_password_file # load file for 'myorg' vault id vault_id's are created implicitly for existing --vault-password-file and --ask-vault-pass options. Vault ids are just for UX purposes and bookkeeping. Only the vault payload and the password bytestring is needed to decrypt a vault blob. Replace passing password around everywhere with a VaultSecrets object. If we specify a vault_id, mention that in password prompts Specifying multiple -vault-password-files will now try each until one works ** Rev vault format in a backwards compatible way The 1.2 vault format adds the vault_id to the header line of the vault text. This is backwards compatible with older versions of ansible. Old versions will just ignore it and treat it as the default (and only) vault id. Note: only 2.4+ supports multiple vault passwords, so while earlier ansible versions can read the vault-1.2 format, it does not make them magically support multiple vault passwords. use 1.1 format for 'default' vault_id Vaulted items that need to include a vault_id will be written in 1.2 format. If we set a new DEFAULT_VAULT_IDENTITY, then the default will use version 1.2 vault will only use a vault_id if one is specified. So if none is specified and C.DEFAULT_VAULT_IDENTITY is 'default' we use the old format. ** Changes/refactors needed to implement multiple vault passwords raise exceptions on decrypt fail, check vault id early split out parsing the vault plaintext envelope (with the sha/original plaintext) to _split_plaintext_envelope() some cli fixups for specifying multiple paths in the unfrack_paths optparse callback fix py3 dict.keys() 'dict_keys object is not indexable' error pluralize cli.options.vault_password_file -> vault_password_files pluralize cli.options.new_vault_password_file -> new_vault_password_files pluralize cli.options.vault_id -> cli.options.vault_ids ** Add a config option (vault_id_match) to force vault id matching. With 'vault_id_match=True' and an ansible vault that provides a vault_id, then decryption will require that a matching vault_id is required. (via --vault-id=my_vault_id@password_file, for ex). In other words, if the config option is true, then only the vault secrets with matching vault ids are candidates for decrypting a vault. If option is false (the default), then all of the provided vault secrets will be selected. If a user doesn't want all vault secrets to be tried to decrypt any vault content, they can enable this option. Note: The vault id used for the match is not encrypted or cryptographically signed. It is just a label/id/nickname used for referencing a specific vault secret.
2017-07-28 21:20:58 +02:00
fix decrypted vault utf8 values (#37539) * Fix errors decrypted non-ascii vault vars AnsibleVaultEncryptedUnicode was just using b"".decode() instead of to_text() on the bytestrings returned from vault.decrypt() and could cause errors on python2 if non-ascii since decode() defaults to ascii. Use to_text() to default to decoding utf-8. add intg and unit tests for value of vaulted vars being non-ascii utf8 based on https://github.com/ansible/ansible/issues/37258 Fixes #37258 * yamllint fixups
2018-05-25 05:22:46 +02:00
# test a playbook with a host_var whose value is non-ascii utf8 (see https://github.com/ansible/ansible/issues/37258)
ansible-playbook -i ../../inventory -v "$@" --vault-id vault-password test_vaulted_utf8_value.yml
Support multiple vault passwords (#22756) Fixes #13243 ** Add --vault-id to name/identify multiple vault passwords Use --vault-id to indicate id and path/type --vault-id=prompt # prompt for default vault id password --vault-id=myorg@prompt # prompt for a vault_id named 'myorg' --vault-id=a_password_file # load ./a_password_file for default id --vault-id=myorg@a_password_file # load file for 'myorg' vault id vault_id's are created implicitly for existing --vault-password-file and --ask-vault-pass options. Vault ids are just for UX purposes and bookkeeping. Only the vault payload and the password bytestring is needed to decrypt a vault blob. Replace passing password around everywhere with a VaultSecrets object. If we specify a vault_id, mention that in password prompts Specifying multiple -vault-password-files will now try each until one works ** Rev vault format in a backwards compatible way The 1.2 vault format adds the vault_id to the header line of the vault text. This is backwards compatible with older versions of ansible. Old versions will just ignore it and treat it as the default (and only) vault id. Note: only 2.4+ supports multiple vault passwords, so while earlier ansible versions can read the vault-1.2 format, it does not make them magically support multiple vault passwords. use 1.1 format for 'default' vault_id Vaulted items that need to include a vault_id will be written in 1.2 format. If we set a new DEFAULT_VAULT_IDENTITY, then the default will use version 1.2 vault will only use a vault_id if one is specified. So if none is specified and C.DEFAULT_VAULT_IDENTITY is 'default' we use the old format. ** Changes/refactors needed to implement multiple vault passwords raise exceptions on decrypt fail, check vault id early split out parsing the vault plaintext envelope (with the sha/original plaintext) to _split_plaintext_envelope() some cli fixups for specifying multiple paths in the unfrack_paths optparse callback fix py3 dict.keys() 'dict_keys object is not indexable' error pluralize cli.options.vault_password_file -> vault_password_files pluralize cli.options.new_vault_password_file -> new_vault_password_files pluralize cli.options.vault_id -> cli.options.vault_ids ** Add a config option (vault_id_match) to force vault id matching. With 'vault_id_match=True' and an ansible vault that provides a vault_id, then decryption will require that a matching vault_id is required. (via --vault-id=my_vault_id@password_file, for ex). In other words, if the config option is true, then only the vault secrets with matching vault ids are candidates for decrypting a vault. If option is false (the default), then all of the provided vault secrets will be selected. If a user doesn't want all vault secrets to be tried to decrypt any vault content, they can enable this option. Note: The vault id used for the match is not encrypted or cryptographically signed. It is just a label/id/nickname used for referencing a specific vault secret.
2017-07-28 21:20:58 +02:00
# test with password from password script
ansible-playbook test_vault.yml -i ../../inventory -v "$@" --vault-password-file password-script.py
ansible-playbook test_vault_embedded.yml -i ../../inventory -v "$@" --vault-password-file password-script.py
# with multiple password files
ansible-playbook test_vault.yml -i ../../inventory -v "$@" --vault-password-file vault-password --vault-password-file vault-password-wrong
ansible-playbook test_vault.yml -i ../../inventory -v "$@" --vault-password-file vault-password-wrong --vault-password-file vault-password
ansible-playbook test_vault_embedded.yml -i ../../inventory -v "$@" --vault-password-file vault-password --vault-password-file vault-password-wrong --syntax-check
ansible-playbook test_vault_embedded.yml -i ../../inventory -v "$@" --vault-password-file vault-password-wrong --vault-password-file vault-password
Add back support for vault_password_file config var (#27597) Got removed in arg parsing updates. Now added back in setup_vault_secrets(). The default value for DEFAULT_VAULT_PASSWORD_FILE was also set to '~' for some reason, change to to no default. Add integration tests.
2017-08-02 00:07:33 +02:00
# test with a default vault password file set in config
ANSIBLE_VAULT_PASSWORD_FILE=vault-password ansible-playbook test_vault_embedded.yml -i ../../inventory -v "$@" --vault-password-file vault-password-wrong
Fix config value type for VAULT_IDENTITY_LIST (#28678) Was using the 'value_type' key, but didn't get updated to the new 'type' key in merge. Fix playbooks cli so it uses VAULT_IDENTITY_LIST as well.
2017-08-28 16:13:14 +02:00
# test using vault_identity_list config
ANSIBLE_VAULT_IDENTITY_LIST='wrong-password@vault-password-wrong,default@vault-password' ansible-playbook test_vault.yml -i ../../inventory -v "$@"
Support multiple vault passwords (#22756) Fixes #13243 ** Add --vault-id to name/identify multiple vault passwords Use --vault-id to indicate id and path/type --vault-id=prompt # prompt for default vault id password --vault-id=myorg@prompt # prompt for a vault_id named 'myorg' --vault-id=a_password_file # load ./a_password_file for default id --vault-id=myorg@a_password_file # load file for 'myorg' vault id vault_id's are created implicitly for existing --vault-password-file and --ask-vault-pass options. Vault ids are just for UX purposes and bookkeeping. Only the vault payload and the password bytestring is needed to decrypt a vault blob. Replace passing password around everywhere with a VaultSecrets object. If we specify a vault_id, mention that in password prompts Specifying multiple -vault-password-files will now try each until one works ** Rev vault format in a backwards compatible way The 1.2 vault format adds the vault_id to the header line of the vault text. This is backwards compatible with older versions of ansible. Old versions will just ignore it and treat it as the default (and only) vault id. Note: only 2.4+ supports multiple vault passwords, so while earlier ansible versions can read the vault-1.2 format, it does not make them magically support multiple vault passwords. use 1.1 format for 'default' vault_id Vaulted items that need to include a vault_id will be written in 1.2 format. If we set a new DEFAULT_VAULT_IDENTITY, then the default will use version 1.2 vault will only use a vault_id if one is specified. So if none is specified and C.DEFAULT_VAULT_IDENTITY is 'default' we use the old format. ** Changes/refactors needed to implement multiple vault passwords raise exceptions on decrypt fail, check vault id early split out parsing the vault plaintext envelope (with the sha/original plaintext) to _split_plaintext_envelope() some cli fixups for specifying multiple paths in the unfrack_paths optparse callback fix py3 dict.keys() 'dict_keys object is not indexable' error pluralize cli.options.vault_password_file -> vault_password_files pluralize cli.options.new_vault_password_file -> new_vault_password_files pluralize cli.options.vault_id -> cli.options.vault_ids ** Add a config option (vault_id_match) to force vault id matching. With 'vault_id_match=True' and an ansible vault that provides a vault_id, then decryption will require that a matching vault_id is required. (via --vault-id=my_vault_id@password_file, for ex). In other words, if the config option is true, then only the vault secrets with matching vault ids are candidates for decrypting a vault. If option is false (the default), then all of the provided vault secrets will be selected. If a user doesn't want all vault secrets to be tried to decrypt any vault content, they can enable this option. Note: The vault id used for the match is not encrypted or cryptographically signed. It is just a label/id/nickname used for referencing a specific vault secret.
2017-07-28 21:20:58 +02:00
# test that we can have a vault encrypted yaml file that includes embedded vault vars
# that were encrypted with a different vault secret
ansible-playbook test_vault_file_encrypted_embedded.yml -i ../../inventory "$@" --vault-id encrypted_file_encrypted_var_password --vault-id vault-password
# with multiple password files, --vault-id, ordering
ansible-playbook test_vault.yml -i ../../inventory -v "$@" --vault-id vault-password --vault-id vault-password-wrong
ansible-playbook test_vault.yml -i ../../inventory -v "$@" --vault-id vault-password-wrong --vault-id vault-password
ansible-playbook test_vault_embedded.yml -i ../../inventory -v "$@" --vault-id vault-password --vault-id vault-password-wrong --syntax-check
ansible-playbook test_vault_embedded.yml -i ../../inventory -v "$@" --vault-id vault-password-wrong --vault-id vault-password
# test with multiple password files, including a script, and a wrong password
ansible-playbook test_vault_embedded.yml -i ../../inventory -v "$@" --vault-password-file vault-password-wrong --vault-password-file password-script.py --vault-password-file vault-password
# test with multiple password files, including a script, and a wrong password, and a mix of --vault-id and --vault-password-file
ansible-playbook test_vault_embedded.yml -i ../../inventory -v "$@" --vault-password-file vault-password-wrong --vault-id password-script.py --vault-id vault-password
# test with multiple password files, including a script, and a wrong password, and a mix of --vault-id and --vault-password-file
ansible-playbook test_vault_embedded_ids.yml -i ../../inventory -v "$@" \
--vault-password-file vault-password-wrong \
--vault-id password-script.py --vault-id example1@example1_password \
--vault-id example2@example2_password --vault-password-file example3_password \
--vault-id vault-password
# with wrong password
ansible-playbook test_vault.yml -i ../../inventory -v "$@" --vault-password-file vault-password-wrong && :
WRONG_RC=$?
echo "rc was $WRONG_RC (1 is expected)"
[ $WRONG_RC -eq 1 ]
# with multiple wrong passwords
ansible-playbook test_vault.yml -i ../../inventory -v "$@" --vault-password-file vault-password-wrong --vault-password-file vault-password-wrong && :
WRONG_RC=$?
echo "rc was $WRONG_RC (1 is expected)"
[ $WRONG_RC -eq 1 ]
# with wrong password, --vault-id
ansible-playbook test_vault.yml -i ../../inventory -v "$@" --vault-id vault-password-wrong && :
WRONG_RC=$?
echo "rc was $WRONG_RC (1 is expected)"
[ $WRONG_RC -eq 1 ]
# with multiple wrong passwords with --vault-id
ansible-playbook test_vault.yml -i ../../inventory -v "$@" --vault-id vault-password-wrong --vault-id vault-password-wrong && :
WRONG_RC=$?
echo "rc was $WRONG_RC (1 is expected)"
[ $WRONG_RC -eq 1 ]
# with multiple wrong passwords with --vault-id
ansible-playbook test_vault.yml -i ../../inventory -v "$@" --vault-id wrong1@vault-password-wrong --vault-id wrong2@vault-password-wrong && :
WRONG_RC=$?
echo "rc was $WRONG_RC (1 is expected)"
[ $WRONG_RC -eq 1 ]
Vault secrets empty password (#28186) * Better handling of empty/invalid passwords empty password files are global error and cause an exit. A warning is also emitted with more detail. ie, if any of the password/secret sources provide a bogus password (ie, empty) or fail (exception, ctrl-d, EOFError), we stop at the first error and exit. This makes behavior when entering empty password at prompt match 2.3 (ie, an error)
2017-08-15 17:01:46 +02:00
# with empty password file
ansible-playbook test_vault.yml -i ../../inventory -v "$@" --vault-id empty@empty-password && :
WRONG_RC=$?
echo "rc was $WRONG_RC (1 is expected)"
[ $WRONG_RC -eq 1 ]
Better handling of malformed vault data envelope (#32515) * Better handling of malformed vault data envelope If an embedded vaulted variable ('!vault' in yaml) had an invalid format, it would eventually cause an error for seemingly unrelated reasons. "Invalid" meaning not valid hexlify (extra chars, non-hex chars, etc). For ex, if a host_vars file had invalid vault format variables, on py2, it would cause an error like: 'ansible.vars.hostvars.HostVars object' has no attribute u'broken.example.com' Depending on where the invalid vault is, it could also cause "VARIABLE IS NOT DEFINED!". The behavior can also change if ansible-playbook is py2 or py3. Root cause is errors from binascii.unhexlify() not being handled consistently. Fix is to add a AnsibleVaultFormatError exception and raise it on any unhexlify() errors and to handle it properly elsewhere. Add a _unhexlify() that try/excepts around a binascii.unhexlify() and raises an AnsibleVaultFormatError on invalid vault data. This is so the same exception type is always raised for this case. Previous it was different between py2 and py3. binascii.unhexlify() raises a binascii.Error if the hexlified blobs in a vault data blob are invalid. On py2, binascii.Error is a subclass of Exception. On py3, binascii.Error is a subclass of TypeError When decrypting content of vault encrypted variables, if a binascii.Error is raised it propagates up to playbook.base.Base.post_validate(). post_validate() handles exceptions for TypeErrors but not for base Exception subclasses (like py2 binascii.Error). * Add a display.warning on vault format errors * Unit tests for _unhexlify, parse_vaulttext* * Add intg test cases for invalid vault formats Fixes #28038
2017-11-10 20:24:56 +01:00
# test invalid format ala https://github.com/ansible/ansible/issues/28038
EXPECTED_ERROR='Vault format unhexlify error: Non-hexadecimal digit found'
ansible-playbook "$@" -i invalid_format/inventory --vault-id invalid_format/vault-secret invalid_format/broken-host-vars-tasks.yml 2>&1 | grep "${EXPECTED_ERROR}"
EXPECTED_ERROR='Vault format unhexlify error: Odd-length string'
ansible-playbook "$@" -i invalid_format/inventory --vault-id invalid_format/vault-secret invalid_format/broken-group-vars-tasks.yml 2>&1 | grep "${EXPECTED_ERROR}"
Copy permalink