This adds a new type of vault-password script (a 'client') that takes advantage of and enhances the
multiple vault password support.
If a vault password script basename ends with the name '-client', consider it a vault password script client.
A vault password script 'client' just means that the script will take a '--vault-id' command line arg.
The previous vault password script (as invoked by --vault-password-file pointing to an executable) takes
no args and returns the password on stdout. But it doesnt know anything about --vault-id or multiple vault
passwords.
The new 'protocol' of the vault password script takes a cli arg ('--vault-id') so that it can lookup that specific
vault-id and return it's password.
Since existing vault password scripts don't know the new 'protocol', a way to distinguish password scripts
that do understand the protocol was needed. The convention now is to consider password scripts that are
named like 'something-client.py' (and executable) to be vault password client scripts.
The new client scripts get invoked with the '--vault-id' they were requested for. An example:
ansible-playbook --vault-id my_vault_id@contrib/vault/vault-keyring-client.py some_playbook.yml
That will cause the 'contrib/vault/vault-keyring-client.py' script to be invoked as:
contrib/vault/vault-keyring-client.py --vault-id my_vault_id
The previous vault-keyring.py password script was extended to become vault-keyring-client.py. It uses
the python 'keyring' module to request secrets from various backends. The plain 'vault-keyring.py' script
would determine which key id and keyring name to use based on values that had to be set in ansible.cfg.
So it was also limited to one keyring name.
The new vault-keyring-client.py will request the secret for the vault id provided via the '--vault-id' option.
The script can be used without config and can be used for multiple keyring ids (and keyrings).
On success, a vault password client script will print the password to stdout and exit with a return code of 0.
If the 'client' script can't find a secret for the --vault-id, the script will exit with return code of 2 and print an error to stderr.
* Set missing default values for EC2 inventory
* Make it run even with no ec2.ini file
* Fixing INI file reading
* Refactor how defaults are handeled
Define defaults in a dictionary and use .get rather than if statements with has_option
* Removing double keys and fixing logic for instance_filter
* Removing one more doubled key
* Allow filtering RDS instances by tags in the ec2.py dynamic inventory script
* PEP8 fix
* Fix no-bastring code smell
* Simplify logic in ec2.py RDS filtering by tag
* Update docker inventory to use APIClient
docker-py has been updated, and the `Client` class no longer
exists. We use the new `APIClient` class.
To provide graceful failure when docker-py is not installed,
we need to create a dummy `Client` class so that the inventory
script will get as far as displaying a useful error message
Before
```
$ contrib/inventory/docker.py --pretty
Traceback (most recent call last):
File "contrib/inventory/docker.py", line 418, in <module>
class AnsibleDockerClient(Client):
NameError: name 'Client' is not defined
```
After
```
$ contrib/inventory/docker.py --pretty
Failed to import docker-py. Try `pip install docker-py` - cannot import name Client
```
* docker inventory configuration file location
Allow docker.yml to live next to docker.py, as well as in the
current directory
Ansible documentation states that env variable based authentication bases on variable GCE_CREDENTIALS_FILE_PATH while gce.py reads only GCE_PEM_FILE_PATH (see https://docs.ansible.com/ansible/guide_gce.html). This commit adds GCE_CREDENTIALS_FILE_PATH to the configuration chain; if set it will be used.
* Can be set via env, credential profile, or module arg
* Valid values defined by Azure Python SDK, currently `AzureCloud`,`AzureChinaCloud`,`AzureUSGovernment`,`AzureGermanCloud` or any Azure Stack metadata discovery URL.
* Fixup the property collection for dictionaries vs. objects
* Remove debug lines
* Do not attempt to sort because it's a waste
* Remove unused code
* Remove extra code
* Capture lowercase keys
* Fix 'the the' typos, fix 'pahting' filename typo
* Change 'the the' typos to a single 'the'.
* Change `playbook_pahting.rst` to `playbook_pathing.rst`.
* Delete trailing space in ec2_vol example
Delete the trailing space in `instance: "{{ item.id }} "`, which makes the
example fail when run because it looks for instance "i-xxxx ".
* Compatibility of gce.py (inventory) with Python 3
* Revert './secrets.py' file check (will import 'secrets' from PYTHONPATH)
Instead of checking if secrets.py exists in the current directory, this
commit will make gce import 'secrets' from one of PYTHONPATH's paths.
There are 2 possibilities:
1. secrets.py will be used if secrets.GCE_PARAMS and
secrets.GCE_KEYWORD_PARAMS are declared.
2. secrets.py will be ignored if secrets.GCE_PARAMS and
secrets.GCE_KEYWORD_PARAMS aren't declared. This could happen in Python
>=3.6 where a module named 'secrets' could be imported if a custom
secrets.py doesn't exist in PYTHONPATH.
Check out https://www.python.org/dev/peps/pep-0506/ and
https://docs.python.org/3/library/secrets.html for more information.
Consolidate the module_utils, constants, and config functions that
convert values into booleans into a single function in module_utils.
Port code to use the module_utils.validate.convert_bool.boolean function
isntead of mk_boolean.
Make pyca/cryptography the preferred backend for cryptographic needs (mainly vault) falling back to pycrypto
pyca/cryptography is already implicitly a dependency in many cases
through paramiko (2.0+) as well as the new openssl_publickey module,
which requires pyOpenSSL 16.0+. Additionally, pyca/cryptography is
an optional dep for better performance with vault already.
This commit leverages cryptography's padding, constant time comparisons,
and CBC/CTR modes to reduce the amount of code ansible needs to
maintain.
* Handle wrong password given for VaultAES format
* Do not display deprecation warning for cryptography on python-2.6
* Namespace all of the pycrypto imports and always import them
Makes unittests better and the code less likely to get stupid mistakes
(like using HMAC from cryptogrpahy when the one from pycrypto is needed)
* Add back in atfork since we need pycrypto to reinitialize its RNG just in case we're being used with old paramiko
* contrib/inventory/gce: Remove spurious require on pycrypto
(cherry picked from commit 9e16b9db275263b3ea8d1b124966fdebfc9ab271)
* Add cryptography to ec2_win_password module requirements
* Fix python3 bug which would pass text strings to a function which
requires byte strings.
* Attempt to add pycrypto version to setup deps
* Change hacking README for dual pycrypto/cryptography
* update dependencies for various CI scripts
* additional CI dockerfile/script updates
* add paramiko to the windows and sanity requirement set
This is needed because ansible lists it as a requirement. Previously
the missing dep wasn't enforced, but cryptography imports pkg_resources
so you can't ignore a requirement any more
* Add integration test cases for old vault and for wrong passwords
* helper script for manual testing of pycrypto/cryptography
* Skip the pycrypto tests so that users without it installed can still run the unittests
* Run unittests for vault with both cryptography and pycrypto backend
* create Ansible groups from host collections
* fix paging logic in _get_json
* replace Satellite with Foreman
* improve comment for environment variables
* enable host collections by default
* use requests params instead of urllib.quote_plus
* disable host collections by default
* change organization filter
* clean up load_cache methods
* remove usage of function defaults
* replace environment variables with host_filters
OpenStack dynamic inventory is still using the deprecated
ansible_ssh_host. This patch adds ansible_host until such
time as ansible_ssh_host is removed
* Do not ask for templates in Brook inventory if they do not exist
* Add availability zone variable to Brook inventory
* Add provider id variable to Brook inventory
* Error output sent to stderr in Brook inventory
* A basic start on an lxd dynamic inventory script
The script is a fairly basic start on an lxd dynamic inventory script. Only tested on ubuntu 16.04 and currently only reports ipv4 container addresses. Improvements to come.
* Updated formatting, indenting and python3 support
Updated to work on python3
* Additional options read from the ini file
Added options for connection and group to the ini file and modified to use these.
Host returned is now the name by default. For non-lxd connection, the ansible_host var is populated with the ip address.
* Additional options for group and connection
* Minor change to re-run tests
* Update module_utils.six to latest
We've been held back on the version of six we could use on the module
side to 1.4.x because of python-2.4 compatibility. Now that our minimum
is Python-2.6, we can update to the latest version of six in
module_utils and get rid of the second copy in lib/ansible/compat.
When specifying a literal whitelist of AWS EC2 regions in the dynamic
inventory configuration file, it should not be necessary to also include
a literal blacklist, especially as the blacklist is not honored in this
case anyway. By reading the literal blacklist only when necessary, it is
possible for a user to provide a more minimal EC2 configuration file.
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>
* EC2 inventory can now connect using an IAM role
* Fix comment indentation
* Make sure that Ec2Inventory.iam_role is always defined
* Add missing import
* Add ads server itself as an host in the inventory
* Comment all value in example
* Add Id in variable list per device
* Centralize code to add device status to variables
* Fix device variable name for blueprint
* Add Nagios livestatus inventory plugin.
* Add new capabilities for the nagios_livestatus inventory:
- host_field: set the name returned (default: 'name')
- group_field: set the field used for group (default: 'groups')
- host_filter: filter host using this filter (default: None)
To be more consistent, prefix was renamed into var_prefix.
* Fix py34 runtests errors against print call.
With this proposed PR, we want to make the use of many ec2 dynamic inventory files more flexible.
We are using multiple AWS accounts. We want to use different ini file (one for every account) and only one ec2.py.
* Use with statement when doing rw on files
* Deserialize file-like object directly instead of a string
For python 2/3 compatibility reasons, per PR feedback.
* adding route53_hostnames option to set the hostnames from route 53
* checking whether the route53_hostnames option is present as suggested by @s-hertel
* setting route53_hostnames to None when config option not present
* skip the to_safe only when using route53_hostnames option, as suggested by @ryansb
* skipping the to_safe strip only for the hostnames that came from route53 as suggested by @ryansb
Allows users to specify a key name in a given project’s `ansible.cfg`
file and thus handle keyring integration with vaults with different
passwords. If no key name is specified, the original default `ansible`
key name will be used.
Other improvements:
* `username` is now optional; defaults to user that invokes the script
* change string interpolation to new `.format()` style
* clean up and expand upon documentation
* enforce PEP 8 compliance
* Add dynamic inventory script and config for Packet.net
* The script and config have been shamelessly cargo
culted from the `ec2.py` and `ec2.ini` dynamic inventory
script.
* This is an initial version and could very well be
enhanced and made better.
Examples:
`PACKET_NET_API_KEY=<MY_AUTH_TOKEN> --list` to get inventory for
all hosts in Packet.net in all projects (defaults to `--list`
if no argument provided).
`PACKET_NET_API_KEY=<MY_AUTH_TOKEN> --host HOST` to get variables
for a single host.
* improvements in Packet host dynamic inventory
Ensure command line profile argument and AWS_PROFILE environment variable
overrides config file
Remove unnecessary `lambda` function
Fix cache file path construction to be more pythonic (and windows-ready)
use inkey attribute in _process_object_types recursive loop to generate key name in skip_keys directive.
This permit to ignore nested variables, for example summary.vm to optimize inventory collect
* vmware_inventory: permit to group by custom field
This permits to create instances, affect some custom fields like EC2 tags and then retrieve groups from custom fields like EC2 inventory
* vmware_inventory: Customize skip_keys & add resourceconfig to skip_keys
Verify if customfield is a str before processing custom fields for a host
* Fix many points reported by PyCharm as PEP 8 code style
* Improve inventory performance by dropping vim.HostSystem & vim.VirtualMachine collect when depth >= 2
* Declare some class variables properly
* Remove some unused variables
* Add documentation in vmware_inventory.ini for VMWARE_USERNAME & VMWARE_PASSWORD env vars
This forces basic auth to be used. Using the normal HTTPPasswordMgrWithDefaultRealm
password manager from urllib2 fails since collins doesn't send a 401 retry on failure.
More about this can be seen here http://stackoverflow.com/questions/2407126/python-urllib2-basic-auth-problem.
I added a small comment about the format of the host so others don't waste time like i did.
