This fix adds support for multiple jinja filters for groupby_patterns
and host_filters.
Fixes: #37961
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Set `raw=True` when reading passwords from ConfigParser files.
The assumption is that no one is actually doing interpolation on the
password value, even if they may for other configuration values, and
passwords are far more likely to contain '%'.
* Add vmware_inventory as well.
Workaround for https://pagure.io/freeipa/issue/7345.
mentions:
- IPA_CONFDIR environment variable when directory doesn't exist
- domain, xmlrpc_uri or jsonrpc_uri parameters when one is missing
Exception was:
$ IPA_CONFDIR=/path/to/empty/dir ./ansible/contrib/inventory/freeipa.py --list
Traceback (most recent call last):
File "./ansible/contrib/inventory/freeipa.py", line 95, in <module>
api = initialize()
File "./ansible/contrib/inventory/freeipa.py", line 18, in initialize
api.finalize()
File "local/lib/python2.7/site-packages/ipalib/plugable.py", line 738, in finalize
self.__do_if_not_done('load_plugins')
File "local/lib/python2.7/site-packages/ipalib/plugable.py", line 425, in __do_if_not_done
getattr(self, name)()
File "local/lib/python2.7/site-packages/ipalib/plugable.py", line 618, in load_plugins
for package in self.packages:
File "local/lib/python2.7/site-packages/ipalib/__init__.py", line 949, in packages
ipaclient.remote_plugins.get_package(self),
File "local/lib/python2.7/site-packages/ipaclient/remote_plugins/__init__.py", line 120, in get_package
server_info = ServerInfo(api)
File "local/lib/python2.7/site-packages/ipaclient/remote_plugins/__init__.py", line 26, in __init__
hostname = DNSName(api.env.server).ToASCII()
AttributeError: 'Env' object has no attribute 'server'
Start using this to construct shade OpenStack Cloud objects in a
consistent manner. This will let us centralize things like dealing with
password arguments and whatnot. It also allows us to introduce the
ability to pass a fully formed config dict directly to the module.
Migrate all OpenStack modules to use openstack_cloud_from_module.
Have it return the shade library since it's responsible for
importing shade and shade is needed for the exceptions.
Only pull specific OpenStack arguments for the constructor
Rather than passing **module.params to the shade constructor, pull out
only the values that make sense. This should prevent the issues with
module parameters stepping on shade parameters.
Replace module.params.pop with module.params.get
We don't need to pop these anymore since the shade constructor is now
using opt-in values.
Using real urls is ungood. Use example.com domains. Also, get rid of the
antiquated port numbers.
In order to authenticate with the Scaleway API for the contrib dynamic
inventory Scaleway the environment variable SCALEWAY_TOKEN was read
only if a value was set in the configuration .ini file.
This PR fixes that allowing to read only the environment variable
without needing to set a token in the configuration file.
Closes#35693
Currently Cloudforms can return a mix of IPv4 and IPv6 addresses in the
ipaddresses field and this mix comes in a "random" order (that is the
first entry may be IPv4 sometimes but IPv6 other times). If you wish to
always use IPv4 for the ansible_ssh_host value then this is problematic.
This change adds a new prefer_ipv4 flag which will look for the first
IPv4 address in the ipaddresses list and uses that instead of just the
first entry.
* refactors nios api shared code to handle provider better
This change refactors the shared code to be easily shared between
modules, plugins and dynamic inventory scripts. All parts now implement
the provider arguments uniformly.
This also provides a centralized fix to suppress urllib3 warnings coming
from the requests library implemented by infoblox_client
* fix up pep8 errors
* fix missing var name
* Add ability to append suffix to host names for Cloudforms Inventory
Allows for use of a suffix that will be appended to host names returned
from Cloudforms API if that suffix is not present.
For example with a suffix of 'example.org', the following results
would be shown for a particular Cloudforms host name:
someexample -> someexample.example.org
someexample.example.org -> someexample.example.org
The main use-case for this is when Cloudforms has short names rather
than FQDN and there is a desire to to use the FQDN as the name.
* Add example line into Cloudforms INI file
* Clarify that leading fullstop needed
* Add validation to ensure leading fullstop for suffix
provider
Provides a dynamic inventory plugin for Scaleway cloud provider with
the following features:
- Configurable scaleway.ini file
- Cache API responses
- Choose public or private IPs
- Create groups per Scaleway 'tags'
- Create groups per Scaleway regions
Fix typo. The use of `dest` for group_by_aws_account causes the group to be produced using IP and to duplicate the hosts in the inventory.
Fixes#23772
Signed-off-by: bo <robert@jbanetwork.com>
This fix adds replacement for exit() to sys.exit(), as
exit() is not recommended way to exit from the program.
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
Fix adds default 'vmware' section in configuration,
when this section is not found.
Fixes: #31549
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
This fix changes default value of host_filter in vmware_inventory.py
and vmware_inventory.ini to 'runtime.powerstate' from 'guest.gueststate'.
This change is added as 'guest.gueststate' requires vmware tools to be
installed on given VM, which may not be the case everytime.
Fixes: #25086
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
This patch add new environment variables to oVirt dynamic inventory
to be consistent with all other oVirt modules:
OVIRT_URL
OVIRT_USERNAME
OVIRT_CAFILE
OVIRT_PASSWORD
Those variables are used as fallback if user don't specify a ini file,
with appropriate variables there.
This adds a new type of vault-password script (a 'client') that takes advantage of and enhances the
multiple vault password support.
If a vault password script basename ends with the name '-client', consider it a vault password script client.
A vault password script 'client' just means that the script will take a '--vault-id' command line arg.
The previous vault password script (as invoked by --vault-password-file pointing to an executable) takes
no args and returns the password on stdout. But it doesnt know anything about --vault-id or multiple vault
passwords.
The new 'protocol' of the vault password script takes a cli arg ('--vault-id') so that it can lookup that specific
vault-id and return it's password.
Since existing vault password scripts don't know the new 'protocol', a way to distinguish password scripts
that do understand the protocol was needed. The convention now is to consider password scripts that are
named like 'something-client.py' (and executable) to be vault password client scripts.
The new client scripts get invoked with the '--vault-id' they were requested for. An example:
ansible-playbook --vault-id my_vault_id@contrib/vault/vault-keyring-client.py some_playbook.yml
That will cause the 'contrib/vault/vault-keyring-client.py' script to be invoked as:
contrib/vault/vault-keyring-client.py --vault-id my_vault_id
The previous vault-keyring.py password script was extended to become vault-keyring-client.py. It uses
the python 'keyring' module to request secrets from various backends. The plain 'vault-keyring.py' script
would determine which key id and keyring name to use based on values that had to be set in ansible.cfg.
So it was also limited to one keyring name.
The new vault-keyring-client.py will request the secret for the vault id provided via the '--vault-id' option.
The script can be used without config and can be used for multiple keyring ids (and keyrings).
On success, a vault password client script will print the password to stdout and exit with a return code of 0.
If the 'client' script can't find a secret for the --vault-id, the script will exit with return code of 2 and print an error to stderr.
* Set missing default values for EC2 inventory
* Make it run even with no ec2.ini file
* Fixing INI file reading
* Refactor how defaults are handeled
Define defaults in a dictionary and use .get rather than if statements with has_option
* Removing double keys and fixing logic for instance_filter
* Removing one more doubled key
* Allow filtering RDS instances by tags in the ec2.py dynamic inventory script
* PEP8 fix
* Fix no-bastring code smell
* Simplify logic in ec2.py RDS filtering by tag
* Update docker inventory to use APIClient
docker-py has been updated, and the `Client` class no longer
exists. We use the new `APIClient` class.
To provide graceful failure when docker-py is not installed,
we need to create a dummy `Client` class so that the inventory
script will get as far as displaying a useful error message
Before
```
$ contrib/inventory/docker.py --pretty
Traceback (most recent call last):
File "contrib/inventory/docker.py", line 418, in <module>
class AnsibleDockerClient(Client):
NameError: name 'Client' is not defined
```
After
```
$ contrib/inventory/docker.py --pretty
Failed to import docker-py. Try `pip install docker-py` - cannot import name Client
```
* docker inventory configuration file location
Allow docker.yml to live next to docker.py, as well as in the
current directory
Ansible documentation states that env variable based authentication bases on variable GCE_CREDENTIALS_FILE_PATH while gce.py reads only GCE_PEM_FILE_PATH (see https://docs.ansible.com/ansible/guide_gce.html). This commit adds GCE_CREDENTIALS_FILE_PATH to the configuration chain; if set it will be used.
* Can be set via env, credential profile, or module arg
* Valid values defined by Azure Python SDK, currently `AzureCloud`,`AzureChinaCloud`,`AzureUSGovernment`,`AzureGermanCloud` or any Azure Stack metadata discovery URL.
* Fixup the property collection for dictionaries vs. objects
* Remove debug lines
* Do not attempt to sort because it's a waste
* Remove unused code
* Remove extra code
* Capture lowercase keys
* Fix 'the the' typos, fix 'pahting' filename typo
* Change 'the the' typos to a single 'the'.
* Change `playbook_pahting.rst` to `playbook_pathing.rst`.
* Delete trailing space in ec2_vol example
Delete the trailing space in `instance: "{{ item.id }} "`, which makes the
example fail when run because it looks for instance "i-xxxx ".
* Compatibility of gce.py (inventory) with Python 3
* Revert './secrets.py' file check (will import 'secrets' from PYTHONPATH)
Instead of checking if secrets.py exists in the current directory, this
commit will make gce import 'secrets' from one of PYTHONPATH's paths.
There are 2 possibilities:
1. secrets.py will be used if secrets.GCE_PARAMS and
secrets.GCE_KEYWORD_PARAMS are declared.
2. secrets.py will be ignored if secrets.GCE_PARAMS and
secrets.GCE_KEYWORD_PARAMS aren't declared. This could happen in Python
>=3.6 where a module named 'secrets' could be imported if a custom
secrets.py doesn't exist in PYTHONPATH.
Check out https://www.python.org/dev/peps/pep-0506/ and
https://docs.python.org/3/library/secrets.html for more information.
Consolidate the module_utils, constants, and config functions that
convert values into booleans into a single function in module_utils.
Port code to use the module_utils.validate.convert_bool.boolean function
isntead of mk_boolean.
Make pyca/cryptography the preferred backend for cryptographic needs (mainly vault) falling back to pycrypto
pyca/cryptography is already implicitly a dependency in many cases
through paramiko (2.0+) as well as the new openssl_publickey module,
which requires pyOpenSSL 16.0+. Additionally, pyca/cryptography is
an optional dep for better performance with vault already.
This commit leverages cryptography's padding, constant time comparisons,
and CBC/CTR modes to reduce the amount of code ansible needs to
maintain.
* Handle wrong password given for VaultAES format
* Do not display deprecation warning for cryptography on python-2.6
* Namespace all of the pycrypto imports and always import them
Makes unittests better and the code less likely to get stupid mistakes
(like using HMAC from cryptogrpahy when the one from pycrypto is needed)
* Add back in atfork since we need pycrypto to reinitialize its RNG just in case we're being used with old paramiko
* contrib/inventory/gce: Remove spurious require on pycrypto
(cherry picked from commit 9e16b9db275263b3ea8d1b124966fdebfc9ab271)
* Add cryptography to ec2_win_password module requirements
* Fix python3 bug which would pass text strings to a function which
requires byte strings.
* Attempt to add pycrypto version to setup deps
* Change hacking README for dual pycrypto/cryptography
* update dependencies for various CI scripts
* additional CI dockerfile/script updates
* add paramiko to the windows and sanity requirement set
This is needed because ansible lists it as a requirement. Previously
the missing dep wasn't enforced, but cryptography imports pkg_resources
so you can't ignore a requirement any more
* Add integration test cases for old vault and for wrong passwords
* helper script for manual testing of pycrypto/cryptography
* Skip the pycrypto tests so that users without it installed can still run the unittests
* Run unittests for vault with both cryptography and pycrypto backend
