Turns out on devices where only Gigabit is available (like our
csr nodes in CI), duplex is not configurable, thus the test fail.
Removing, since we are testing modification of other properties like
description and mtu and have good coverage.
* Use correct pip version in ansible-test.
* Add git fallback for validate-modules.
* Run sanity tests in a docker container.
* Use correct python version for sanity tests.
* Pin docker completion images and add default.
* Split pylint execution into multiple contexts.
* Only test .py files in use-argspec-type-path test.
* Accept identical python interpeter name or binary.
* Switch cloud tests to default container.
* Remove unused extras from pip install.
* Filter out empty pip commands.
* Don't force running of pip list.
* Support delegation for windows and network tests.
* Fix ansible-test python version usage.
* Fix ansible-test python version skipping.
* Use absolute path for log in ansible-test.
* Run vyos_command test on python 3.
* Fix windows/network instance persistence.
* Add `test/cache` dir to classification.
* Enable more python versions for network tests.
* Fix cs_router test.
* update ec2_vpc_net_facts module to boto3
updated with RETURN values and other requested changes
removed errant extra blank line
another errant extra line removed...auto-linter not working apparently
updates per review
fix typo in RETURN docs
* fix trailing whitespace issue
* Add nxos_file_copy IT
* Restructure nxos_igmp tests
* add nxos_igmp_interface IT
* add nxos_igmp_snooping IT
* add nxos_ntp_auth IT
* Add nxos_ntp_options IT
* update nxos.yaml with new tests
* update nxos_ntp_options test
* update nxos_ntp_auth IT
In some IOS tests, we can't just assume a certain interface, as
we test on IOS and IOS-XE and in our CI they scheme numbering is
different.
Introspect version on the tests where needed.
* ec2_group: add support for rule descriptions.
* Document rule description feature and add an example using it.
* Fix removing rule descriptions.
* Add integration tests to verify adding/modifying/removing rule descriptions works as expected.
* Add permissions to hacking/aws_config/testing_policies/ec2-policy.json for updating ingress and egress rule descriptions.
* ec2_group: add backwards compatibility with older versions of botocore for rule descriptions.
* Add compatibility with older version of botocore for ec2_group integration tests.
* ec2_group: move HAS_RULE_DESCRIPTION to be checked first.
* Make requested change
* Pass around a variable instead of client
* Make sure has_rule_description defaults to None
* Fail if rule_desc is in any ingress/egress rules and the the botocore version < 1.7.2
* Remove unnecessary variable
* Fix indentation for changed=True when updating rule descriptions.
* minor refactor to remove duplicate code
* add missing parameter
* Fix pep8
* Update test policy.
* - Adds iosxr_netconf module to configure netcong service on Cisco
IOS-XR devices
* - Adds Integration test for module
- Handles diff return from load_config
* - Adds unit test for iosxr_netconf module
* windows: add #AnsibleRequires to set whether a module requires module or a specific version
* fix up pep8 issues
* changed psversion to use the actual ps Requires -Version syntax
* missed the check on #Requires -Version
* fix #Requires module extensions
* module_utils #Requires should not have .psm1 extension if "real" Powershell will ever execute them
* updated validate-modules to enforce this
* added check to disallow multi-module syntax on Ansible.ModuleUtils #Requires
* Start using ClientRequestTokens in event lists
* Include request token in all reqs that support it (basically all but check mode/changeset)
* Update placebo recordings
* Add comments for CRQ popping
* Add openssh-client to default docker container.
* Include Azure requirements in default container.
To do so, handling of pip requirements was updated to install each
set of requirements separately and then run a verification pass to
make sure there are no conflicts between requirements.
* Add missing --docker-no-pull option.
* Add documentation for the azure-requirements test.
* Handle timezone updates on Ubuntu 16.04+ on containers
Although Ubuntu 16.04 will use timedatectl by default,
containers without a working timedatectl need to use the
old method.
A bug in Ubuntu for the old method means having to write
a nasty hack
https://bugs.launchpad.net/ubuntu/+source/tzdata/+bug/1554806
* Add tests for timezones
Ensure timezone changes work across various OSs
* added win_audit_rule with integration test
* Updated integration testing to target files as well as directories
and registry keys. Split testing files apart to be more organized.
Updated powershell for better handling when targetting file objects
and optimized a bit. Removed duplicated sections that got there from a
previous merge I think.
* Decided to make all the fact names the same in integration testing.
Seemed like there would be less change of accidentally using the wrong
variable when copy/pasting that way, and not much upside to having
unique names.
Did final cleanup and fixed a few errors in the integration testing.
* Fixed a bug where results was displaying a wrong value
Fixed a bug where removal was failing if multiple rules existed due to
inheritance from higher level objects.
* Resolved issue with unhandled error when used didn't have permissions
for get-acl.
Changed from setauditrule to addauditrule, see comment in script for reasoning.
Fixed state absent to be able to remove multiple entries if they exist.
* fixed docs issue
* updated to fail if invalid inheritance_rule when defining a file rather than warn
Module allows you to wait for a bigip device to be
"ready" for configuration. This module will wait for things like
the device coming online as well as the REST API and MCPD being
ready.
If all of the above is not online and ready, then no configuration
will be able to be made.
* better cleanup on task results display
callbacks get 'clean' copy of result objects
moved cleanup into result object itself
removed now redundant callback cleanup
moved no_log tests
* moved import as per feedback
In this refactor we moved to the most recent coding standards for
both F5 and Ansible. Many bugs were fixed and some features were
also added (such as ipv6 support).
New conventions for ansible warrant fixes to accomodate those
in bigip_partition.
This patch also includes an import fix that can raise an error when
Ansible unit tests run
This adds a new type of vault-password script (a 'client') that takes advantage of and enhances the
multiple vault password support.
If a vault password script basename ends with the name '-client', consider it a vault password script client.
A vault password script 'client' just means that the script will take a '--vault-id' command line arg.
The previous vault password script (as invoked by --vault-password-file pointing to an executable) takes
no args and returns the password on stdout. But it doesnt know anything about --vault-id or multiple vault
passwords.
The new 'protocol' of the vault password script takes a cli arg ('--vault-id') so that it can lookup that specific
vault-id and return it's password.
Since existing vault password scripts don't know the new 'protocol', a way to distinguish password scripts
that do understand the protocol was needed. The convention now is to consider password scripts that are
named like 'something-client.py' (and executable) to be vault password client scripts.
The new client scripts get invoked with the '--vault-id' they were requested for. An example:
ansible-playbook --vault-id my_vault_id@contrib/vault/vault-keyring-client.py some_playbook.yml
That will cause the 'contrib/vault/vault-keyring-client.py' script to be invoked as:
contrib/vault/vault-keyring-client.py --vault-id my_vault_id
The previous vault-keyring.py password script was extended to become vault-keyring-client.py. It uses
the python 'keyring' module to request secrets from various backends. The plain 'vault-keyring.py' script
would determine which key id and keyring name to use based on values that had to be set in ansible.cfg.
So it was also limited to one keyring name.
The new vault-keyring-client.py will request the secret for the vault id provided via the '--vault-id' option.
The script can be used without config and can be used for multiple keyring ids (and keyrings).
On success, a vault password client script will print the password to stdout and exit with a return code of 0.
If the 'client' script can't find a secret for the --vault-id, the script will exit with return code of 2 and print an error to stderr.
* documentation was not inline with other Ansible modules
* Python 3 specific imports were missing
* monitor_type is no longer required when creating a new pool; it is now the default.
* A new monitor_type choice of "single" was added for a more intuitive way to specify "a single monitor". It uses "and_list" underneath, but provides additional checks to ensure that you are specifying only a single monitor.
* host and port arguments have been deprecated for now. Please use bigip_pool_member instead.
* 'partition' field was missing from documentation.
* A note that "python 2.7 or greater is required" has been added for those who were not aware that this applies for ALL F5 modules.
* Unit tests were fixed to support the above module
* Correct usage for shutil.rmtree
Fix adds correct usage of shutil.rmtree in git module
Fixes: #31225
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Include archive tests so they get run
* Use new include syntax
* Cleanup syntax on git tests
- use multi-line YAML
- remove unneeded {{ }} around vars in conditionals
- remove unneeded quotes
- add task file name to task names for easier troubleshooting when things fail
* Make archive tests work for RHEL/CentOS 6
The older versions of Jinja2 in RHEL/CentOS 6 required assertion tasks using the map filter to be skipped.
The older version of git required gzip compression to be skipped on RHEL/CentOS 6.
* Account for ansible_distribution_major_version missing
* Adding a cli transport option for the bigip_command module.
* Fixing keyerror when using other f5 modules. Adding version_added for new option in bigip_command.
* Removing local connection check because the F5 tasks can be delegated to any host that has the libraries for REST.
* Using the network_common load_provider.
* Adding unit test to cover cli transport and updating previous unit test to ensure cli was not called.
* Initial CD-ROM support
* create cdrom bugfix
* Improving CDROM change detection and fixing template creation bug
Running MarkAsTemplate on an existing template will fail with an error
* Better change detection for guest ID
Should only mark a change in case it actually changes
* Adding integration tests
* Pep8 compliance fixes
* Adding CDROM support, including iso, client and none types
* Updating added release version for CDROM option
* Fix rollback in junos_config
Fixes#30778
* Call `load_configuration` with rollback id in case
the id is given as input
* Pass rollback id to `get_diff()` to fetch diff from device
* Fix unit test
- old functionality is still available direct lookup use, the following are equivalent
with_nested: [[1,2,3], ['a','b','c']]
loop: "{{lookup('nested', [1,2,3], ['a','b','c'])}}"
- avoid squashing with 'loop:'
- fixed test to use new intenal attributes
- removed most of 'lookup docs' as these now reside in the plugins
* Remove sysctl entry when state=absent
* Cleanup sysctl integration test syntax
* Correct grammar on error message
* Add sysctl integration test for state=absent
* [rpm_key] Fix to import first key on the system
Fixes: #31483
* [rpm_key] removed unsafe_shell and "throwaway" underscore
* [rpm_key] adding test to add the first key on system
* Added warning for 'force' option
* Changed 'profiles' type to list
* Changed 'interfacetypes' type to list
* Added deprecation warning and fixed doc
* updated force parameter
* Remove cloudstack, f5, and nxos from the wildcard import whitelist
* Remove nxos and f5 from the get_exception whitelist
* Remove some files from the pep8 legacy whitelist
* win_become: Added support to become a service user
* fixes for linting
* changes to get local and network service working
* fixed linting issues again
* pleasing pepe
* Make ansible_selinux facts a consistent type
Rather than returning a bool if the Python library is missing, return a dict with one key containing a message explaining there is no way to tell the status of SELinux on the system becasue the Python library is not present.
* Fix unit test
* Add update_only parameter for yum module
When using latest, `update_only: yes` will ensure that only existing
packages are updated and no additional packages are installed.
* Update yum.py
Update version added for `update_only` parameter to 2.5
* add unit tests for update_only flag in yum module
* Add new lines to end of config file lines
* Properly write out selinux config file
Change module behavior to not always report a change but warn if a reboot is needed and return reboot_required.
Improve the output messages.
Add strip parameter to get_file_lines utility to help with parsing the selinux config file.
* Add return documentation
* Add integration tests for selinux module
* Use consistent capitalization for SELinux
* Use atomic_move in selinux module
* Don't copy the config file initially
There's no need to make a copy just for reading.
* Put message after set_config_policy in case the change fails
* Add aliases to selinux tests
* win_become: move error handling to Ansible outside of shell
* trimmed the output so double newlines don't get set
* added test for non-zero exit code
* missed issue URL on test
* changed exit to SetShouldExit
The /etc/os-release based distro detection doesn't
seem to work for Ubuntu 10.04 (no /etc/os-release?).
So it was testing the next case which was /etc/lsb-release to
see if it is 'Mandriva'. Since the check for existence of
(/etc/lsb-release, Mandrive) was the first non-empty dist
file match, 'ansible_distribution' was being set to 'Mandriva'
expecting to be corrected by the data from the dist file content.
But since the dist file parsing for Mandriva didn't match for
Ubuntu 10.04 /etc/lsb-release _and_ there is no Debian specific
lsb-release check, 'ansible_distribution' stayed at 'Mandriva'
and the dist file checking loop keeps going and eventually off
the end of the list before finding a better match.
Adding a debian/ubuntu specific check for /etc/lsb-release after
the debian os-release sets the info correctly and stops further
checking of dist files.
Fixes#30693
In cli.CLI.unfrack_path callback, special case if the
value of '--output' is '-', and avoid expanding
it to a full path.
vault cli already has special cases for '-', so it
just needs to get the original value to work.
Fixes#30550
* Added support to retrieving LIG resources in HPE OneView
* Fixing copyright header according to review
* Swapping out config for full credentials in parameter for documentation
* Added support to retrieving Enclosures in HPE OneView
- Added unit tests
* Updated version_added to 2.5
* Changing return type of enclosure_script to string
* Fixing copyright header according to review
* Replaced config for credentials in parameters for documentation
Fix adds a new module 'vmware_guest_powerstate' to manage
power states of virtual machine.
Fixes: #30371
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Fix fact failures cause by ordering of collectors
Some fact collectors need info collected by other facts.
(for ex, service_mgr needs to know 'ansible_system').
This info is passed to the Collector.collect method via
the 'collected_facts' info.
But, the order the fact collectors were running in is
not a set order, so collectors like service_mgr could
run before the PlatformFactCollect ('ansible_system', etc),
so the 'ansible_system' fact would not exist yet.
Depending on the collector and the deps, this can result
in incorrect behavior and wrong or missing facts.
To make the ordering of the collectors more consistent
and predictable, the code that builds that list is now
driven by the order of collectors in default_collectors.py,
and the rest of the code tries to preserve it.
* Flip the loops when building collector names
iterate over the ordered default_collectors list
selecting them for the final list in order instead
of driving it from the unordered collector_names set.
This lets the list returned by select_collector_classes
to stay in the same order as default_collectors.collectors
For collectors that have implicit deps on other fact collectors,
the default collectors can be ordered to include those early.
* default_collectors.py now uses a handful of sub lists of
collectors that can be ordered in default_collectors.collectors.
fixes#30753fixes#30623
Avoid the following seen when running ec2_ami tests on python3,
presumably because the return type of `map` is different between
python2 and python3.
```
Traceback (most recent call last):
File "/tmp/ansible_e44v27uj/ansible_module_ec2_snapshot_facts.py", line 242, in <module>
main()
File "/tmp/ansible_e44v27uj/ansible_module_ec2_snapshot_facts.py", line 238, in main
list_ec2_snapshots(connection, module)
File "/tmp/ansible_e44v27uj/ansible_module_ec2_snapshot_facts.py", line 193, in list_ec2_snapshots
snapshots = connection.describe_snapshots(SnapshotIds=snapshot_ids, OwnerIds=owner_ids, RestorableByUserIds=restorable_by_user_ids, Filters=filters)
File "/usr/local/lib/python3.5/dist-packages/botocore/client.py", line 312, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/usr/local/lib/python3.5/dist-packages/botocore/client.py", line 575, in _make_api_call
api_params, operation_model, context=request_context)
File "/usr/local/lib/python3.5/dist-packages/botocore/client.py", line 630, in _convert_to_request_dict
api_params, operation_model)
File "/usr/local/lib/python3.5/dist-packages/botocore/validate.py", line 291, in serialize_to_request
raise ParamValidationError(report=report.generate_report())
botocore.exceptions.ParamValidationError: Parameter validation failed:
Invalid type for parameter OwnerIds, value: <map object at 0x7ff577511048>, type: <class 'map'>, valid types: <class 'list'>, <class 'tuple'>
```
https://github.com/ansible/ansible/pull/30435#issuecomment-330750498
* Save the serialized values instead of their types
* Add tests for creating and modifying VMs without using a template
* Remove blank line
* Add tests for vm deletion
* Fix cloudwatchevent_rule exception handling
Where it is currently present, this change fixes the exception handling.
However, there are many places that it is lacking.
Fixes#30806
* Add new exception handling for cloudwatchevent_rule
Ensure all API calls are wrapped with exception handling
* PEP8 tidy up
* Remove unnecessary HAS_BOTO3 import and checks
Tidy up documentation so that NO_QA can be removed
* Use vault_id when encrypted via vault-edit
On the encryption stage of
'ansible-vault edit --vault-id=someid@passfile somefile',
the vault id was not being passed to encrypt() so the files were
always saved with the default vault id in the 1.1 version format.
When trying to edit that file a second time, also with a --vault-id,
the file would be decrypted with the secret associated with the
provided vault-id, but since the encrypted file had no vault id
in the envelope there would be no match for 'default' secrets.
(Only the --vault-id was included in the potential matches, so
the vault id actually used to decrypt was not).
If that list was empty, there would be an IndexError when trying
to encrypted the changed file. This would result in the displayed
error:
ERROR! Unexpected Exception, this is probably a bug: list index out of range
Fix is two parts:
1) use the vault id when encrypting from edit
2) when matching the secret to use for encrypting after edit,
include the vault id that was used for decryption and not just
the vault id (or lack of vault id) from the envelope.
add unit tests for #30575 and intg tests for 'ansible-vault edit'
Fixes#30575
* Fix 'distribution' fact for ArchLinux
Allow empty wasn't breaking out of the process_dist_files
loop, so a empty /etc/arch-release would continue searching
and eventually try /etc/os-release. The os-release parsing
works, but the distro name there is 'Arch Linux' which does
not match the 2.3 behavior of 'Archlinux'
Add a OS_RELEASE_ALIAS map for the cases where we need to get
the distro name from os-release but use an alias.
We can't include 'Archlinux' in SEARCH_STRING because a name match on its keys
but without a match on the content causes a fallback to using the first
whitespace seperated item from the file content as the name.
For os-release, that is in form 'NAME=Arch Linux'
With os-release returning the right name, this also supports the
case where there is no /etc/arch-release, but there is a /etc/os-release
Fixes#30600
* pep8 and comment cleanup
* Fix pkg_mgr fact on OpenBSD
Add a OpenBSDPkgMgrFactCollector that hardcodes pkg_mgr
to 'openbsd_pkg'. The ansible collector will choose the
OpenBSD collector if the system is OpenBSD and the 'Generic'
one otherwise.
This removes PkgMgrFactCollectors depenency on the
'system' fact being in collected_facts, which also
avoids ordering issues (if the pkg mgr fact is collected
before the system fact...)
Fixes#30623
* Fix nxos provider transport warning issue
* Add default value of transport arg in provider spec
* Remove default value if transport arg in top level spec
This ensure deprecation warning is seen only in case transport
is given as a top level arg in task
* Refactor nxos modules to reference transport value from provider
spec
* Fix unit test
* Remove transport arg assignment in nxos action plugin
* As assigning transport value is handled in provider spec
top level task arg assignment is no longer required
* win_scheduled_task_stat: add new module to get stat on scheduled tasks
* fixed up linting errors and aliases file
* I should learn how to spell
* removing URI from test
* added state information for the task
* removed argument so task stays running
* Undeprecate ec2_elb_*
* Make ec2_elb* full fledged modules rather than aliases
* Split tests for ec2_elb_lb and elb_classicb_lb
* Change names in documentation of old and new elb modules
Add tests for ec2_elb_lb
As discussed before we selected win_environment to the documentation,
and point to win_uri for a more advanced module.
If we want to make this the reference module, we have to get this one
absolutely right in every possible way.
This PR cleans up both win_environment and win_uri, and makes the
required changes to the windows module development section.
This PR includes:
- An important fix to charset encoding of from address
- Documentation and examples cleanup
- PEP8 fixes
- Warning on insecure access
- Strict parameter typing
- More modern interface (using lists rather than comma, space or pipe-delimited strings)
- Warn on failure to send mail to some recipients
```
[WARNING]: Failed to send mail to 'foobar': 550 5.1.1 <foobar>:
Recipient address rejected: User unknown in local recipient table
```
- Warn on failure to parse some headers
```
[WARNING]: Skipping header 'Foobar', unable to parse
```
- Return failed recipients as return value
- Changed default encoding to utf-8
* Add Routing Engine Facts
- Map routing engine output information to routing_engines facts dict.
- Add fact 'has_2RE', which is a quick way to determine how many REs
the chassis has.
* Fix a typo
* Fix more typos
* Add slot number to routing_engine dict
* Add facts about the installed chassis modules
* Fix typo
* Fixed another typo
* Fix Path
* Change path again.
* More Typos
* Add some deubgging
* Add additional information for hardware components.
- Return information about the Routing Engines.
- Return a fact to easily determine if the device
has two routing engines.
- Return information about the hardware modules.
* Addressed pep8 stardard failures.
* Add unit test fixtures.
* Rename fixture.
* Fix unit test failures.
- Rename the fixture file to what the unit test expects.
- Strip out junos namespace attributes.
Rename file to match what the unit test expects.
* Scrubbed the routing engine serial numbers.
* Add unit test facts for new tests.
- Add unit test for ansible_net_routing_engines fact
- Add unit test for ansible_net_modules fact
- Add unit test for ansible_net_has_2RE
* Fixed spacing.
Provide all necessary permissions for AMI tests
Allow tests to run in us-east-2
Ensure `always` section gets used
Update tests to ensure that cleanup works better, and add
deletion idempotency test
* win_scheduled_task: rewrite for additionality functionality and bug fixes
* fixes for docs and os version differences
* started with the testing
* doc fix
* added more tests
* added principals tests
* finished tests for win_scheduled_task rewrite
* feedback from PR
* change to fail when both new and deprecated args are set
* change diff variable to match new standard and update doc sentance
* Don't ask for password confirm on 'ansible-vault edit'
This is to match the 2.3 behavior on:
ansible-vault edit encrypted_file.yml
Previously, the above command would consider that a 'new password'
scenario and prompt accordingly, ie:
$ ansible-vault edit encrypted_file.yml
New Password:
Confirm New Password:
The bug was cause by 'create_new_password' being used for
'edit' action. This also causes the previous implicit 'auto prompt'
to get triggered and prompt the user.
Fix is to make auto prompt explicit in the calling code to handle
the 'edit' case where we want to auto prompt but we do not want
to request a password confirm.
Fixes#30491
Unittests are sometimes run without network connectivity in build
systems. Make that work correctly by mocking out _get_url_data with the
expected return value.
* windows: fix list type in legacy module utils
* only change the return for the list type instead of affecting it all
* additional null check when using an array
As-merged, had several issues that prevented idempotent usage. Some args were defined at the wrong UI level. Dual-state args didn't match up with typical Ansible UI.
* Adds nxos_pim_rp_address integration test role for group_list,
prefix_list and route_map (cli and nxapi)
* * Adds explicit removal of static RP configs to match cli behaviour
* * Removes config deletion using nxos_config module (for 2.4 only)
* * Attempt short and long delete config command
* Add a platform check for N3K for bidir
* Replace pause in integration tests with until.
Use resource prefix instead of generating a random number
Only try to delete keys if they exist
* Add alias to tests
keyUsage and extendedKeyUsage are currently statically limited via a
static dict defined in modules_utils/crypto.py. If one specify a value
that isn't in there, idempotency won't work.
Instead of having static dict, we uses keyUsage and extendedKyeUsage
values OpenSSL NID and compare those rather than comparing strings.
Fixes: https://github.com/ansible/ansible/issues/30316
Current openssl_certificate is mistakenly taking its derivating its
version number from the csr version number.
Thos two fields are completly unrelated and hence the version number of
the certificate should be able to be directly specified (via
selfsigned_version parameter).
* openssl_certificate: Fix parameter assertion in Python3
Parameter assertion in Python3 is broken. pyOpenSSL get_X() functions
returns b'' type string and tries to compare it with '' string, leading
to failure.
The error mentionned above has been fixed by sanitizing the inputs from
a user to the assert only backend.
Also, this error was hidden by the fact that the improper check method
was called in the generate() functions.
* Add simple integration test for openssl_certificate
* remove subject == issuer assertion
* run integration tests only on supported hosts
* change min supported version to 0.15.x
* Add test for more CSR fields
* also convert dict members to bytes
* fix version_compare
* openssl_{csr, certificate}: Fail if pyOpenSSL <= 0.15
Previous 0.13 pyOpenSSL was a C-binding, and required the parameter
passed to add_extention to be in ASN.1. This has changed with the move
to 0.14 and it is now all pythong and string based.
Previous the 0.15 release, the `get_extensions()` method didn't exist,
since the modules rely heavily on it we ensure pyOpenSSL version is at
last 0.15.0.
* check pyopenssl version in openssl_csr integration test
* Clean up nxos_snmp_contact & nxos_snmp_location
* Bring nxos_snmp_community in line
* Bring nxos_snmp_host in line
* And I would have gotten away with it too,
if it weren't for those meddling sanity tests
* Bring nxos_snmp_traps & nxos_snmp_user in line
* Appease Shippable
* ini_file: add integration test
Start integration tests for ini_file module.
* ini_file test: add comments for lisibility
* update from review: use var instead of checksum to assert content
* fixes#26623
* Test-Path (and thus `-type path` in Get-AnsibleParam) fail on a nonexistent drive letter, since it can't be mapped to a PSProvider.
* added support and basic smoke tests for
* Correctly validate module name for modules with aliases
If a module has an alias (ie is a symlink) then we need to ensure that
DOCUMENTATION.module is set to the main name, not the aliased name
* formatting
* Show warning when using pylint on Python 2.6.
* Add pylint disable entries for Python 2.
* Fix unicode handling in ansible-test.
* Add missing documentation.
* refactor firewalld module with object abstraction
This change creates a FirewallTransaction object that each
individual transaction type is a sub-class of as they all follow the
same pattern to enable or disable something in the firewall.
Also, there's a few bugfixes here:
- Fix the "source" type to handle permanent operations
- Remove ambiguity of required parameters for only specific use
cases that can lead to transactions effectively being a no-op.
Instead, pick sane defaults and document them.
- Change how imports are done so globals are no longer needed
This is based on the original feedback by Toshio from the last
refactor attempt:
https://github.com/ansible/ansible-modules-extras/pull/3383
Signed-off-by: Adam Miller <maxamillion@fedoraproject.org>
* fix line too long for pep8 for shippable tests
Signed-off-by: Adam Miller <maxamillion@fedoraproject.org>
* remove firewalld from pep8/legacy-files
Signed-off-by: Adam Miller <maxamillion@fedoraproject.org>
from __future__ unicode_literals leads to developer confusion as
developers no longer can tell whether a bare literal string is a byte
string or a unicode string. Explicit marking as u"" or b"" is the way
to solve the same problem in the Ansbile codebase.
* This commit includes a unit test to exercise the _is_role
function and make sure it doesn't break in any Python version.
* Import os.path and other minor fixups
On setup we set it to 'switch', so teardown should be 'switch'.
Also, using inventory_hostname breaks the test, since in our CI
it's a long UUID string, which exceeds the 32 chars maximum for setting
a hostname on NXOS.
* Update elasticsearch_plugin.py
Change module to work with Elasticsearch 2.x and 5.x automatically.
Update examples and docs.
Supersedes #21989
* Check system paths for elasticsearch-plugin binary
Use get_bin_path from basic.py for searching paths.
* Create a copy of PLUGIN_BIN_PATHS rather than modifying the global
* Use provided plugin_bin path first before trying other places
Change global PLUGIN_BIN_PATHS to a tuple
* Divide Windows integration tests into 2 groups.
* Support `none` for `--changed-all-target`.
* Run 2 separate Windows groups on Shippable.
* Only run smoketest and minimal for the group1 job.
* Add EOS provider options as subspec
* Add IOS provider options as subspec
* Add IOS XR provider options as subspec
* Add Junos provider options as subspec
* Add NX-OS provider options as subspec
* Add Vyos provider options as subspec
* Remove password checks from check_args
* Do the same to aireos, aruba, ce, dellos*, & sros, as they work the same way
* VyOS does not support `transport`
* module should fail if eos_user is added without configured_password or nopassword or sshkey
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
* fix eos_user unit test
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
* fix eos_user integration test
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
Ansible-test will consume group_vars from test/integration, as it runs
from that working directory. This causes problems if we specify vars
in inventory as they have higher precedence, plus it gives the impression
to users those group_vars can be reused.
Leaving vyos since it's configured to run in Shippable and apparently
it breaks if there are no group_vars in the test/integration folder.
* Fix junos_user pruge option failures
Fixes#25989
Add seperate handling for purge option which
fetches configured users on remote device
and delete the one not present in aggregate
list.
* Minor changes
* Remove network integration group_vars/host_vars
We use our own inventory in DCI, which is passed to ansible-test
with --inventory.
However, as the working directory of ansible-test is set to
test/integration, ansible consumes the repo group_vars/host_vars.
That imposes a problem, since they have greater precedence to
inventory variables.
Let's just remove group_vars/host_vars so end-users can assume
those vars can be used, the inventory and group/host vars are
environment dependent, each user should create their own.
* Restore files I git rm'd by mistake
* Add FcoeNetworkFactsModule for retrieving HPE OneView FCoE Networks
- Allow querying for FCoE Network resources in HPE OneView
- Adds unit tests to new module
* Fix "required: no"
* Add NetworkSetFactsModule for retrieving HPE OneView Network Sets
- Allow querying for Network Set resources in HPE OneView
- Adds unit tests to new module
- Updates oneview_module_loader copyright header to short GPL3 version
* Adding possibility to pass in credentials as parameters
* Removed required false and changed format of filter_by_name declaration
* Updated examples in docs to reflect new way to pass in credentials
- All examples of the oneview_network_set_facts updated to use
credential parameters
- All required=False from oneview base module removed
- Shared docs updated to bring attention to API version being used
* add template for az func
* (wip) add basic azure functions support
* add support to add app settings to azure function
* add support for updating based off of app settings
* add integration tests and refactor required param
* support check mode and add facts module
* add test for azure functions facts module
* add necessary checks and registrations for web client
* fix documentation
* change return type from complex to dict
* disable azure_rm_functionapp tests until stable
* remove dict comprehension for py2.6
* pepe has whitespace tumor
* Adding acs module
* linter issue
* Reduce the VM Size for the int. tests
* Short the name of the cluster
* Fixing the asserts and title
* Fixing VM Scale count in int. tests
* Changing the location of the tests
* trying eastus2
* disabling acs CI tests until stabilized
* new facts module for dns zone
integration tests and new module for dns zone facts
* use vairable for domain name
* add nondeterministic piece to domain name
* fix azure_rm_dnszone_facts examples
* create new module for record set facts
added new module to get facts for dns zone record sets
* use variable for domain name
* correct lint error
* add nondeterministic piece to name
* fix azure_rm_dnsrecordset_facts examples
* replace duff commit version of win_toast
* change expire_mins to expire_secs and add example showing use of async
* fix metadata version to keep sanity --test validate-modules happy
* code review fixes and change expire_secs to expire_seconds
* add first pass integration tests for win_toast
* win_toast no longer fails if there are no logged in users to notify (it sets a toast_sent false if this happens)
* yaml lint clean up of setup.yml in win_toast integration tests
* improve exception and stack trace if the notifier cannot be created, following feedback from dag
* removed unwanted 'echo' input parameters from return vals; added to CHANGELOG.md, removed _seconds units from module params; updated tests to match
lint
Update integration test
handle check_mode
handle warnings
Removing the empty tags check
Updating author handle
To use github handle
Changing from warn to fail
disabled tests
* First version of managed disks: Multiple methods for creating, one get and one delete
* My name is too cool to be there
* Passing pep8 test
* Create and delete empty working
* Module for testing, lacks sense of setting state changed for unchanged operations
* Checking that actual changes are done to update status changed
* First version of the integration test
* Adding the dictionary to translate the facts, without using serializer
* Adding the serializer to managed disk
* Using native serializer in managed disks facts
* Added DiskSku to pass the whole class to the disk instead of a string
* Bumping version
* Passing sanity checks
* Aparently they Id is not returned by the serilizer
* Integration test
* Latest version
* Using my own serializer, the one in commons is not working for me
* Managed disks module support for ansible
* Updating my github account username in the file
* Sanity checks were missing in facts and integration test
* Fixing typo in source_resource_uri and removed AzureHTTPErrors
* Update tags are working now
* Integration test for tags
* Added support for tags, added check mode, corrected style
* (wip) add partial loadbalancer module
* (wip) add ability to use a public ip for a load balancer
* fix shebang
* add backend address pool to load balancer
* remove unncessary error variable
* add probe support to load balancer
* add ability to add load distribution rule to load balancer
* add nat pool functionality to azure load balancer
* fix pep8 errors from sanity check
* add documentation for load balancer
* refactor imports
* fix license header copyright
* add facts module for azure load balancer
* fix ansible-test failures
* add integration tests for load balancer
* fix metadata version
* add complex integration test to azure_rm_loadbalancer
* Adds win_pagefile module
* Fixed win_pagefile doc
* Fix win_pagefile doc
* Fix win_pagefile doc variable convention
* Added check_mode
* Changed win_pagefile module&doc to the convention
* added win_pagefile integration tests
* Changed check_mode blocks to whatif, fixed a bug
* Added whatif to set-wmiinstance, changed docs
Added whatif in Set-WMIInstance
Added dots to end of decription lines
* Returns to original state at the end, more tests
Added override and no override integration tests
Pagefiles now return to same state as before at the end of the integration test
* Remove extra line
* Added test_path var to win_pagefile
* Set test_path as 'no' in integration
* Added unit to docs and enclosed exception message
* More granular try-catch blocks
* Added workaround to avoid value out of range
* Deleted wrong line ending
* Changed license to one-line
* Removed space in line ending
* Try to fix python2.6 error
* Try 2 to fix python2.6 error
* Add separating line again
* Add the ability to modify shard count to kinesis_stream module
* Fixed an issue in kinesis_stream where update() reports not changed when it is changed
* Remove unreachable message and make the try and catch block shorter
* added windows module win_power_plan with integration test
integration testing updated to handle skipping 2008 while testing that
module provides helpful error
* minor docs fix
* my fault, too many spaces
* #18807 win_firewall_rule uses HNetCfg.FwPolicy2 COM object
* Added missing tests
* Added support for InterfaceTypes property
* Added support for EdgeTraversalOptions property
* Added SecureFlags property
* Port ranges are not possible in W2K8
* Added windows version checks
* Fixed doc: removed 'force' option and all notes
* Fixed copirights and docs
* added azure vm extensions support
* added auto_update_minor_version and protected_settings
* update docs
* added get_poller_result
* fixed test failures
* remove tags and check mode
* Include Integration Tests
Include Integration Test for azure_rm_virtualmachine_extensions.py
* Correct location for integration test files
* Revert "Fix leading slashes being stripped from mount src (#24013)"
This reverts commit 3251aecd95.
* Revert "Convert value of port and timeout in provider to integer (#28732)"
This reverts commit a914a39975.
* Revert "fix nxapi nxos tests (#28762)"
This reverts commit 4137169b21.
* Tidy mount module for testing
Fix spelling mistakes in comments. I *think* the example for omitting parents
root has the wrong parent ID.
Make mountinfo file a parameter for testing.
* Don't strip leading slash from mounts
The current code does not follow the example, it produces src=tmp/aaa instead
of src=/tmp/aaa. This causes problems with bind mounts under /rootfs.
* Use dictionary to store mounts by ID
Instead of looping over each one to check if the ID matches. This does not
preserve the order of the output on < Python3.6, but that is not necessary.
* Make linux_mounts a dict
Always accessed by 'dst', so avoid looping by just making it a key.
* Add test case for get_linux_mounts
We need to use IP address, as inventory_hostname returns an UUID from
openstack.py inventory in our CI.
Also, commenting the username/pass as we set them in the inventory (we shall
change this in the future).
* rewrite of win_package to enable win_msi deprecation
* fix some minor doc issues
* Removed exe tests
* dag's changes seem to be missing, re-add them
* fixed yaml for return values
* Check registry to find RabbitMQ installation path
* Integration tests for win_rabbitmq_plugin
* Added himself to BOTMETA.yml
* Skipped running tests on Windows 2008 SP2
* npm: fix idempotence
* Better idempotency fix
More intelligently add --production rather than depending on hard coded order in args list
Cleanup boilderplate imports and license
PEP8 fixes
* make groups magic var dependant on inventory
it was overtly restricted by 'host'
minor fixes to test_var_manager.py, need to test other values also
* pepe hates extra blank line
- Fixes to lambda
- reformatting + tests for lambda_facts
- lambda module integration test
- switch lambda and lambda_facts to AnsibleAwsModule
- Get the account ID from STS, GetUser, and finally error message
* Update RDS parameter group for boto3
* Update to boto3
* Update to latest ansible standards
* Remove choices list for valid engines (See #19221 for context)
* Allow tagging
* Return some useful information, and document that information
* Add tests for rds_param_group
* Improve testing of rds_param_group
* Add purge_tags option for rds_param_group
* Fix remaining broken rds_param_group tests
* Ensure the group name is lowercased. Fixes integration tests when run on OSX
Fix adds refactor of code for find_host_by_cluster_datacenter
API in vmware_host and vmware_resource_pool.
Also, adds strict check for cluster and datacenter object find.
Testcase for vmware_resource_pool.
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Added FcNetworkFactsModule to query FC Network Resources in HPE OneView
- Added unit tests
* Removing unnecessary quotes from example and documentation
* Module for uploading templates into vDirect server
Module for uploading configuration and workflow templates
into Radware vDirect server
* Module for uploading templates into vDirect server
Module for uploading configuration and workflow templates
into Radware vDirect server
This PR includes:
- Improvements to change-detection by comparing 2 objectified XML trees
- Implement better integration tests by comparing 2 files using copy
* Add VPN module and unittests
* remove unnecessary imports
* fix documentation
* raise custom exception rather than passing module everywhere
* remove recordings
Rerecord tests
Fix docs
Ensure vpn_connection_id is a list of strings when checking if it exists
* fix check mode
* Rerecord tests
* remove superfluous code and comments and make exception handling uniform
fix docs
* Fix ec2_vpc_vpn documentation
* make ec2_vpc_vpn compatible with python 2.6 and make check mode logic more succinct
* fix comparison of list of dicts
* Fix typos and docstrings
make requested changes for imports
make code clearer
* Fix copyright and metadata version.
* Adding module to manage ethernet network on HPE OneView
* Adding unit tests to EthernetNetwork module
* Added OneViewModuleException custom exceptions to module
- Removed exception imports from hpOneView
- Updated unit tests
* Fixing mock import inside ethernet network module unit test
* Fixing issues found in METADATA by CI
* Updated paths to use solution name instead of vendor name
* Fixed documentation, removed redundant if and improved readability
* Updated _bulk_present to use and return `result`, same way as _present
* Changed __ to _ in private methods following ansible style
* Fixed some example inconsistencies and turned states doc into a list
* Added adriane-cardozo to list of maintainers
So the module was now not doing anything except adding a nic to a
vswitch. This PR adds idempotency checks, and removal checks.
We should probably add modification chacks as well, but I don't know
what the limitations are for vcsim.
* New module for AWS Direct Connect connections
Unittests for new module
Added utils that will be used by other Direct Connect modules
* pep8 fixes
* Correct aws_direct_connect_connection EXAMPLES to be valid yaml
* Make requested changes
* remove use of the variable changed
get rid of unnecessary parentheses
* Fix unit test
* Rename variable.
* Fix metadata version
* Add module_utils/aws/direct_connect.py for frequently used functions
* new AWS Direct Connect link aggregation group module with tests and placebo recordings
* remove extra argument
* Remove use of undefined var
* Fix param name for extra exception codes for AWSRetry to use.
* Fix undefined var and line length and metadata version number
* Fix copyright headers
* Added new oneview_fcoe_network module and unit tests
* Fixing metadata issues and importing unittest from ansible.compat.tests
* Fixing shebang and adding correct copyright header
* Renamed remote_management/hpe to remote_management/oneview
* Updated documentation and default state according to review comments
- Added present as the default state
- Added delegate_to: localhost in all examples
- Changed config path from a variable to illustrate a location
- Other documentation fixes
* Added the ability to extend the exception list in CloudRetry
* AWSRetry boto and boto compatible
* Updated tests to reflect boto/boto3
* Added boto to shippable requirements
* Have base_class and added_exceptions default to None in CloudRetry
AWSRetry - only retry on boto3 exceptions and remove boto requirement from tests
* Make requested changes.
* win_wait_for: added module
* Increased timeout on async task
* added more debug messages for test debug
* revert to using win_shell instead of win_command
* rebased from master and updated copyright string
* Updated metadata version
* Capitalised start of short_description
* disabled win_wait_for tests until async issues are solved
* Add tags support to cloud/amazon/ec2_group
* Finish making ec2_group tag support boto3 compatible.
Add integration tests to validate that tags are working as expected.
* Improvements and fixes in the packet_device module
* add version_added to new args
* remove default value from facility
* changed 'lock' from deprecated arg to alias of 'locked'
* openssl_privatekey: Extend test coverage
Extend the coverage of the integration test for the module
openssl_privatekey.
New tests have been added:
* passphrase
* idempotence
* removal
Co-Authored-By: Pierre-Louis Bonicoli <pierre-louis.bonicoli@gmx.fr>
* openssl_publickey: Extend test coverage
Extend the coverage on the integration test for the module
openssl_publickey.
New tests have been added:
* OpenSSH format
* passphrase
* idempotence
* removal
* Fix 'the the' typos, fix 'pahting' filename typo
* Change 'the the' typos to a single 'the'.
* Change `playbook_pahting.rst` to `playbook_pathing.rst`.
* Delete trailing space in ec2_vol example
Delete the trailing space in `instance: "{{ item.id }} "`, which makes the
example fail when run because it looks for instance "i-xxxx ".
* win_copy rewrite with new tests and functionality
* minor pep fixes
* Handle UTF-8 filenames in zip
* fix for template
* when zip assemblies are not available in .net revert to old behaviour of copying one by one
* typo fix
* some more typos
* updated logic to correctly handle when new directories can be created
* removed testing file as it is not needed
* updated documentation based on PR
* Move sanity into directory.
* Omit abstract classes from returned subclass list.
* Split sanity tests out into plugins.
* Fix abstract class handling for Python 3.
This was causing wrong behaviour when `prev_state` was `hard`-link,
since the `file` module tried to apply the same `state` on the new
file, causing unexpected errors.
Particularly, both `overlay` and `devicemapper` storage drivers in
docker use hardlinks to share files between layers. This causes
most ansible playbooks to fail when working with files from layers
below.
This PR includes:
- PEP8 compliancy
- A fix to ensure the module fails when it failed for a package
- Various cosmetic changes to documentation
- Make `state: present` the default (and not required)
* Handle errors in jmespath in json_query better
Catch any exceptions raised from jmespath and raise
an AnsibleFilterError instead.
Avoid a traceback.
Fixes#20379
* pep8
* Include .github in test targets. Fix BOTMETA.yml.
* Include bin in compile tests.
* Exclude links from test targets.
* Include bin in pep8 and pylint tests.
* Fix pep8 and pylint issues in bin dir.
We need to run network-integration against platform agnostic modules,
which they all are named as net_* .
There is a hardcoding in executor where if prefix is 'net' the hosts is set
to all, removing these lines to have desired behaviour.
* alternatives: add integration tests
* alternatives: handle absent link (fix AttributeError)
Error occurred at least on Debian Stretch and OpenSuse 42.2:
Traceback (most recent call last):
File "/tmp/ansible_RY6X41/ansible_module_alternatives.py", line 161, in <module>
main()
File "/tmp/ansible_RY6X41/ansible_module_alternatives.py", line 113, in main
current_path = current_path_regex.search(display_output).group(1)
AttributeError: 'NoneType' object has no attribute 'group'
update-alternatives stdout sample:
dummy - manual mode
link best version is /usr/bin/dummy1
link currently absent
link dummy is /usr/bin/dummy
* alternatives: PEP 8 fixes
* alternatives: fix copyright in integration tests
* alternatives: nested loops handle more than 2 items
Thanks to Michael Scherer (@mscherer) for pointing that.
* alternatives: enable integration tests
* refactors nxos_vrf_af module
fixes#27595
* fix up unit test cases
* add commands to result dict
* add route-target on afi create
* adds deprecation note to safi argument
* Add network value to support_by field.
* New support_by value, certified
* Deprecate curated in favor of certified
* Add conversion from 1.0 to 1.1 to metadata-tool
* Add supported by Red Hat field to ansible-doc output
* Ansible Config part2
- made dump_me nicer, added note this is not prod
- moved internal key removal function to vars
- carry tracebacks in errors we can now show tracebacks for plugins on vvv
- show inventory plugin tracebacks on vvv
- minor fixes to cg groups plugin
- draft config from plugin docs
- made search path warning 'saner' (top level dirs only)
- correctly display config entries and others
- removed unneeded code
- commented out some conn plugin specific from base.yml
- also deprecated sudo/su
- updated ssh conn docs
- shared get option method for connection plugins
- note about needing eval for defaults
- tailored yaml ext
- updated strategy entry
- for connection pliugins, options load on plugin load
- allow for long types in definitions
- better display in ansible-doc
- cleaned up/updated source docs and base.yml
- added many descriptions
- deprecated include toggles as include is
- draft backwards compat get_config
- fixes to ansible-config, added --only-changed
- some code reoorg
- small license headers
- show default in doc type
- pushed module utils details to 5vs
- work w/o config file
- PEPE ATE!
- moved loader to it's own file
- fixed rhn_register test
- fixed boto requirement in make tests
- I ate Pepe
- fixed dynamic eval of defaults
- better doc code
skip ipaddr filter tests when missing netaddr
removed devnull string from config
better becoem resolution
* killed extra space with extreeme prejudice
cause its an affront against all that is holy that 2 spaces touch each other!
shippable timing out on some images, but merging as it passes most
* - deprecated panos_address and panos_service in lieu of common panos_object
* - deprecated/removed panos_address and panos_service in lieu for panos_object
* squash! - deprecated/removed panos_address and panos_service in lieu for panos_object
* - fixed PEP8 issues
* - ansible_metadata requires metadata_version instead of just version key in 2.4
* add > to multi line descriptions
* update version string to 2.4
* Update legacy-files.txt
* prompt for new pass on create/encrypt if none specified
Make 'ansible-vault' edit or encrypt prompt for a password
if none or provided elsewhere.
Note: ansible-playbook does not prompt if not vault password
is provided
* dont show vault password prompts if not a tty
* Fail if an empty string is set as src for copy module
Fixes#27363
* Cleanup task formatting on copy tests
Use multi-line YAML
Add debug statements with verbosity: 1 rather than leave them in there commented out.
* Add test for empty string as source
* Do more checks in order to add more specific errors messages
Add more integration tests for the various failure scenarios.
Cleanup some syntax on existing integration test tasks.
* Add config option for a default list of vault-ids
This is the vault-id equilivent of ANSIBLE_DEFAULT_PASSWORD_FILE
except ANSIBLE_DEFAULT_VAULT_IDENTITY_LIST is a list.
* Better handling of empty/invalid passwords
empty password files are global error and cause an
exit. A warning is also emitted with more detail.
ie, if any of the password/secret sources provide
a bogus password (ie, empty) or fail (exception,
ctrl-d, EOFError), we stop at the first error and exit.
This makes behavior when entering empty password at
prompt match 2.3 (ie, an error)
* Add comment option to authorized_keys
* Update version_added for authorized_keys comment
* PEP8
* Include index rank in parsed_key_key
* Properly display diff
Only display diff if specificed via settings
* Fix PEP8 test failure
Removed from legacy files since it is now properly formatted
* Cleanup integration test formatting and add test for new comment feature
* Correct version_added for new option
* Add intent arguments for ios_interface
* Intent argument support
* Integration test case for intent arguments
* Fix ci issue
* Add intent arguments for iosxr_interface
* Add intent check support for iosxr_interface
* Integration test for intent + configuration
* Fix ci failure
* Handle common agrument in aggregate parameter for vyos module
* Add supoort to set parameter in aggregate to it's respctive
top level argument if value not provided in aggregate.
* Aggregate argument spec validation
* Documentation for aggregate
* Fix unit test failure
* circonus_annotation: clean description
- add 'default' field
- default value for 'required' field is false
- use formatting function
* circonus_annotation: clean argument_spec
remove useless conversion
default of 'required' False
use 'default' when possible
* circonus_annotation: fix pep8
* circonus_annotation: add RETURN block
* circonus_annotation: check_mode isn't supported, add a note
* [password] _random_password -> random_password and moved to util/encrypt.py
* [passwordstore] Use built-in random_password instead of pwgen utility
* [passwordstore] Add integration tests
* First batch of modules renamed from plural to singular
Related to this proposal: https://github.com/ansible/proposals/issues/10
* Emit rename deprication warning
* Update legacy-files.txt and skip.txt to reflect new names
1fe67f9 introduced retries to the ssh connection put file and fetch
file. Unfortunately, that change broke the smart transport because it
started raising exceptions instead of returning from _run(). This
breakage is documented in #23711.
An attempt to fix it was made at #23717 but the first attempt was
objected to as needing to touch too much code. The second attmept was
objected to as smart was forced to encapsulate retries (thus retrying
a sftp "rety" times before trying scp "retry" times and then finally
moving onto piped). This third attempt has retries encapsulate smart.
So each sub-transport is tried once and if all three fail, another retry
attempt is made which tries each of the three again.
Fixes#23711Fixes#23717
* Add state intent argument in vyos interface
* State intent argument support
* Integration test for supported intent arguments
* Add intent testcase
* FIx ci issue
* junos_interface intent arguments
* Add check for intent argument in junos_interface
* Integration test for intent arguments
* Minor type fixes
* Add delay only if config diff is present
* add enabled configuration argument
* net_interface test case changes
* Minor doc change
Tested with:
from datetime import datetime, timedelta
from dateutil.relativedelta import relativedelta
n = datetime.now()
assert n + timedelta(days=365) == n + relativedelta(years=1)
* Fix idempotency for Unix permissions in zip files.
This fix prevents the unarchive module from reporting 'changed' when a zipfile contains items with Unix permissions that differ from the system default.
* Update zip unarchive tests.
Additional tests for the unarchive module with zip files:
- Test file in zip archive with non-default permissions
- Test file added to zip archive with Windows permissions
* Additional fix for mixed win/unix archives.
Turns out my original fix fails under some mixed archives, as setting the umask to zero can be applied to those files. This creates a per-file umask variable, so a mix of permission types don't cause problems.
* CI Checks
CI checks for archives with:
* non default Unix permissions
* Windows permissions
* Workaround for BSD differences.
Using Zipinfo due to lack of support in BSD unzip.
Permissions handling is also different in BSD -- always applies UMASK to file permissions.
* Added checks for creating directories and SSH keys for existing users.
username param is read by eos action plugion load_provider, if present
it is set as the play context remote user.
That means if you have a play remote_user: admin it won't be used, but instead
whatever is put in the task argument username.
Changing the tests to use name fixes the issue, we may want to remove username
as valid module argument in other change.
* replace boto with boto3 for the s3 module
make s3 pep8 and remove from legacy files
fix s3 unit tests
* fix indentation
* s3 module - if we can't create an MD5 sum return None and always upload file
* remove Location.DEFAULT which isn't used in boto3 and tidy up the docs
* pep8
* s3: remove default: null, empty aliases, and required: false from documentation
fix incorrectly documented defaults
* Porting s3 to boto3. Simplify some logic and remove unused imports
* Fix s3 module variables
* Fix a typo in s3 module and remove from pep8 legacy files
* s3: add pagination for listing objects.
Fix logic and use head_object instead of get_object for efficiency.
Fix typo in unit test.
* Fix pagination to maintain backwards compatibility.
Fix incorrect conditional.
Remove redundant variable assignment.
Fix s3 list_object pagination to return all pages
* Use the revised List Objects API as recommended.
* Wrap call to paginated_list in a try/except
Also remembered to allow marker/prefix/max_keys to modify what keys are listed
* Simplify argument
* win_hotfix: new module to install Windows hotfixes
* revert to older module util loader style to satisfy the checks for now
* Changes from PR
* changed the test hotfix so we can run tests in shippable
* win_user_right: add module with tests
* fixed up name of module in docs
* forgot the test module
* fixed up whitespace
* changes made to win_user_right based on feedback
* moved away from using secedit to Win32 with P/Invoke
* tidied up copyright for documentation
The prompt_formats dict didn't get the 'prompt_ask_vault_pass'
item added for interactive --ask-vault-pass, which
caused "KeyError: u'prompt_ask_vault_pass'"
Fixes#27885
In answer to #2540, `aptitude` was introduced as tool of choice for running
upgrades in the apt module and installing new packages that arise as
dependencies during upgrades.
This recently lead to problems, as for example Ubuntu Xenial (16.04) ships
without aptitude (installed).
Studying the man pages of both apt-get and aptitude, it appears that we can
achieve the effects of `aptitude safe-upgrade` using
```
apt-get upgrade --with-new-pkgs --autoremove
```
while `aptitude full-upgrade` seems to be identical to `apt-get dist-upgrade`.
We use `apt-get` as described above as a fall-back in case that `aptitude`
cannot be found, issuing a warning when it does so.
Furthermore it introduces a flag `force_apt_get` which may be used to enforce
usage of apt-get (which does not issue a warning).
The integration tests are updated accordingly.
Cf. also the discussion in #27370.
Fixes#18987
* Add new ITs for nxos_bgp, nxos_bgp_af, nxos_bgp_neighbor, and nxos_evpn_global
* Add nxos_facts IT and enhance existing tests
* switch nxos_feature tests to use bgp
* misc. test fixes
* Add checks for titanium in IT
* Handle non-titanium case in tests
* Add aggregate functionality to eos_vrf
* Add tests for eos_vrf aggregate option
* Remove test2 and test3 vrfs at the beginning of the eos_vrf tests
* Pull all vrfs
With aggregate, we need to get all VRFs and we then compare with
desired VRFs, instead of assuming it will be just one.
* Adding VMware tools module
Functionality: Waits for VMware tools to become available (running
state)
* Adding base integration test preparations
Until govcsim supports actual guest tool status, the tests are disabled
* Cleanup and better getvm method
* Updating Changelog
* Adding required metaclass and future import
* Rename to vmware_guest_tools_wait
* Cleanup of documentation
* Fixing review remarks
* template: fix KeyError: 'undefined variable: 0
For compatibility with the Context.get_all() implementation
in jinja 2.9, make AnsibleJ2Vars implement collections.Mapping.
Also, make AnsibleJ2Template.newcontext() handle dict type
for the 'vars' parameter.
See: d67f0fd4cc
Fixes: https://github.com/ansible/ansible/issues/20494
* add units/template/test_vars
* intg tests for jinja-2.9 issues like 20494
test cases here are based on
https://github.com/ansible/ansible/issues/20494#issue-202108318
This provides a sensu_client module in order to be able to dynamically
configure a Sensu client.
It takes a different approach than the existing Sensu modules such as
sensu_check but is hopefully a much more flexible and simple way of
handling configurations.
* nxos_bgp_neighbor_af does not want required_together
* fixup tests
* Fix max_prefix_* issues
* Require address-family
* Fix idempotency for next_hop_third_party
* Fix idempotency for allowas_in*
* Fix idempotency for *_in and *_out
* Reorder command generation again
`default` is first, then `max-prefix`, then booleans
* Added in support for 'agent' and 'node' types.
* Tidies and moves `consul_acl` module closer to PEP8 compliance.
* Switched from using byspoke code to handle py2/3 string issues to using `to_text`.
* Made changes suggested by jrandall in https://github.com/ansible/ansible/pull/23467#pullrequestreview-34021967.
* Refactored consul_acl to support scopes with no pattern (and therefore a different HCL defintion).
* Corrects whitespace in Consul ACL HCL representation.
* Fixes Consul ACL to return the HCL equivalent JSON (according to the Consul docs) for the set ACLs.
* Repositioned import to align with Ansible standard (!= PEP8 standard).
* Adds Python 2.6 compatibility.
* Fixes PEP8 issues.
* Removes consul_acl.py as it now passes PEP8.
* Follows advice in the "Documenting Your Module" guide and moves imports up from the bottom.
* Tidies consul_acl module documentation.
* Updates link to guide about Consul ACLs.
* Removes new line spaces from error message string.
* Provide better error message if user forgets to associate a value to a Consul ACL rule.
* Minor refactoring of Consul ACL module.
* Fixes bug that was breaking idempotence in Consul ACL module.
* Detects redefinition of same rule.
* Adds test to check the Consul ACL module can set rules for all supported scopes.
* Fixes return when updating an ACL.
* Clean up of Consul ACL integration test file.
* Verify correct changes to existing Consul ACL rule.
* Adds tests for idempotence.
* Splits Consul ACL tests into cohesive modules.
* Adds test for deleting Consul ACLs.
* Test that Consul ACL module can set all rule scopes.
* Fixes issues surrounding the creation of ACLs.
Thanks for the comments by manos in https://github.com/ansible/ansible/pull/25800#issuecomment-310137889.
* Stops Consul ACL's name being "forgotten" if ACL updated by token.
* Fixes incorrect assignment when a Consul ACL is deleted.
* Fixes value of `changed` when Consul ACL is removed.
* Fixes tests for Consul ACL.
* Adds interal documentation.
* Refactors to separate update and create (also makes it possible to unit test this module).
* Improves documentation.
* Completes RETURN documentation for Consul ACL module.
* Fixes issue with equality checking for `None` in ACL Consul.
* Fixes Python 2 issue with making a decision based on `str` type.
* Fixes inequality check bug in Python 2.
* Adds tests for setting ACL with token.
* Adds support for creating an ACL with a given token.
* Outputs operation performed on Consul ACL when changed.
* Fixs issue with test for creating a Consul ACL with rules.
* Corrects property used to set ACL token in python-consul library.
* Fixes tear-down issue in test that creates a Consul ACL using a token.
The tests rely on setting the lldp IP on the management interface.
However, the IP discovered is the private IP of the node, and tests
require accessing it via Nodepool node public IP.
Removing that test for now to get CI green again, we'll reasses once we
release 2.4.
There's been a change in persistent connect framework that switches
playbook timeout (which corresponds to 'timeout' param) to command_timeout.
While we fix this and return the functionality, let's put the command_timeout
in place to avoid CI being red.
* Import original unmodified upstream version
This is another attempt to get the xml module upstream.
https://github.com/cmprescott/ansible-xml/
This is the original file from upstream,
without commit 1e7a3f6b6e2bc01aa9cebfd80ac5cd4555032774
* Add additional changes required for upstreaming
This PR includes the following changes:
- Clean up of DOCUMENTATION
- Rename "ensure" parameter to "state" parameter (kept alias)
- Added EXAMPLES
- Remove explicit type-case using str() for formatting
- Clean up AnsibleModule parameter handling
- Retained Python 2.4 compatibility
- PEP8 compliancy
- Various fixes as suggested by abadger during first review
This fixescmprescott/ansible-xml#108
* Added original integration tests
There is some room for improvement wrt. idempotency and check-mode
testing.
* Some tests depend on lxml v3.0alpha1 or higher
We are now expecting lxml v2.3.0 or higher.
We skips tests if lxml is too old.
Plus small fix.
* Relicense to GPLv3+ header
All past contributors have agreed to relicense this module to GPLv2+, and GPLv3 specifically.
See: https://github.com/cmprescott/ansible-xml/issues/113
This fixescmprescott/ansible-xml#73
* Fix small typo in integration tests
* Python 3 support
This PR also includes:
- Python 3 support
- Documentation fixes
- Check-mode fixes and improvements
- Bugfix in check-mode support
- Always return xmlstring, even if there's no change
- Check for lxml 2.3.0 or newer
* Add return values
* Various fixes after review
* rm unneeded parens following assert
* rm unused parse_vaulttext_envelope from yaml.constructor
* No longer need index/enumerate over vault_ids
* rm unnecessary else
* rm unused VaultCli.secrets
* rm unused vault_id arg on VaultAES.decrypt()
pylint: Unused argument 'vault_id'
pylint: Unused parse_vaulttext_envelope imported from ansible.parsing.vault
pylint: Unused variable 'index'
pylint: Unnecessary parens after 'assert' keyword
pylint: Unnecessary "else" after "return" (no-else-return)
pylint: Attribute 'editor' defined outside __init__
* use 'dummy' for unused variables instead of _
Based on pylint unused variable warnings.
Existing code use '_' for this, but that is old
and busted. The hot new thing is 'dummy'. It
is so fetch.
Except for where we get warnings for reusing
the 'dummy' var name inside of a list comprehension.
* Add super().__init__ call to PromptVaultSecret.__init__
pylint: __init__ method from base class 'VaultSecret' is not called (super-init-not-called)
* Make FileVaultSecret.read_file reg method again
The base class read_file() doesnt need self but
the sub classes do.
Rm now unneeded loader arg to read_file()
* Fix err msg string literal that had no effect
pylint: String statement has no effect
The indent on the continuation of the msg_format was wrong
so the second half was dropped.
There was also no need to join() filename (copy/paste from
original with a command list I assume...)
* Use local cipher_name in VaultEditor.edit_file not instance
pylint: Unused variable 'cipher_name'
pylint: Unused variable 'b_ciphertext'
Use the local cipher_name returned from parse_vaulttext_envelope()
instead of the instance self.cipher_name var.
Since there is only one valid cipher_name either way, it was
equilivent, but it will not be with more valid cipher_names
* Rm unused b_salt arg on VaultAES256._encrypt*
pylint: Unused argument 'b_salt'
Previously the methods computed the keys and iv themselves
so needed to be passed in the salt, but now the key/iv
are built before and passed in so b_salt arg is not used
anymore.
* rm redundant import of call from subprocess
pylint: Imports from package subprocess are not grouped
use via subprocess module now instead of direct
import.
* self._bytes is set in super init now, rm dup
* Make FileVaultSecret.read_file() -> _read_file()
_read_file() is details of the implementation of
load(), so now 'private'.
* Changed rpm-keyid extraction and verification method
* minor style fixes
* fixed rpm key deletion,added integration test for mono key,fixed wording in integration tests
* moved aws elasticache module to boto3
* fixed error and improved code
* implemented requested changes
* now checking for missing boto3 packages in a better way
* now dynamically setting the default port depending on the engine if it is not set
* moved standard import in front of ansible ones
* now case insensitive in regards to engine name
* removed superfluous spaces
* now checking for None in the correct way
* removed elasticache module from exceptions to pep8 testing
* removed hardcoded default ports and letting aws decide if no port is given
Updates ec2_lc module to use boto3. Adds parameters:
instance_id
placement_tenancy
Also added a second example using instance_id and updated the docs with the new parameters.
We are reserving the _ identifier for i18n work. Code should use the
identifier dummy for dummy variables instead.
This test is currently skipped as someone needs to generate the list of
files which are currently out of compliance before this can be turned
on.
This PR includes:
- Documentation improvements (mostly related to boolean defaults)
- Make PEP8 compliant
- Ensure imports are specific
- Few cosmetic changes (sort lists, casing, punctuation)
* Add new Fedora docker images with Python 3.
* Use consistent env var for lookup test.
* Fix testing of virtualenv with Python 3.
* Fix docker_secret tests on Fedora 26.
* Add Python 3 support to Fedora postgresql test.
* Add Python 3 support to Fedora mysql tests.
* Fix uri test server for Python 3 on Fedora.
* Fix iso_extract test for Python 3 on Fedora.
* Add Python 3 support for Fedora to openssl tests.
* Fix dnf group test for Python 3 on Fedora.
* Use force with user deletion in become test.
* Reimplement iso_extract using 7zip (not requiring root)
So one of the drawbacks of the original implementation is that it required root for mounting/unmount the ISO image.
This is now no longer needed as we use 7zip for extracting files from the ISO.
* Fall back to using mount/umount if 7zip not found
As discussed with others.
Also improved integration tests.
This PR includes:
- RETURN information (since the difference between status_code and
status was confusing)
- Improvements to parameter definition (and docs)
- PEP8 compliancy
Fix 'module' object is not callable
* rhn_register: fix Python 3 compatibility
* rhn_register: update requirements
* rhn_register: add unit tests
* Add missing method name
* use a dedicated line for XML related requirements
* rhn_register: drop support for Python 2.4
* rhn_register unit tests: fix Python 3 compatibility
* refactor in order to check order of the requests
* Fix for issue ansible/ansible#27715
* Also fixing mutually exclusive check
* Updating subspec checks
These changes take into account a spec with all features enabled and do
the following tests for subspecs:
1. Test proper specs
2. Test Alias
3. Test missing required param
4. Test mutually exclusive params
5. Test required if params
6. Test required one of params
7. Test required together params
8. Test required if params with a default value
9. Test basis subspec params
10. Test invalid subsec params
* adds new filter plugins for network use cases
* adds parse_cli filter
* adds parse_cli_textfsm filter
* adds Template class to network_common
* adds conditional function to network_common
* fix up PEP8 issues
The test assumes the node has the hostname set as the inventory_hostname_short.
That's not the case in our CI, we the inventory_hostname is a UUID, returned
by the openstack dynamic inventory.
We are getting this error message:
"Advertisement-interval should be greater than or equal to four times the tx-delay".
Changing transmit delay to 2 meets that constraint.
* Add aggregate for junos modules and sub spec validation
* aggregate support of junos modules
* aggregate sub spec validation
* relevant changes to junos integration test
* junos module boilerplate changes
* Add new boilerplate for junos modules
* Fix CI issues
* Create get_exception and wildcard import code-smell tests
* Add more detail to boilerplate and no-basestring descriptions
* Remove the no-list-cmp test as the pylint undefined-variable test covers it
* s3_bucket: fix policy sorting for python3 so strings are evaluated as less than tuples.
Add tests to ensure this behavior is maintained.
* Fix s3_bucket comparison function to work on both Python 3.5 and 3.6
* s3_bucket: document that cmp_to_key is used for python 2.7.
Add another test for s3_bucket to compare policies of different sizes.
* fix pep8
* Work around code-smell grepping by not using the word 'cmp'.
* New module for managing AWS Datapipelines
* Supports create/activate/deactivate and deletion
* Handles idempotent creation by embeding the version in the
uniqueId field
* Waits for requested state to be reached, as Botocore doesn't
have waiters yet for datapipelines
* rename module, fix imports, add tags option, improve exit_json results, fix a couple bugs, add a TODO so I don't forget
fix pep8
allow timeout to be used for pipeline creation
make .format syntax uniform
fix pep8
fix exception handling
allow pipeline to be modified, refactor, add some comments, remove unnecessary imports
pipeline activation may not be in the activated state long
remove datapipeline version option
change a loop to a list comprehension
create idempotence by hashing the options given to the module minus the objects (which can be modified)
small bugfix
* data_pipeline unittests
make unittests pep8
fix bug in unittests
* remove exception handling that serves no purpose
* Fix python3 incompatibilities in datapipeline tests and add placebo fixture maybe_sleep for faster tests
Fix python3 incompatibilities in data_pipeline build_unique_id()
Don't delete a pipeline in diff_pipeline() because it's unexpected
Don't use time.time() because it causes an issue with placebo testing
re-recorded tests
fix pep8 in data_pipeline
Remove disable_rollback from tests
Make sure unique identifier is a string
re-record tests
* improve documentation and add another example
* use a placebo fixture instead of redundant code in tests
fix tests for PLACEBO_RECORD=false
* Fix data_pipeline docs
use isinstance instead of type()
fix documentation
* fix documentation
* Remove use of undefined variable from data_pipeline module and fix license
* fix copyright header
Fix adds missing imports and boilerplate for proxysql.
It also remove get_exception calls in-favor of native exception.
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Initial commit for integration of HPE OneView resources with Ansible Core. Adding FC Network and FC Network Fact modules and unit tests, and OneView base class for all OV resources.
* Add 'cacheable' param to set_fact action and module.
Used just like set_fact, except facts set with cacheable: true
will be stored in the fact cache if fact caching is enabled.
set_fact normally only sets facts in the non_persistent_fact_cache, so they
are lost between invocations.
* update set_facts docs
* use 'ansible_facts_cacheable' in module/actions result
* pop fact cacheable related items out of args/results
We dont want to use 'ansible_facts_cacheable' result item
or 'cacheable' arg as actual facts, so pop them out of the
dicts.
previously gather_subset=['!all'] would still gather the
min set of facts, and there was no way to collect no facts.
The 'min' specifier in gather_subset is equilivent to
exclude the minimal_gather_subset facts as well.
gather_subset=['!all', '!min'] will collect no facts
This also lets explicitly added gather_subsets override excludes.
gather_subset=['pkg_mgr', '!all', '!min'] will collect only the pkg_mgr
fact.
* Add module cv_server_provision for integration with Arista CloudVision Portal.
* Doc update.
* Remove shebang from test file. Update short description with company and product name.
* Update exception syntax to Python3 style.
* Remove blank line between imports.
* Remove newlines from RETURN documentation.
* Add cvprac to unittest requirements.
* Update unittest format. Add a few additional tests.
* Mock exceptions from cvprac so the library is not needed for unittests.
* Mock cvprac imports.
* Update unit tests to support python 3.5.
* Mock full cvprac library for unittests.
* Update Jinja2 import to pass updated CI checks.
* Update cvprac imports format for new CI tests.
* Add __metaclass__ and __future__.
Create preserved_copy function in basic.py to perserve file ownership.
* Add a test for template preserved backup
* Use a script to get the random names
* bytes to strings
* Remove dump of hostvars
* Stop being fancy and create a testuser instead
* Fix pep8
* set file attributes
* Pass the correct data to set_attributes_if_different
* Use -j instead -b and pass the attributes as a string instead of a list
* remove debugging message
* Use shell to softly set the attr
Fixes#24408
When parsing a vaulttext blob, use .splitlines()
instead of split(b'\n') to handle \n newlines and
windows style \r\n (CRLF) new lines.
The vaulttext enevelope at this point is just the header line
and a hexlify()'ed blob, so CRLF is a valid newline here.
Fixes#22914
We set the ansible_ssh_user and ansible_ssh_pass on the Junos
group. However, that has lower precedence than group_vars.
Commenting the group_vars so we have the creds for all Nodepool nodes
within the inventory.
Got removed in arg parsing updates. Now added back in
setup_vault_secrets().
The default value for DEFAULT_VAULT_PASSWORD_FILE was also
set to '~' for some reason, change to to no default.
Add integration tests.
* Added new module interfaces_file
* interfaces_file: added unit tests
* interfaces_file: added golden files for unit tests
* interfaces_file: moved to system modules
* interfaces_file: fixed code formatting and convention issues
* ios implementation for net_interface
* ios_interface implementation
* ios_interface integration test
* net_interface integration test for ios and other refactor
* Update boilerplate and minor refactor
* Add 2.0-2.3 facts api compat (ansible_facts(), get_all_facts())
These are intended to provide compatibilty for modules that
use 'ansible.module_utils.facts.ansible_facts' and
'ansible.module_utils.facts.get_all_facts' from 2.0-2.3 facts
API.
Fixes#25686
Some related changes/fixes needed to provide the compat api:
* rm ansible.constants import from module_utils.facts.compat
Just use a hard coded default for gather_subset/gather_timeout
instead of trying to load it from non existent config if the
module params dont include it.
* include 'external' collectors in compat ansible_facts()
* Add facter/ohai back to the valid collector classes
facter/ohai had gotten removed from the default_collectors
class used as the default list for all_collector_classes by
setup.py and compat.py
That made gather_subset['facter'] fail.
* Add aggregate parameter validation
aggregate parameter validation will support checking each individual dict
to resolve conditions for aliases, no_log, mutually_exclusive,
required, type check, values, required_together, required_one_of
and required_if conditions in argspec. It will also set default values.
eg:
tasks:
- name: Configure interface attribute with aggregate
net_interface:
aggregate:
- {name: ge-0/0/1, description: test-interface-1, duplex: full, state: present}
- {name: ge-0/0/2, description: test-interface-2, active: False}
register: response
purge: Yes
Usage:
```
from ansible.module_utils.network_common import AggregateCollection
transform = AggregateCollection(module)
param = transform(module.params.get('aggregate'))
```
Aggregate allows supports for `purge` parameter, it will instruct the module
to remove resources from remote device that hasn’t been explicitly
defined in aggregate. This is not supported by with_* iterators
Also, it improves performace as compared to with_* iterator for network device
that has seperate candidate and running datastore.
For with_* iteration the sequence of operartion is
load-config-1 (candidate db) -> commit (running db) -> load_config-2
(candidate db) -> commit (running db) ...
With aggregate the sequence of operation is
load-config-1 (candidate db) -> load-config-2 (candidate db) -> commit
(running db)
As commit is executed only once per task for aggregate it has
huge perfomance benefit for large configurations.
* Fix CI issues
* Fix review comments
* Add support for options validation for aliases, no_log,
mutually_exclusive, required, type check, value check,
required_together, required_one_of and required_if
conditions in sub-argspec.
* Add unit test for options in argspec.
* Reverted aggregate implementaion.
* Minor change
* Add multi-level argspec support
* Multi-level argspec support with module's top most
conditionals options.
* Fix unit test failure
* Add parent context in errors for sub options
* Resolve merge conflict
* Fix CI issue
* Make camel_to_snake work on capitalized plurals
`TargetGroupARNs` should become `target_group_arns`, not
`target_group_ar_ns`
Promote `camel_to_snake` to top layer function but prefix
it with an underscore.
Add tests for improved `_camel_to_snake` function.
Reduce use of `re.compile` as it makes no sense when the
compilation result is not reused.
* Remove unused LooseVersion check
* Fix PLURALs case for camel_to_snake
Also renamed EXPECTED_CAMELIZATION to EXPECTED_SNAKIFICATION
* iosxr implemetation for net_interface
* iosxr_interface implementation
* Add integration test
* iosxr_interface integration test
* net_interface intergration test for iosxr
* update boilerplate
* Add tests for group in a VPC
* Improve ec2_group output and documentation
Update ec2_group to provide full security group information
Add RETURN documentation to match
* Fix ec2_group creation within a VPC
Ensure VPC ID gets passed when creating security group
* Add test for auto creating SG
* Fix ec2_group auto group creation
* Add backoff to describe_security_groups
Getting LimitExceeded from describe_security_groups is definitely
possible (source: me) so add backoff to increase likelihood of
success.
To ensure that all `describe_security_group` calls are backed off,
remove implicit ones that use `ec2.SecurityGroup`. From there,
the decision to remove the `ec2` boto3 resource and rely on the client
alone makes good sense.
* Tidy up auto created security group
Add resource_prefix to auto created security group and delete
it in the `always` section.
Use YAML argument form for all module parameters
* win_service: added support for paused services
* change pausable service for local computers
* more fixes for older hosts
* sigh
* skip pause tests for Server 2008 as it relies on the service
* set output_dir_expanded using module result
'path' values are expanded using 'expandvars' too
* foo.txt is located in 'files' directory
* Use 'role_path' and 'connection: local' for local paths
'{{ role_path }}/tmp' is used for generated paths
* Use local connection with local paths
/tmp/ansible-test-abs-link and /tmp/ansible-test-abs-link-dir are
defined by targets/copy/files/subdir/subdir1/ansible-test-abs-link
and targets/copy/files/subdir/subdir1/ansible-test-abs-link-dir links.
* task names: add a suffix when same name is reused
* Check that item exists before checking file mode
then error message is more explicit when item doesn't exist
* Use output_dir_expanded only when necessary
* Enforce remote_user when root is required
* Fix remote path
* Use different local & remote user
this is useful when controller and managed hosts are identical
* Checks must not expect output of tested module to be right
* Use a temporary directory on the controller
* Use sha1 & md5 filters instead of hardcoded values
* Use 'remote_dir' for directory on managed host
* Workaround tempfile error on OS X
Error was:
temp_path = tempfile.mkdtemp(prefix='ansible_')
AttributeError: 'module' object has no attribute 'mkdtemp'"
* initial commit for win_group_member module
* fix variable name change for split_adspath
* correct ordering of examples/return data to match documentation verbiage
* change tests setup/teardown to use new group rather than an inbult group
* ACI module_utils library for ACI modules
This PR includes:
- the ACI argument_spec
- an aci_login function
- an experimental aci_request function
- an aci_response function
- included the ACI team
* New prototype using ACIModule
This PR includes:
- A new ACIModule object with various useful methods
* prepare_ovs call gather facts
As we are no longer using run_ovs_integration_tests.yml we need to
explicitly gather facts so we can call the correct package manager.
* typo
* Enable specific tests (this lets us disable a group and then
enable a particular test inside of it)
* Comment out tests in the enable and disable files
Made the following changes:
* Removed wildcard imports
* Replaced long form of GPL header with short form
* Removed get_exception usage
* Added from __future__ boilerplate
* Adjust division operator to // where necessary
For the following files:
* web_infrastructure modules
* system modules
* linode, lxc, lxd, atomic, cloudscale, dimensiondata, ovh, packet,
profitbricks, pubnub, smartos, softlayer, univention modules
* compat dirs (disabled as its used intentionally)
Absolute path trailing slash handling in absolute directories
find_needle() isn't passing a trailing slash through verbatim. Since
copy uses that to determine if it should copy a directory or just the
files inside of it, we have to detect that and restore it after calling
find_needle()
Fixes#27439
Fixes#13243
** Add --vault-id to name/identify multiple vault passwords
Use --vault-id to indicate id and path/type
--vault-id=prompt # prompt for default vault id password
--vault-id=myorg@prompt # prompt for a vault_id named 'myorg'
--vault-id=a_password_file # load ./a_password_file for default id
--vault-id=myorg@a_password_file # load file for 'myorg' vault id
vault_id's are created implicitly for existing --vault-password-file
and --ask-vault-pass options.
Vault ids are just for UX purposes and bookkeeping. Only the vault
payload and the password bytestring is needed to decrypt a
vault blob.
Replace passing password around everywhere with
a VaultSecrets object.
If we specify a vault_id, mention that in password prompts
Specifying multiple -vault-password-files will
now try each until one works
** Rev vault format in a backwards compatible way
The 1.2 vault format adds the vault_id to the header line
of the vault text. This is backwards compatible with older
versions of ansible. Old versions will just ignore it and
treat it as the default (and only) vault id.
Note: only 2.4+ supports multiple vault passwords, so while
earlier ansible versions can read the vault-1.2 format, it
does not make them magically support multiple vault passwords.
use 1.1 format for 'default' vault_id
Vaulted items that need to include a vault_id will be
written in 1.2 format.
If we set a new DEFAULT_VAULT_IDENTITY, then the default will
use version 1.2
vault will only use a vault_id if one is specified. So if none
is specified and C.DEFAULT_VAULT_IDENTITY is 'default'
we use the old format.
** Changes/refactors needed to implement multiple vault passwords
raise exceptions on decrypt fail, check vault id early
split out parsing the vault plaintext envelope (with the
sha/original plaintext) to _split_plaintext_envelope()
some cli fixups for specifying multiple paths in
the unfrack_paths optparse callback
fix py3 dict.keys() 'dict_keys object is not indexable' error
pluralize cli.options.vault_password_file -> vault_password_files
pluralize cli.options.new_vault_password_file -> new_vault_password_files
pluralize cli.options.vault_id -> cli.options.vault_ids
** Add a config option (vault_id_match) to force vault id matching.
With 'vault_id_match=True' and an ansible
vault that provides a vault_id, then decryption will require
that a matching vault_id is required. (via
--vault-id=my_vault_id@password_file, for ex).
In other words, if the config option is true, then only
the vault secrets with matching vault ids are candidates for
decrypting a vault. If option is false (the default), then
all of the provided vault secrets will be selected.
If a user doesn't want all vault secrets to be tried to
decrypt any vault content, they can enable this option.
Note: The vault id used for the match is not encrypted or
cryptographically signed. It is just a label/id/nickname used
for referencing a specific vault secret.
* Revert "Update conventions in azure modules"
This reverts commit 30a688d8d3.
* Revert "Allow specific __future__ imports in modules"
This reverts commit 3a2670e0fd.
* Revert "Fix wildcard import in galaxy/token.py"
This reverts commit 6456891053.
* Revert "Fix one name in module error due to rewritten VariableManager"
This reverts commit 87a192fe66.
* Revert "Disable pylint check for names existing in modules for test data"
This reverts commit 6ac683ca19.
* Revert "Allow ini plugin to load file using other encoding than utf8."
This reverts commit 6a57ad34c0.
We do want to allow certain from __future__ imports in modules that make
it easier to code compatible python2 and python3. Note that
unicode_literals is specifically left out and should never be allowed.
Now that python-3.4+ allows u"" there's no good reason to use
unicode_literals.
Also switch tables in the validate_modules documentation to simple table format
- New option for ini plugins: encoding
- Add a new option encoding to _get_file_contents
- Use replace option in test/runner/lib/util.py when calling decode on stdout/err
output when diff have non-utf8 sequences
* Module argument_spec now accepts a callable for the type argument, which is passed through and called with the value when appropriate. On validation/conversion failure, the name of the callable (or its type as a fallback) is used in the error message.
* adds basic smoke tests for custom callable validator functionality
* changed collection arg to argregate on 2.4 network modules
* replace users with aggregate in eos_user, junos_user, nxos_user
* added version_added to places where we replaced users with aggregate in the docs
* fix ios_static_route test
* update tests to reference aggregate instead of collection/users
* Enable the pylint no-name-in-module check. Checks that identifiers in
imports actually exist. When we do this, we also have to ignore
_MovedItems used in our bundled six. This means pylint won't check
for bad imports below ansible.module_utils.six.moves but that's
something that pylint punts on with a system copy of six so this is
still an improvement.
* Remove automatic use of system six. The exec in the six code which
tried to use a system library if available destroyed pylint's ability
to check for imports of identifiers which did not exist (the
no-name-in-module check). That test is important enough that we
should sacrifice the bundling detection in favour of the test.
Distributions that want to unbundle six can replace the bundled six in
ansible/module_utils/six/__init__.py to unbundle. however, be aware
that six is tricky to unbundle. They may want to base their efforts
off the code we were using:
2fff690caa/lib/ansible/module_utils/six/__init__.py
* Update tests for new location of bundled six Several code-smell tests
whitelist the bundled six library. Update the path to the library so
that they work.
* Also check for basestring in modules as the enabled pylint tests will
also point out basestring usage for us.
The OpenSSLObject class has been merged[1]. This commit makes the
openssl_privatekey rely on this class and standardize the way openssl
module should be written.
Co-Authored-By: Christian Pointner <cpointner@mgit.at>
[1] https://github.com/ansible/ansible/pull/26945
* Nuage module and unit tests with requested changes
* Cleanup of imports
* Adding check on python version
* Adding import try and catch wrappers
* Cleanup of requirements and adding integration tests
* Using pypi package for simulator
* Cleanup of requirements and adding integration tests
* Adding aliases for integration tests
* Adding module to import sanity test skip list
* Revert "Adding module to import sanity test skip list"
This reverts commit eab23af8c5ca7c503af63c05610b5db66d31fae4.
* Adding check for importlib and cleanup of requirements
Crypto namespace contains the openssl modules. It has no integration
testing as of now.
This commits aims to add integration tests for the crypto namespace.
This will make it easier to spot breaking changes in the future.
This tests currently apply to:
* openssl_privatekey
* openssl_publickey
* openssl_csr
* Move tests to their own file
* Port to a pytest parametrized test so it's easier to define new tests
* Now that _symbolic_mode_to_octal is a classmethod, we don't have to
instantiate an AnsibleModule to test it.
* Add tests for #14994, having more than one operator per role and umask
chmod
* vmware_host: Small fixes and docs updates
This PR includes:
- A fix to no longer require a datacenter folder for adding a host
- Documentation improvements
- Ensure imports are specific
* Update vmware_host
Fix adds following:
* Update logic in vmware_host
* Update example documentation
* Added test case for vmware_host
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Add example of templating inline using copy module
The **copy** module documentation implies that `content:` only works
for 'simple values' and for complex stuff you need the **template**
module, but that is an understatement. You can use **copy** to template
anything you desire.
So I changed the wording, added an example, and also added a note
to the template module that the **copy** module could be used for
'inline templating'.
This fixes#19741.
Sometimes MacOSX's pwd doesn't return an expanded path. Not sure why
but this test is still valid if we expand it via a playbook filter so
go ahead and do that.
* We need a directory walker that can handle symlinks, empty directories,
and some other odd needs. This commit contains a directory walker that
can do all that. The walker returns information about the files in the
directories that we can then use to implement different strategies for
copying the files to the remote machines.
* Add local_follow parameter to copy that follows local symlinks (follow
is for remote symlinks)
* Refactor the copying of files out of run into its own method
* Add new integration tests for copy
Fixes#24949Fixes#21513
* add unit test: nested dynamic includes
* nested dynamic includes: avoid AnsibleFileNotFound error
Error was:
Unable to retrieve file contents
Could not find or access 'include2.yml'
Before 8f758204cf, at the end of
'path_dwim_relative' method, the 'search' variable contained amongst
others paths:
'/tmp/roles/testrole/tasks/tasks/included.yml' and
'/tmp/roles/testrole/tasks/included.yml'.
The commit mentioned before removed the last one despite the method
docstrings specify 'with or without explicitly named dirname subdirs'.
* add integration test: nested includes
* list failed tests for iosxr
* list failed tests for ovs
* list failed tests for junos
* list failed tests for ios
* list failed tests for eos
* list failed tests for nxos
* list failed tests for vyos
* Revert change to docker_common as it's not as good as the try: except fix
* limit docker_volume fix to ImportErrors
* fix docker_secret i nthe same way
* Remove docker_secret from import tests
* Ensure that include_role properly fires handlers
include_role needs to ensure that any handlers included
with the role are added to the _notified_handler and
_listening_handler lists of the TaskQueueManager, otherwise
it fails when trying to run the handler.
Additionally, the handler needs to be added to the
PlayIterator's `_uuid_cache` or it fails after running
the handler
Add more uuid debug statements - this code was hard
to debug with existing debug statements, so add more
uuid information at little additional output cost.
Fixes#18411
* Add tests for include_role handlers
Tests for #18411
* Adding ciscowlc_command module and unit tests.
* Adding __init__.py for unit test.
* Fixing PEP8 W503.
* Renaming module from ciscowlc_command to aire_command.
* Renaming aire_command to aireos_command.
* Only use `git verify-tag` when verifying annotated tags
The command `git verify-tag` only applies to annotated tags. When
verifying lightweight tags, which are more similar to non-moving
branches, one has to use `git verify-commit` instead.
Using ':' as a separator is appropriate since that is one of the
characters not allowed in a Git reference name.
See also https://www.kernel.org/pub/software/scm/git/docs/git-check-ref-format.html
* Improve testing of the Git module's gpg verification
* Add default description string to vyos_interface
* If `state=up` it should remove the `disable` configuration
for interface. However, if no other interface parameter is configured
this ends up deleting the interface itself which is not the desired
behaviour. Hence adding a default description field to avoid such
scenario's.
* Minor changes
* Add default description to aggregate
* Adding aruba_command module along with unit tests.
* Fixing PEP8 E303 too many blank lines.
* Adding default for timeout.
* Removing unused arguments. Moving default for timeout argument. Fixing cliconf to find hostname.
* Fixing PEP8 E302.
* Add unit tests for parted module
Test the current expected behavior of the module:
- mock parted() and get_device_info()
- Use some of the examples and test the 'script' passed to parted.
- mock check_parted_label() to return false, as if parted version is > 3.1
- assert get_device_info output is correct
Current implementation of the module runs parted several time while going
through all parameters (flags, name, ...). Between calls it uses get_device_info
to update the dictionary. Use check_mode for some of the tests to force module
to go through all the parameters even is dictionary is not updated.
* test_parted.py: add "name" param into expected results
since 78fff751ab, parse_partition_info
fetch the partition name. This commit adds 'name' key and value into
the expected results.
Consolidate the module_utils, constants, and config functions that
convert values into booleans into a single function in module_utils.
Port code to use the module_utils.validate.convert_bool.boolean function
isntead of mk_boolean.
* aci_rest: New module to access Cisco ACI
This PR includes:
- Relicense as GPLv3+
- Check-mode support
- Cosmetic changes to documentation
- Examples in YAML format
- Removal of incorrect requirements (for this module)
- Do not log passwords
- Implement native fetch_url instead of requests
- Use standard hostname, username and password parameters
- Add alias src for parameter config_file
- Add mutual exclusive content option for inline data (and show some inline examples)
- Add timeout parameter
- Add validate_certs parameter
- Handling ACI result output (identical for JSON as XML input)
- Parse/expose ACI error output to user
* Lower case method, add use_ssl, Use python dicts
This commit includes:
- Use lowercase method names
- Add `use_ssl` parameter (not the `protocol` parameter)
- Use a python dict for the request data (not a JSON string)
- Documentation improvements
* Ensure one of 'content' or 'src' is provided
* Fix issue with totalCount being a string in JSON
This fixes the problem with JSON output where totalCount is a string and
not an integer.
This fixesjedelman8/aci-ansible#7
* Improve code documentation
* Improve error handling and module response
* Small typo
* Improve documentation and examples
* Keep protocol parameter, but deprecate it
* Extrude aci functions from module_utils
* aci_rest: Add unit tests
The 'admin' word was being masked by Ansible as potential cred.
Let's just use network-operator since we are just testing here
we can create users in aggregate.
* Use Boto3 for ec2_group
Currently boto doesn't support ipv6. To support ipv6 in ec2_group, we need boto3.
boto3 has significant API changes, which caused more re-factoring for ec2_group module.
Added additional integration test to test_ec2_group role.
* Follow the standard for boto3 ansible
Fixed imports. Use boto3 ansible exception with camel_dict_to_snake_dict.
Refactored the call to authorize/revoke ingress and egress.
* Removed dependancy with module ipaddress
Added new parameter called cidr_ipv6 for specifying
ipv6 addresses inline with how boto3 handles ipv6 addresses.
* Updated integration test
* Added ipv6 integration test for ec2_group
* Set purge_rules to false for integration test
* Fixed import statements
Added example for ipv6.
Removed defining HAS_BOTO3 variable and import HAS_BOTO3 from ec2.
Cleaned up import statements.
* Fixed exception handling
* Add IAM permissions for ec2_group tests
Missing AuthorizeSecurityGroupEgress necessary for latest tests
* Wrapped botocore import in try/except block
Import just botocore to be more similar to other modules
* junos implementation for net_l3_interface module
* junos_l3_interface implementation
* junos_l3_interface integration test
* net_l3_interface integration test for junos
* Fix module name typo
* All integration commands support --continue-on-error
* The network-integration command supports --inventory
* Add landing page for compile test docs.
* Add bot documentation links.
* once smoketest is implemented in CI, will run a much smaller set of Windows module integration tests for changes to module_utils or connection subsystems
* win_secedit: Added module with tests/diff mode
* fixed up test issues
* Added missing return value
* change for win_secedit based on review
* updated win_security_policy examples for rename
These integration tests were used for testing the exact behaviour of
Ansible for YAML-style syntax and key=value syntax.
This includes fixes to win_shortcut (as `src` can be a URL too)
* win_regedit: rewrite to support edge cases and fix issues
* fix up byte handling of single bytes and minor doc fix
* removed unused method
* updated with requested changes
* vyos implementation for net_interface module
* vyos_interface implementation module
* vyos_interface integration test
* net_interface integration test for vyos
* Change collection to aggregate
* Fix the editable condition into pip module (#19028)
* Add editable to tests
Default changed to False, so now editable: True is needed explicitly in
tests
* Revert "Temporarily disable zypper tests due to timeouts."
This reverts commit 25f943e22f.
* Revert "Disable zypper_repository tests due to errors."
This reverts commit 5833197a19.
* until we can have a robust internal source for chocolatey (and/or PSScriptAnalyzer sanity tests), we want to at least make sure that the module is syntactially valid.
* vmware_guest: fixes for cache objects and datacenter association
* find_all_objs was only looking for datastores
* Clear the result if it's datacenter is not correct.
* Re-enable pyvmomi installation
Addresses #25011
Addresses #26511
* add first, last and next usable
* add usable ip filters
* add size usable, range usable and wildcard
* add ip prefix and netmask filter
* add network formatting and check if ip in subnet
* clean up order, add comments
* fix pep8
* update format by index
* clean up and updates from jmcgill298
* Slight refactor on vmware_guest to fix path searching and vcsim compatibility.
* Clean up pep8 errors
* Fix more pep8 issues
* Add assertions
* Remove debug failure
* Update docsting for folder with examples
* Revise _get_vm_prop based on suggestions
* Implement folder walker to find a folder path
* More enhancements for datacenter->datacenter cloning
* Fix a few pep8 issues
* Remove useless check for subclass of None
* Fix a few suggestions
* Move serialize_spec to a util function
* Group exceptions together
Fixes#25011
* add connection plugin for buildah
Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>
* fixup
* create a method to invoke buildah
* mount container filesystem persistently so we can access it
during put and fetch
* use copyfile function for copying files
Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>
* revert tests
Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>
* fixup
Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>
* progress
(this will be squashed into a single commit before merge)
* add docs for the conn plugin
* fix issue invoking the integration tests
* add a way to invoke commands inside the container as a different user
Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>
* fix shellcheck warning
Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>
* adds more intelligent save logic and diff to network config modules
* adds sha1 property to NetworkConfig
* adds new argument save_when to argument_spec
* adds new argument diff_against to argument_spec
* adds new argument intended_config to argument_spec
* renames config argument to running_config with alias to config
* deprecates the use of the save argument
* before and after now work with src argument
* misc module clean
Modules updated
* nxos_config
* ios_config
* eos_config
Most notably this makes the save mechanism more intelligent for config
modules for devices that need to copy the ephemeral config to
non-volatile storage.
The diff_against argument allows the playbook task to control what the
device's running-config is diff'ed against. By default it will return
the diff of the startup-config.
* removes ios_config from pep8/legacy_files.txt
* extends the ignore lines argument to the module
* clean up CI errors
* add missing list brackets
* fixes typo
* fixes unit test cases
* remove last line break when returning config contents
* encode config string to bytes before hashing
* fix typo
* addresses feedback in PR
* update unit test cases
Module was importing '*' from facts to get to TimeoutError
but that has moved to facts.timeout, so import is updated.
Also rm old style imports to new style imports at the start
of the module.
'signal' py module was used and referenced but never imported,
presumably it was using the 'signal' previously imported into
module_utils.facts. Now imported directly.
'AnsibleModule' was also from a * import, so now imported directly.
A ref to 'module' was in _delete_disks_when_detached(), so now it
is updated to raise an AzureException() with its message, and
let its caller catch it and call module.fail_json()
It was in lib/ansible/modules/system/setup.py since it
was the only thing using it, but move it back to module_utils
and add a ansible_collector.get_ansible_collector() to build
a facts collector just like the one used by setup.py
mv test_setup.py -> test_ansible_collector.py
All the code it was testing is now in ansible_collector
rm code to create 'ansible_facts' subkey from namespace
Just leave it up to the caller to do, and just return a
flat dictionary from AnsibleFactCollector.collect()
* junos_linkagg implementation and junos modules refactor
* junos_linkagg implementation
* junos_linkagg integration test
* net_linkagg integration test for junos
* decouple `load_config` and `commit` operations,
to allow single commit (in case on confirm commit) and
to perform batch commit (multiple `load_config` followed by single
`commit`)
* Other related refactor
* Fix CI issues
* Fix unit test failure
* invalid drive letter breaks new Powershell path validation (this is a separate issue that needs to be solved)
* test also exposes a minor bug in win_get_url where a nonexistent directory in the root of a drive that exists passes the "does the parent dir exist" check and causes DownloadFile to bomb (this should be fixed by dag's rewrite of win_get_url in https://github.com/ansible/ansible/pull/26612)
* Feature #2731: added postgres import and dump
* Feature #2731: be more permissive of arguments
```
hacking/test-module -m ./ppostgresql_db.py -a "db=example state=dump target=/tmp/out"`
```
failed previously since host, user, and port were required as keywords
in the pg_dump / pg_import methods.
* Feature #2731: fixed doc string for validate-modules
```
$ ansible-validate-modules database/postgresql/
```
now passes.
* Feature #2731: disable 'password' for dump/restore
* Feature #2731: bump added version to 2.3
* Feature #2731: replace db_import with db_restore
* Feature #2731: add missing version description
* Feature #2731: fix 'state' description
* Feature #2731: fix pep8 issues
* Feature #2731: put state documentation in a single string
* Bump added version from 2.3 to 2.4
* Fix pep8 and pylint errors
* Attempt yaml formatting of documentation string
* Add integration tests for postgres_db:dump/restore
* Update dump/restore logic to support new kw-args
Also attempt to support password; integration tests are
still failing.
* Revert to postgres user for dump/restore
Passing PGPASSWORD is not working for subprocesses. For the
moment, reverting to the strategy of failing if login_password
is set and using `postgres` for all testing of dump/restore.
* Various cleanups to have tests passing
* Working tests for {sql,tar} x {,bz2,gz,xz}
* Use pg_user to support FreeBSD
* Revert login_ prefixes and re-enable password support
All `login_` keywords are mapped to their non-prefix versions
so the previous changes were effectively using `postgres` for
all actions. With the proper keywords, PGPASSWORD-passing to
the subprocess is now working.
* Optionally add password
environ_update doesn't handle None values in the
dictionary to be added to the environment. Adding
check.
* Quick fixes
* Refactor login arguments after fixes from pchauncey
The fixes introduced by pchaunchy pointed to further issues
(like no --dbname on PG<=9.2) with the login parameters. This
refactors them and adds further tests.
Note: this will still not pass integration tests due to a further
issue with pg_dump as a non-admin user:
pg_restore: [archiver (db)] Error while PROCESSING TOC:
pg_restore: [archiver (db)] Error from TOC entry 1925; 0 0 COMMENT EXTENSION plpgsql
pg_restore: [archiver (db)] could not execute query: ERROR: must be owner of extension plpgsql
* Introduce target_opts for passing limiting dumped/restored schemas
The current integration tests (PG version and template DBs) don't
permit a regular user (`{{ db_user1 }}`) access to plpgsql causing
restores to fail. By adding an option for passing arbitrary args to
pg_dump and pg_restore, testing is made easier. This also paves the
way for `-j` usage, once the PG version is bumped.
The iApp service module worked fine previously, but this patch
adds enhancements to it to include more fields that can be
specified when creating iapp services.
* add enhanced run option support for win_scheduled_task
* changed run_level option to runlevel
* correct merge conflicts since task path fix
* changed run_level option to runlevel
* changed do_not_store_password to store_password, and other minor fixes
* conditional logic swap, and documentation change for password
* postgresql_user module - transaction logic hacks to allow recovery from failed select
* postgresql_user - PEP8 and style fixes to make debugging easier
* postgresql_user - move password changing logic to separate function
* postgresql_user - trap failure in case where there is no access to pg_authid
* postgresql_user - further PEP8 fixes
* postgresql_user - Simplify password change logic and improve imports according to suggestions from PR review
* postgresql_user - Eliminate pep8/blank line errors introduced in merge
* Check behaviour when pg_authid relation isn't readable
TASK [postgresql : Normal user isn't allowed to access pg_authid relation:
password comparison will fail, password will be updated] ***
An exception occurred during task execution. To see the full traceback,
use -vvv. The error was: psycopg2.ProgrammingError: permission denied
for relation pg_authid
* Don't reintroduce passlib, remove useless query
This PR includes:
- Checkmode improvements
- Integration tests
- A fix for python3
- PEP8 fixes
This backports improvements from the win_wakeonlan module.
* Implementation of junos_static_route module
* junos implementation of net_static_route
* integration test for junos_static_route
* integration test for junos net_static_route
* Minor change
* Doc change
* Fix CI issue
* Move Config classes from executor.py to config.py.
* Move Environment and Test config to config.py.
* Move Coverage/CoverageReport Config to config.py.
* Clean up type hints.
* junos_user declarative module changes
* Active/Deactivate support
* junos_user integration test
* net_user intergration test for junos
* Add version_added for active param
restored 'rc' inspection but only when failed is not specified
removed redundant changed from basic.py as task_executor already adds
removed redundant filters, they are tests
added aliases to tests removed from filters
fixed test to new rc handling
* as a result of recent core engine changes to ignore rc, modules are responsible to set `failed` on nonzero RC if they want that behavior
* the `failed` filter currently triggers on nonzero RC, which caused the tests to false-pass
* updated tests to explicitly check both rc and failed keys, as well as using the failed filter.
Thanks to the report by Matt Clay at [1], yamllint now supports Python
2.6. Tests were enabled on Travis for 2.6 to make sure there will be no
regressions in the future.
[1]: https://github.com/adrienverge/yamllint/issues/55
Currently, when using this test, it fails with the following error
message:
> AttributeError: 'NoneType' object has no attribute 'rfind'
This is because there is no _raw_params value for parent_include.args
here
https://github.com/ansible/ansible/blob/devel/lib/ansible/playbook/included_file.py#L104
This commit ensure the value is specified so it can be reused and hence
not fail at this specific line.
Previously tests would stop running on the first failure.
With this PR we move onto running the tests for the next module. This
allows DCI runs to have a fuller picture of what's broken.
This only effects manual and DCI runs, it doesn't effect Shippable runs
as this the modules are tested directly.
* Fix ansible-doc traceback when a plugin doesn't parse correctly
* Change extract_metadata ivocation to take either an ast or source
code. When given source code, it can find file offsets for the start
and end of dict. When given the ast, it is quicker as it doesn't have
to reparse the source. Requires changing the call to the function to
use a keyword arg.
* Fix reading of metadata to find the last occurrence of
ANSIBLE_METADATA instead of the first.
* Add some more unittests to get closer to complete coverage
* fixes become_method: runas for unprivileged users
* sets permissions on tempdir appropriately
* allows automatic system environment generation for new token (old Process.Start way prevents this)
* add basic become runas tests
* correct, cleanup & simplify dwim stack
latlh chIS logh HeS qar wej chel laD
better errors
update find_file to new exception
* addressed latest comments
* test should not use realpath as it follows symlink
this fails when on OS X as /var is now a symlink to /private/var
but first_found was not supposed to follow symlinks
* Unittests for extracting metadata from plugins
* Port plugin_docs to use the generic extract_metadata function
* Make the helper functions seek_end_of{string,dict} private
* win_say: Fix issue, add integration test
This PR includes:
- Make speed_speech an integer parameter
- Test for empty parameters too
- Add integration tests
* Improve the $speech_speed parameter handling
As requested
include_vars will now also return a key 'ansible_included_var_files'
which contains the list of files that were successfully loaded.
This is useful information and, amongst other things, a way for users
to know exactly what files were included when debugging their
playbooks.
This also allows us to improve the integration tests around
include_vars.
So in an effort to verify if Windows modules are feature complete
compared to the python equivalent, I stumbled upon these differences.
This PR includes:
- Add missing 'data' option from documentation
- Simplify ping module
- Update integration tests to test exception
So I thought I fixed it before, but there's still one location where
the `rc` value is influential to decide whether a task failed or not.
We already established in #24867 that it is up to the module to decide
what the return code actually means, not the task executor. We modified
the existing modules to move that logic into the module (eg. for
command, shell, etc.)
This relates to the integration tests of win_robocopy, where different
return codes have different meanings:
- 0 -- No files copied.
- 1 -- Files copied successfully! (changed)
- 2 -- Some Extra files or directories were detected. No files were copied. (warning)
- 3 -- (2+1) Some files were copied. Additional files were present. (changed)
- 4 -- Some mismatched files or directories were detected. Housekeeping might be required! (changed + warning)
- 5 -- (4+1) Some files were copied. Some files were mismatched. (changed + warning)
- 6 -- (4+2) Additional files and mismatched files exist. No files were copied. (warning)
- 7 -- (4+1+2) Files were copied, a file mismatch was present, and additional files were present. (changed + warning)
- 8 -- Some files or directories could not be copied! (changed + failed)
- 9 - 15 -- Fatal error. Check log message! (failed)
- 16 -- Serious Error! No files were copied! Do you have permissions to access $src and $dest? (failed)
This also fixes#24652
* Extend tests to have multiple device representations
* Move filepath munging to nxos_module
* Device needs to be kwarg so we can leave it off
* Update other nxos tests
* Update tests that fell through
Make pyca/cryptography the preferred backend for cryptographic needs (mainly vault) falling back to pycrypto
pyca/cryptography is already implicitly a dependency in many cases
through paramiko (2.0+) as well as the new openssl_publickey module,
which requires pyOpenSSL 16.0+. Additionally, pyca/cryptography is
an optional dep for better performance with vault already.
This commit leverages cryptography's padding, constant time comparisons,
and CBC/CTR modes to reduce the amount of code ansible needs to
maintain.
* Handle wrong password given for VaultAES format
* Do not display deprecation warning for cryptography on python-2.6
* Namespace all of the pycrypto imports and always import them
Makes unittests better and the code less likely to get stupid mistakes
(like using HMAC from cryptogrpahy when the one from pycrypto is needed)
* Add back in atfork since we need pycrypto to reinitialize its RNG just in case we're being used with old paramiko
* contrib/inventory/gce: Remove spurious require on pycrypto
(cherry picked from commit 9e16b9db275263b3ea8d1b124966fdebfc9ab271)
* Add cryptography to ec2_win_password module requirements
* Fix python3 bug which would pass text strings to a function which
requires byte strings.
* Attempt to add pycrypto version to setup deps
* Change hacking README for dual pycrypto/cryptography
* update dependencies for various CI scripts
* additional CI dockerfile/script updates
* add paramiko to the windows and sanity requirement set
This is needed because ansible lists it as a requirement. Previously
the missing dep wasn't enforced, but cryptography imports pkg_resources
so you can't ignore a requirement any more
* Add integration test cases for old vault and for wrong passwords
* helper script for manual testing of pycrypto/cryptography
* Skip the pycrypto tests so that users without it installed can still run the unittests
* Run unittests for vault with both cryptography and pycrypto backend
* Add new windows module win_psmodule
* Add checkmode, allow_clobber parameter, integration tests
* Add aliases, replace win_raw with win_shell
* restore original test_win_group1.yml, add powershel version test
* fix var type
* add conditional on assert
* integration tests conditional tasks review
* documentation fix, test fix, adds result.change
* fix yml
* fix railing whitespace
* add nuget_changed and repository_changed in result
There are too many possible special cases for Ansible to be able to
precheck known_hosts files without introducing all kinds of false
failures.
* Alternative known_hosts paths
* Alternative host name aliases
* ssh host certificates
* SSHFP + DNSSEC
Fixes#24860
The py-psycopg2 package now requires postgresql95-server instead of
postgresql93-server. Installing py-psycopg2 will automatically remove
postgresql93-server if it is installed, breaking integration tests.
* Sort packages to install.
* Add python-argparse to centos6 docker image.
* Add gcc and python dev lib to docker images.
* Add python cryptography to docker images.
* Add coverage using pip instead of OS packages.
* Update old pip versions in docker images.
* Exclude */pyshared/* from coverage reporting.
* Add junos_system declartive module and other related change
* junos_system declartive module
* integration test for junos_system
* integration test for net_system (junos platform)
* pep8 fixes for junos modules
* move to lxml from elementree for xml parsing as it support
complete set of xpath api's
* other minor changes
* Fix CI and doc changes
* Fix unit test failures
* Fix typo in import
* Fix import issue for py2.6
* Add missed Element in import
This is a cleanup of the win_uri module to make it feature-complete.
This PR includes:
- Added check-mode support
- Add as many options from the uri module as possible
- Added creates
- Added follow_redirects
- Added maximum_redirection
- Added password
- Added removes
- Added return_content
- Added status_code
- Added timeout
- Added user
- Added validate_certs
- Fixed list-handling for comma-separated strings
- Added basic integration tests (should come from uri module)
* added integration tests for apt upgrade
changed version number for hello to 2.6 so that it works with Ubuntu 12.04
prevent tests from checking if aptitude is installed on non ubuntu systems
changed ordering on when statements for safe and full upgrade types so that the OS check happens before the aptitude check
added integration tests for apt upgrade
changed version number for hello to 2.6 so that it works with Ubuntu 12.04
* Moved additions to tasks/main.yml to make revisions easier. Changed tasks to multiline format
* added integration tests for yum repository
* fixed escaped single quote
* extended yum_repository tests to include fedora
* removed unused variable file
* added check for return values
* replaced escaped double quotes with single quotes, switched to lookup for reading repofile
* Add junos_banner declartive module
* junos_banner implementation
* Integration test for junos_banner
* Integration test for net_banner (junos)
* Minor fixes
* Minor doc change
* Add net_interface declartive module
* Add net_interface module
* Add junos_interface implementation module
* Other minor changes
* Add integration test
* Integration test for net_interface
* Integration test for junos_interface
* Fix CI failures
* Documentation changes
* adds new common functions for declarative intent modules
* adds Entity and EntityCollection
* adds dict_diff and dict_combine
* update for CI PEP8 compliance
* more CI PEP8 fixes
* more PEP8 CI clean up
* refactors the lambda assignments into top level classes
this is to be in compliant the PEP8 CI sanity checks
* one last pep8 ci fix
* Add nxos_nxapi tests
* Simple changes to nxos_nxapi
* Move validation to check_args
* Don't mark protocol change unless change is requested
* Add different regex to handle HTTP{,S} ports on a different version of nxos
finished normalizing of path handling
removed overloaded '-p' from init_paths option, it is for role_paths
removed expand_tilde and get_opt methods as both were redundant, adjusted rest of code
updated tests to match
* Add vyos_user implementation module
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
* Integration test for vyos_user
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
* Make state absent work
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
* Unit test for vyos_user
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
* Standardize user names
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
* Modify integration test with idempotent case
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
* Add role as alias to level
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
* Allow EC2-VPC instances to update SG
make ec2 pep8
* use sets instead of loop and a break
* bring things in an indentation level
* Use to_text instead of str, text_type instead of basestring, - instead of difference
* basestrings not unicode
* simplifying syntax
* Start of ansible config project
moved configuration definitions to external yaml file vs hardcoded
* updated constants to be a data strcutures that are looped over and also return origin of setting
changed to manager/data scheme for base classes
new cli ansible-config to view/manage ansible configuration settings
* prints green for default/unchanged and yellow for those that have been overriden
* added list action to show all configurable settings and their associated ini and env var names
* allows specifying config file to see what result would look like
* TBD update, edit and view options
removed test for functions that have been removed
env_Vars are now list of dicts
allows for version_added and deprecation in future
added a couple of descriptions for future doc autogeneration
ensure test does not fail if delete_me exists
normalized 'path expansion'
added yaml config to setup packaging
removed unused imports
better encoding handling
updated as per feedback
* pep8
* allows win_scheduled_task to support adding and removing task paths
* fix line length for documentation
* added integration tests for path creation and removal
* removing ability to remove TaskPath if a task isn't removed. also removed superfluous line of code in Invoke-TaskPathCheck function
* Various fixes to VM customizations (from template)
This patch implements:
- New find_obj() function from vmware.py replacing get_obj()
- Implement proper resource_pool selection
- Fix productId implementation (was not working)
- Ensure that we are not changing anything that is not mandatory (hostName, orgName, fullName)
This is an alternative proposal to #24283
This does not fix#19860 yet though.
For our use-case, we do not want to customize the network information (or any information in fact).
What is used in the template should remain intact.
* Added find_obj() function
* Fix the returned object-list (unused yet)
* Small improvement
* Support DHCP type and fix customizations
* Small fix
* Support resource_pool also for reconfiguring VM
* Remove redundant
* Fix short hostname, specific resource_pool, PEP8
* Improve docs and examples
* Fix missing hostsystem
* Make folder absolute path
* Improve docs, add missing 'mac'
* Add more mount point statvfs info including sizes
Based on https://github.com/ansible/ansible/pull/12073
facts.utils.get_mount_size() now returns a dict of most
of the posix statvfs data, including block_size and inode
counts.
Update the facts.hardware classes that use get_mount_size() to
use the new info by mount_info.update(mount_statvfs_inof) to merge.
* add back unit tests for LinuxHardware mount/fs facts
* add test cases for facts.utils.get_mount_size
* trying to delete a nonexistent bucket should not fail
* Improve error handling for deleting s3 bucket
* Allow successful deletion
* Add test for deleting a nonexistent bucket
rename integration test target from s3 to aws_s3
* ec2_asg and ec2_asg_facts module improvements
Return target group information for both ec2_asg and ec2_asg_facts
modules
Provide RETURN documentation for ec2_asg module
PEP8 fixes for ec2_asg_facts
* ec2_asg: use pagination when describing target groups
In case an ASG has 100s of target groups, ensure that
we get the full result using build_full_result
* WIP Add openvswitch_db integration tests
* Add ovs.yaml playbook entrypoint for OVS
* Add test case pattern filtering
* Clear out aliases for now
Running tests on posix group just fails, containers do not have ovs
* Factorize tests related to no_password_change using an include task
* Refactor: deduplicate tasks
* postgresql_user: test 'expires' parameter
* Change 'valid until' even it's the only updated field
* value is changed when another value is provided
* value isn't returned when unset
* Remove unused variable
* psycopg2.extras.DictRow is able to handle comparison
* postgresql_user: simplify helper method
* postgresql_user: define variable just before using it
* Fix comparison between user input and applied configuration
* new test: adding an invalid attribute
* Refactor, add cleaning task
* Check that using same attribute a 2nd time does nothing
* Always try to remove created user
* postgresql_user: fix pep8
* Support NetBSD 7.1+ style ifconfig -a output
network facts on NetBSD after 7.1 cvs would fail
because of format changes in 'ifconfig -a' output.
update code to support new and old format.
add unit tests for both based on
examples from Bruce V Chiarelli.
* wrap use of interfaces.keys() in list() for py3 compat
* sort interface ids for stability
* Fix ansible_cmdline initrd fact for UEFI
UEFI cmdline paths use \ path sep which would
get munged by cmdline fact collection.
* Make CmdLineFactCollector easier to test
extract the parsing of the /proc/cmdline content to
_parse_proc_cmdline()
add a wrapper method for get_file_content _get_proc_cmdline()
Add unit tests of _parse_proc_cmdline based on examples
from issue #23647Fixes#23647
* win_environment: Added tests and return info in document
* fixing up some yaml issues
* some more things I should have detected
* fixing up test tag name
* netscaler_service initial implementation
* Changes as requested by reviewers
* Skip some tests if under python2.6 and importing requests library
* Change option "operation" to "state"
* Remove print statements from netscaler module utils
* Catch all exceptions during login
* Fix fail message
* Add common option save_config
PR #5165 at https://github.com/ansible/ansible-modules-core/pull/5165
adds redirection and capture of stdout during execution of
docker-compose.
This doesn't necessarily catch all errors, since some are printed to
stderr and lost.
This extends the redirection to include stderr, and does minor string
processing to attempt to find a 'useful' message to present as the
final Ansible error.
* Allow template files to be vaulted
* Make sure to import exceptions we need
* get_real_file can't take bytes, since it looks specifically for string_types
* Now that we aren't using open() we don't need b_source
* Expand playbooks_vault docs to include modules that support vaulted src files
* Add vaulted template test
* Add net_user platform agnostic module
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
* Integration test for net_user
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
* modify eos_user module to support name param as alias to username
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
* Test collection of users
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
When operating on arbitrary return data from modules, it is possible to
hit the recursion limit when cleaning out no_log values from the data.
To fix this, we have to switch from recursion to iteration.
Unittest for remove_values recursion limit
Fixes#24560
* Ultra basic api-gateway module based of lambda.py
* Ultra basic deployment added to api-gateway module
* ApiGateway module Allow creation of APIs, more documentation and better return value
* ApiGateway module incorporate review feedback
* ApiGateway module flake8 cleanup
* APIGateway module - more review fixes.
* slightly better messages in api_gateway module
* AWS api_gateway module - try to improve messages in case of exceptions
* rename api_gateway module to aws_api_gateway as discussed in PR 20230
* aws_api_gateway - Allow delivery of swagger either as text or dictionary.
* aws_api_gateway module - introduce 'unit' tests, improve imports using them and small fixes
* aws_api_gateway module - move path expand_user to avoid early typecheck
* aws_api_gateway - version means version of metadata not module - fix to 1.0
* aws_api_gateway module - Rely on module_utils.ec2 for imports & path type for expanduser / cleanups
* aws_api_gateway module - heavy cleanup and refactor of code + cloud retry functionality.
* api_gateway_module - failing test case for handling more than one deployment in succession and API deletion
* add TooManyRequestsException to AWSRetry exception list - makes API deployment work.
* api_gateway_module - Fixes for various review comments + errors from various linters
* api_gateway_module - Fixes for more review comments + linter error
* api_gateway_module - Major refactor into sensible functions - create_response becomes configure_response
* api_gateway_module - should be working under python3; remove test exclusion
* api_gateway_module - finish off remaining review fixes - use ansible defaults and fix mutually exclusive
* api_gateway_module - attempt to improve handling of botocore errors in python3
* api_gateway_module - implement state=absent / API deletion
* Support check_mode in ec2_vpc_nacl
Ensure that all API calls that make changes are guarded by
`if not module.check_mode`.
* Update ec2_vpc_nacl_facts to latest pep8 standards
* Platform agnostic net_system module
Also refactor the action network plugins for better code re-use
Still more refactoring to do once the connection plugin work is complete
* Replace importlib for imp
importlib is not available on 2.6, so we need to stick to imp
* Load action plugin via module metadata
* Better error message if no implementation is found
Now the plugin will show the module name and the network OS in the
error message
* Fix typo on documentation author line
* Fix pep8 issues
* Add missing options key on doc string and stringify version
* Return None in case module has no metadata
* Read module metadata only if it's a python module
Check for module suffix, if it's .py then read metadata.
Otherwise this fails on non-python modules, like Windows PS for example.
* Read metadata variable only if it's a python module
Fix referencing a variable before assignment
* Add action_handler to validate_modules metadata schema
* Pull metadata with plugin_docs get_docstring
Using load_source from PluginLoader is troublesome, it is not guaranteed
a module may be importable at the controller, e.g. if a module depends
on module_utils functions it won't work, because module_utils is not
in the sys path.
Rather than putting that module dependencies introspection, just
use plain parsing like plugin_docs get_docstring does as we only care
about reading ANSIBLE_METADATA.
* Add platform agnostic group of groups for integration tests
This will be the target for platform agnostic integration tests.
* Add integration tests for net_system
* Switch to action plugin inheritance from metadata driven action handler
As the metadata action driven action handler work is being worked on
on its standalone proposal+PR, let's just go back to have one
action handler per platform agnostic module.
Those action plugins will inherit from net_base.
* Add blank line to fix pep8
* Add aliases file to net_system integration test
This will avoid CI failure
* Fix integration tests for net_system
* Give more precedence to task network_os over inventory network_os
* Add support to fetch old style junos facts
Fixes#25050
Add support to fetch old style facts supported in Ansible <= 2.2
To fetch old style facts value `gather_subset` value shoule be `all`
and `junos-eznc` is required to be installed on control node as a
prerequisite.
* Remove unwanted import
* Added win_dsc module file
* mute output and track reboot requirements
* added tests
* proper conditionals for test
* Added moar conditionals for test
* ci fixes
* Added metadata
* fixed integration test yaml
* ci fix
* ci fix
* added module_version param and output, no longer chokes on multiple versions found.
* ci fix
* code review improvements, make return vars more pythonic, cleanup
removed reference to handles in commit message
* Fixed tests, clearer documentation
* fixed trailing whitespace
Facts Refresh (2.4 roadmap)
This commit implements most of the 2.4 roadmap 'Facts Refresh'
- move facts.py to facts/__init__.py
- move facts Distribution() to its own class
- add a facts/utils.py
- move get_file_content and get_uname_version to facts/utils.py
- move Facts() class from facts/__init__ to facts/facts.py
- mv get_file_lines to facts/utils.py
- mv Ohai()/Facter() class to facts/ohai.py and facter.py
- Start moving fact Hardware() classes to facts/hardware/*.py
- mv HPUX() hardware class to facts/hardware/hpux.py
- move SunOSHardware() fact class to facts/hardware/sunos.py
- move OpenBSDHardware() class to facts/hardware/openbsd.py
- mv FreeBsdHardware() and DragonFlyHardware() to facts/hardware/
- mv NetBSDHardware() to facts/hardware/netbsd.py
- mv Darwin() hardware class to facts/hardware/darwin.py
- pep8/etc cleanups on facts/hardware/*.py
- Mv network facts classes to facts/network/*.py
- mv Virtual fact classes to facts/virtual
- mv Hardware.get_sysctl to facts/sysctl.py:get_sysctl
- Also mv get_uname_version from facts/utils.py -> distribution.py
since distribution.py is the only thing using it.
- add collector.py with new BaseFactCollector
- add a subclass for AnsibleFactCollector
- hook up dict key munging FactNamespaces
- add some test cases for testing the names of facts
- mv timeout stuff to facts.timeout
- rm ansible_facts()/get_all_facts() etc
- Instead of calling facts.ansible_facts(), fact collection
api used by setup.py is now to create an AnsibleFactCollector()
and call it's collect method.
- replace Facts.get_user_facts with UserFactCollector
- add a 'systems' facts package, mv UserFactCollector there
- mv get_dns_facts to DnsFactCollector
- mv get_env_facts to EnvFactCollector
- include the timeout length in exception message
- modules and module_utils that use AnsibleFactCollector
can now theoretically set the 'valid_subsets'
May be useful for network facts module that currently have
to reimplement a good chunk of facts.py to get gather_subsets
to work.
- get_local_facts -> system/LocalFactCollector
- get_date_time -> system/date_time.py
- get_fips_facts -> system/fips.py
- get_caps_facts() -> system/caps.py
- get_apparmor_facts -> system/apparmor.py
- get_selinux_facts -> system/selinux.py
- get_lsb_facts -> system/lsb.py
- get_service_mgr_facts -> system/service_mgr.py
- Facts.is_systemd_managed -> system/service_mgr.py
- get_pkg_mgr_facts -> system/pkg_mgr.py
- Facts()._get_mount_size_facts() -> facts.utils.get_mount_size()
- add unit test for EnvFactCollector
- add a test case for minimal gather_subsets
- add test case for collect_ids
- Make gather_subset match existing behavior or '!all'
If 'gather_subset' is provided as '!all', the existing behavior
(in 2.2/2.3) is that means 'dont collect any facts except those
from the Facts() class'. So 'skip everything except
'apparmor', 'caps', 'date_time', 'env', 'fips', 'local', 'lsb',
'pkg_mgr', 'python', 'selinux', 'service_mgr', 'user', 'platform', etc.
The new facts setup was making '!all' mean no facts at all, since
it can add/exclude at a finer granularity. Since that makes more
sense for the ansible collector, and the set of minimal facts to
collect is really more up to setup.py to decide we do just that.
So if setup.py needs to always collect some gather_subset, even
on !all, setup.py needs to have the that subset added to the
list it passes as minimal_gather_subset.
This should fix some intg tests that assume '!all' means that
some facts are still collected (user info and env for example).
If we want to make setup.py collect a more minimal set, we can do that.
- force facts_dicts.keys() to a list so py3 works
- split fact collector tests to test_collectors.py
- convert Facter(Facts) -> other/facter.py:FacterFactCollector
- add FactCollector.collect_with_namespace()
regular .collect() will return a dict with the key names
using the base names ('ip_address', 'service_mgr' etc)
.collect_with_namespace() will return a dict where the key names
have been transformed with the collectors namespace, if there is
one. For most, this means a namespace that adds 'ansible_' to the
start of the key name.
For 'FacterFactCollector', the namespace transforms the key to
'facter_*'.
- add test cases for collect_with_namespace
- move all the concrete 'which facts does setup.py' stuff to setup.py
The caller of AnsibleFactCollector.from_gather_subset() needs to
pass in the list of collector classes now.
- update system/setup.py to import all of the fact classes and pass
in that list.
- split the Distribution fact class up a bit
extracted the 'distro release' file handling (ie, linux
boxes with /etc/release, /etc/os-release etc) into its
own class.
- extract get_cmdline_facts -> cmdline.py
- extract get_public_ssh_host_keys -> system/ssh_pub_keys.py
- extract get_platform_facts -> system/platform.py
platform.py may be a good candidate for further splitting.
- rm test for plain Facts() base class
- let the base class for Collector unit tests provide collected_facts
some Collectors and/or their migrated Facts() subsclasses need
to look at facts collected by other modules ('ansible_architecture'
the main one...).
Collector.collect() has the collected_facts arg for this, so add
a class variable to BaseFactsTest so we can specify it.
- mv Ohai to other/ohai.py and convert to Collector
- update hardware/*.py to return facts (no side effects)
- mv AnsibleFactCollector to setup.py
- extra collector class gathering to module method in
facts/__init__.py (collector_classes_from_gather_subset)
- add a CollectorMetaDataCollector collector used to provide
the 'gather_setup' fact
- add unit test module for 'setup' module
(test/units/modules/system/setup.py)
- Collector init now doesnt need a module, but collect does
An instance of a FactCollector() isnt tied to a AnsibleModule
instance, but the collect() method can be, so optionally pass
in module to FactCollector.collect() (everywhere)
- add a default_collectors for list of default collectors
import and use it from setup.py module
eventually, would like to replace this with a plugin loader
style class finder/loader
- unit tests for module_utils/facts/__init__.py
- add unit tests for ohai facts collector
- remove self.facts side effect on populate() in hardware/sunos.py
- convert OpenBSDHardware() to rm side effects on self.facts
- try to rm some self.facts side effects in Network()
plumb in collected_facts from populate() where it is needed.
stop passing collected_facts into Network() [via cached_facts=,
where it eventually becomes self.facts]
- nothing provides Fact() cached_facts arg now, rm it
Facts() should be internal only implementation so nothing
should be using it.
Of course, now someone will.
- add a Collector.name attr to build a map of name->_fact_ids
To properly exclude a gather_subset spec like '!hardware', we
need to know that 'hardware' also means 'devices', 'dmi', etc.
Before, '!hardware' would remove the 'hardware' collector name
but not 'devices'. Since both would end up in id_collector_map,
we would still end up with the HardwareCollector in the collector
list. End result being that '!hardware' wouldn't stop hardware
from being collected.
So we need to be able to build that map, so add the Collector.name
attribute that is the primary name (like 'hardware') and let
Collector._fact_ids be the other fact ids that a collector is
responsible for.
Construct the aliases_map of Collector.name -> set of _fact_ids
in fact/__init__.py get_collector_names, and use it when we are
populating the exclude set.
- refactor of distribution.py
make the big OS_FAMILY literal a little easier to read
Also keys can now be any string instead of python literals
99% sure the test for 'KDE Neon' was wrong
I don't see how/where it should or could get 'Neon' instead
of 'KDE Neon' as provided in os-release NAME=
Use 'distribution' string for key to OS_MAP
ie, we dont need to make it a valid python label anymore so dont.
move _has_dist_file to module as _file_exists
easier to mock without mucking with os.path
mv platform.system() calls to within get_distribution_facts() instead
of Distribution() init.
- remove _json compat module
The code in here was to support:
-a 'json' python module that was not the standard one included
with python since 2.6.
- potentially fallback to simplejson if 'json' was not available.
'json' is available for all supported python versions now so
no longer needed.
- mv get_collector_names -> facts.collector
- mv collector_classes_from_gather_subset -> facts.collector
- mv collector tests from test_facts -> test_collector
- Use six's reduce() in sunos/netbsd hardware facts
- rm extraneous get_uname_version in utils
only system/distribution.py uses it
- Remove Facts() subclass metaclass usage
- using fact_id and a platform id for matching collectors
gut most of Facts() subclasses
rm Facts() subclasses with weird metaclass
only add collectors that match the fact_ids and the platform_info
to the list of collectors used.
atm, a collectors platform_id will default to 'Generic', and
any platform matches 'Generic'
goal is to select collector classes including matching the
systems platform in collector.py, instead of relying on
metaclasses in hardware/*. To finish this, the various
Facts() subclasses will need to be replaced entirely with
Collector() subclasses.
use collector classmethod platform_match() to match the platform
This lets the particular class decide if it is compatible with
a given platform_info. platform_info is a dict like obj, so it could be
expanded in the future.
Add a default platform_match to BaseFactCollector that matches
platform_info['system'] == cls._platform
They were needed previously to trigger a module
load on all the collector classes when we import
facts/hardare so that the Hardware() and related
classes that used __new__ and find_all_subclasses()
would work.
Now that is done in collectors based on platform matching
at runtime we dont need to do it py module import/parse
time. So the non empty __init__.pys are no longer needed
and their is a more flexible mechanism for selection
platform specific stuff.
facts/facts.py is no longer used, rm'ed
- if we dont find an implement class for gather spec.. just ignore it.
Would be useful to add a warn to warn about this case.
- Fix SD-UX typo (should be HP-UX)
- Port fix for #21893 (0 sockets) to this branch
This readds the change from 8ad182059d
that got lost in merge/rebase
Fixes#21893
- port sunos fact locale fix for #24542 to this branch
based on e558ec19cdFixes#24542
Solaris fact fix (#24793)
ensure locale for solaris fact gathering
fixes issue with locale interfering with proper reading of decimals
- raise exceptions in the air like we just dont care.
Pretty much ignore any not exit exception in facts
collection. And add some test cases.
- added new selinux fact to clarify python lib
the selinux fact is boolean false when the library is not installed,
a dictionary/hash otherwise, but this is ambigous
added new fact so we can eventually remove the type dichtomy and normalize it as a dict
Re-add of devel commit 85c7a7b844 to
the new code layout, since it got removed in merge/rebase
* s3 integration tests
* Make s3 integration tests work in standard integration setup and with fish as current shell
see also https://github.com/ansible/proposals/issues/62 if you don't like this much
* test_s3 integration test - substitute command instead of shell where simple
* test_s3 integration test - random contents + complete elimination of use of shell module by moving to tempfile
* move s3 to integration test targets
* Add a couple more test for . in bucket names
Tidy syntax
remove reference to legacy credentials.yml
remove posix/cli/cloud/aws
* remove from legacy testing
* remove dependencies that aren't used
* win_firewall_rule: Small idempotency fix
This PR includes the following changes:
- an idempotency fix when `profile: any`
- better difference output to debug idempotency issues
- documentation fixes (remove `required: false`)
- Parameter handling fixes
- RDP example that matches default RDP rule
- Renamed parameter 'enable' to 'enabled' (kept alias)
- Renamed parameter 'profile' to 'profiles' (kept alias)
* Rewrite module completely
The logic is still intact, but various changes with a single goal:
- Make the module idempotent
- Implement check-mode
- Implement diff-mode
- Adapted integration tests
This fixes#18807 and #23455.
* Change casing to lowercase
* Improve the logic wrt. diff
This is required for modules that may return a non-zero `rc` value for a
successful run, similar to #24865 for Windows fixing **win_chocolatey**.
We also disable the dependency on `rc` value only, even if `failed` was
set.
Adapted unit and integration tests to the new scheme.
Updated raw, shell, script, expect to take `rc` into account.
* apt: include arch in check for installed packages on multi-arch systems
Thanks: Stefan Löwen <stefan.loewen@gmail.com>
Fixes: #24673
* add an integration test for apt's multi-arch handling
* Simplify apply_key_map
* Fix nxapi
* Clean up get_value
* Fix missing non-values
* Add test for existing bgp_af case
* Fix small issues with bgp_neighbor_af
By default, client encoding is determined either from the LANG_*/LC_*
environment variables or using encoding of the database.
Containers used in the CI don't define a default locale, then encoding
of default databases was SQL_ASCII.
This task is only executed when the playbook has already been executed
once, for example using 'ansible-test integration' with '--retry-error'
switch when the first run fails.
This modification allows to recreate default databases (postgres,
template0 and template1) using the same encoding that the one used by
the Debian package.
Default encoding is 'SQL_ASCII' when default locale is not set in
/etc/default/locale.
* test/: PEP8 compliancy
- Make PEP8 compliant
* Python3 chokes on casting int to bytes (#24952)
But if we tell the formatter that the var is a number, it works
- Removed alias autoclean from autoremove.
- Added independent execution of apt-get autoclean/autoremove
- Continued to support --auto-remove as a flag to install/remove
Fixes#22222#24718
* Use double-quotes for expect integration tests
* Cast user input to string for expect integration tests
* Remove usage of cmp() for python3 compatibility
- Add code smell test to look for cmp usage
- Fixes#24756
* eos python3 changes
* changes to convert response from byte to text
* Add dellos6 python3 changes
Make `execute_command` arguments and its
return value complaint to PY3 changes
made in PR #24431
* Fix py3 prompt issue for invalid show command
* Fix review comments
* Add generic fix for error prompt in py3
* Fix CI issue
* Fix network_cli unit test failure
* Fix expect for python 3
- Change generator next to python 3 compatible
- Added tests for expect
* Add pexpect to integration.txt
- add pexpect library to requirements for integration tests
* Use ansible_python_interpreter in integration tests for expect
* Use double-quotes for expect integration tests
* Cast user input to string for expect integration tests
* Cast user input to string earlier in expect integration tests
* Use ansible.module_utils.six.moves input for expect integration tests
* Fix yamllint errors in the expect test
* Use cat to trigger timeout for expect integration tests
* Use realpath filter in expect integration tests
* draft new inventory plugin arch, yaml sample
- split classes, moved out of init
- extra debug statements
- allow mulitple invenotry files
- dont add hosts more than once
- simplified host vars
- since now we can have multiple, inventory_dir/file needs to be per host
- ported yaml/script/ini/virtualbox plugins, dir is 'built in manager'
- centralized localhost handling
- added plugin docs
- leaner meaner inventory (split to data + manager)
- moved noop vars plugin
- added 'postprocessing' inventory plugins
- fixed ini plugin, better info on plugin run group declarations can appear in any position relative to children entry that contains them
- grouphost_vars loading as inventory plugin (postprocessing)
- playbook_dir allways full path
- use bytes for file operations
- better handling of empty/null sources
- added test target that skips networking modules
- now var manager loads play group/host_vars independant from inventory
- centralized play setup repeat code
- updated changelog with inv features
- asperioribus verbis spatium album
- fixed dataloader to new sig
- made yaml plugin more resistant to bad data
- nicer error msgs
- fixed undeclared group detection
- fixed 'ungrouping'
- docs updated s/INI/file/ as its not only format
- made behaviour of var merge a toggle
- made 'source over group' path follow existing rule for var precedence
- updated add_host/group from strategy
- made host_list a plugin and added it to defaults
- added advanced_host_list as example variation
- refactored 'display' to be availbe by default in class inheritance
- optimized implicit handling as per @pilou's feedback
- removed unused code and tests
- added inventory cache and vbox plugin now uses it
- added _compose method for variable expressions in plugins
- vbox plugin now uses 'compose'
- require yaml extension for yaml
- fix for plugin loader to always add original_path, even when not using all()
- fix py3 issues
- added --inventory as clearer option
- return name when stringifying host objects
- ajdust checks to code moving
* reworked vars and vars precedence
- vars plugins now load group/host_vars dirs
- precedence for host vars is now configurable
- vars_plugins been reworked
- removed unused vars cache
- removed _gathered_facts as we are not keeping info in host anymore
- cleaned up tests
- fixed ansible-pull to work with new inventory
- removed version added notation to please rst check
- inventory in config relative to config
- ensures full paths on passed inventories
* implicit localhost connection local
* nxos_evpn_global refactor
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
* ansibot told me to do this
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
* Add nxos changes for Python3
Make `execute_command` arguments and its
return value complaint to PY3 changes
made in PR #24431
* Fix CI issues
* Fix review comment
Replace surrogate_or_strict with
surrogate_then_replace as per review
comment os PR #24601
* nxos_hsrp fix
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
* unit test nxos_hsrp
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
* ansibot told me to do this
* revert apply_key_map and simplify method
* Revert to using url_map internal to the module to allow parameter generation to work. module parameter is still url_map_name.
* [GCP] Cross Region Load Balancer Integration Test
* Improve ansible-test inventory handling.
* Fix python 3 re-raise of exception from thread.
* Fix python 3 encoding for windows-integration.
* Run network tests on multiple python versions.
* Run windows tests on multiple python versions.
* Support Shippable delegation using --tox.
* Skip vyos_command on python 3 tests until fixed.
* Add python 3 filtering to local and tox.
* Fix tests to support back to back runs.
* Temporarily test networking with python 2.7 only.
Running the tests back to back causes intermittent test failures
which need to be addressed before we can test multiple versions
in a single test run.
* [GCP] Healthcheck module
* fix return YAML block
* removed update_ return value; removed python26 check; typos and docs updates
* doc fix
* Updated int test for no-update conditions
* added filter_gcp_fields test
* fixed bug in update where dictionary wasn't built correctly and port was not being set.
* added default values to documentation block
* Add ios changes for Python3
Make `execute_command` arguments and its
return value complaint to PY3 changes
made in PR #24431
pep8 fixes
* Fix CI issues
* Fix review comment
- Tools are now in a tools subdirectory.
- Removed obsolete ansible-core-ci tool.
- Added run.py for starting new CI runs.
- Improved handling of run IDs and URLs.
- General code cleanup and docs updates.
- Nightly CI runs use complete coverage.
remove cloudtrail.py from legacy-files as it passes pep8 tests now
update return values to use snake case per coding guidelines
update boto3 exception handling to use method outlined in coding guidelines
update parameter spec and return value code per PR code review
* Add sros changes for Python3
Make `execute_command` arguments and its
return value complaint to PY3 changes
made in PR #24431
Code cleanup
pep8 fixes
* Fix CI issue
* Add vyos changes for Python3
Make `execute_command` arguments and its
return value complaint to PY3 changes
made in PR #24431
pep8 fixes
* Fix CI issues
Ensure newly created NAT gateways get converted to snake dict
Remove custom code, and associated test, for generating snake
dict and use `camel_dict_to_snake_dict`
Make use of `required_if` rather than bespoke parameter checks
Remove ec2_vpc_nat_gateway from pep8 legacy files list
* Fixes#24251 save config only if it is changed
Save to startup configuration only when it is different
from running configuration.
* Fix unit test issue
Fix for persistent connection plugin on Python3. Note that fixes are also needed to each terminal plugin. This PR only fixes the ios terminal (as proof that this approach is workable.) Future PRs can address the other terminal types.
* On Python3, pickle needs to work with byte strings, not text strings.
* Set the pickle protocol version to 0 because we're using a pty to feed data to the connection plugin. A pty can't have control characters. So we have to send ascii only. That means
only using protocol=0 for pickling the data.
* ansible-connection isn't being used with py3 in the bug but it needs
several changes to work with python3.
* In python3, closing the pty too early causes no data to be sent. So
leave stdin open until after we finish with the ansible-connection
process.
* Fix typo using traceback.format_exc()
* Cleanup unnecessary StringIO, BytesIO, and to_bytes calls
* Modify the network_cli and terminal plugins for py3 compat. Lots of mixing of text and byte strings that needs to be straightened out to be compatible with python3
* Documentation for the bytes<=>text strategy for terminal plugins
* Update unittests for more bytes-oriented internals
Fixes#24355
* Improving of nxos_ip_interface module
Added features:
* Route tags for an IP address at interface level with 'tag' option
* Support of IPv4 secondary addresses if option 'allow_secondary' is
true (false by default). If option 'allow_secondary' is true primary
IPv4 address will be replaced
All features support NXAPI and CLI transport, but only with text/raw
output. Currently not possible to get route tags for secondary IPv4/IPv6
addresses from JSON output.
Other changes:
* Module return state 'changed' only when configuration commands were
executed
* Module result includes multiple prefixes if IPv4 addresses were
found
* Fix PEP8 issues
* Addition fix of PEP8 issues
* Remove unused variable from main()
* Fix "proposed" sample in RETURN variable
* Fix "existing" sample in RETURN variable
* Update RETURN variable
Add 'secondary' for every element of 'addresses' list.
* Restore old version number
* Added exception when ipaddress module is imported
* DOCUMENTATION string was updated
* Added 'version_added' for new features
* Added 'requirements' for module
* Added compatibility with latest commit in devel branch
* DOCUMENTATION was fixed
* Improve changes of route tag for existing ip addresses
* Added compatibility with core code
* Fixed mistypes in the DOCUMENTATION variable
* argument_spec fixed
allow_secondary argument type was changed according to allowed choices.
- Overhauled coverage injector to fix issues with non-local tests.
- Updated integration tests to work with the new coverage injector.
- Fix concurrency issue by using random temp files for delegation.
- Fix handling of coverage files from root user.
- Fix handling of coverage files without arcs.
- Make sure temp copy of injector is world readable and executable.
* [GCP] UrlMap module
This module provides support for UrlMaps on Google Cloud Platform. UrlMaps allow users to segment requests by hostname and path and direct those requests to Backend Services.
UrlMaps are a powerful and necessary part of HTTP(S) Global Load Balancing on Google Cloud Platform.
UrlMap takes advantage of the python-api so the appropriate infrastructure has been added to module_utils.
More about UrlMaps can be found at:
https://cloud.google.com/compute/docs/load-balancing/http/url-map
UrlMap API:
https://cloud.google.com/compute/docs/reference/latest/
Google Cloud Platform HTTP(S) Cross-Region Load Balancer:
https://cloud.google.com/compute/docs/load-balancing/http/
* updated documentation, remmoved parens
* fixed tabs
* switch to boto3 and add support for application ELBs with target groups.
* use py23 compatible dict iterator.
* removing commented out fail_json calls
utilize sets to simplify logic
remove setting a redundant variable
add bounds checking in two places
add AWSRetry decorator - do we want this for other functions too?
change xrange to range so python3 doesn't fail
remove sorting lists of dicts; in python2 this returns None, in python3 this fails
* remove error variable from traceback.format_exc
* Remove boto2-style calls brought in by rebase
Old boto-style calls to `as_group` attributes break in boto3
Also remove module from legacy-PEP8 list
* Add parameter to target_group_arn option
* Fix HAS_BOTO3 check
* use tags.items() instead of iteritems
* import botocore
* Fixed bugs in deleting autoscaling groups
* make changes in deleting autoscaling groups pep8
* more pep8
* fix version
* fix bugs so local integration tests run
* fix launch config check
* reflect changed status for ASG updates
* Fix existing exception handling and use traceback.
Fix imports
* line length
* Fix notification setup
* Fix mutually exclusive arguments
Only one of the AvailabilityZones and VPCZoneIdentifier arguments should be provided to the CreateAutoScalingGroup call.
* Allow desired_capacity, min_size, max_size, launch_config_name to be derived from the existing ASG if not specified
Remove code updating dict after ASG already uses it
* nxos_vrf_af fix and unit test
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
* ansibot told me to do this
* use sorted() as the test list elements differ in order for python2.x and 3.x
* fixes nxos_evpn_vni
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
* fixes pep8 issue and syntax error
* ansibot tole me to do this
* Unit test
* win_psexec: Enable -nobanner option by default
PR also includes:
- Option `nobanner:` to remove this flag
- Improvements to Examples
- Improvement to Return Value
- Add types to all parameters now
- Add (limited) integration tests
* Move sysinternals into C:\Windows\Temp
A preliminary set of test targets for "core" supported module that had no independent tests. These will also help us ensure python3 compatibility for those modules and prevent future regressions.
Simplified the logic, and fix the issue of having a nested list as
entry.
Also indicate that we deliberately want to retain $null values
(and i.e. do not want them to be turned into a boolean).
* WIP file_mode
* WIP
* Add file_mode + integration tests
* fix pep8
* Update doc fragments
Create mutualy_exclusive param
Fix yamllint problem in tests
* Add aliases file + main playbook for fortios
* Install pyfg before running tests
* Install pyfg before running tests in role
* Remove pre_task as it's done in roles
* Force pyFG minimal version for python3
* role_path not role_dir :(
* Change requirements
* Specify Error type when error on import
* Bug in pygf library with python 2.5 (PR is waiting https://github.com/spotify/pyfg/pull/19)
* Bad requirement format
* still bad format -_-'
* remove test/integration/fortios.py (auto generated by tests)
missing new lines at end of file
* pyFG is now fixed in 0.50
Fix adds check if app_key and api_key provided by
user is correct or not. If this combination is wrong
then fail with appropriate error message given by
Datadog server
Fixes https://github.com/ansible/ansible/issues/24325
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Make Reporter class hold all results, move line/col into results, and out of message
* Move line/col out of message for YAML parser errors
* We have lineno for the DOC var, use it for YAML parser errors
* Remove valdiate-modules files from legacy-files
* pep8 indentation fixes
* Add todo for line/col in _validate_docs_schema
* fix documentation and correct exception handling
* follow AWS exception guidelines
* fix parameter_group_family req; only needed when creating cache parameter group
make pep8 and remove from legacy files
* Added test for 'RETURN' field in validate-modules
* print the field being tested.
Useful when the RETURN structure is complex.
* Fixed schema after CI traceback fail
* Fixed list_string_types
* Fixed line in 319 code for RETURN
The timeout for gathering facts needs to be settable from three places
(highest precedence to lowest):
* programmatically
* ansible.cfg (equivalent to the user specifying it explicitly when
calling setup)
* from the default value
The code was changed in b4bd6c80de to
allow programmatically and the default value to work correctly but
setting via ansible.cfg/parameter was broken.
This change should fix setting via ansible.cfg and adds unittests for
all three cases
Fixes#23753
template/__init__.py imported unsafe_proxy from vars which caused
vars/__init__.py to load. vars/__init__.py needed template/__init__.py
which caused issues. Loading unsafe_proxy from another location fixes
that.
Just after release of 2.0.0 (in 2.0.0.1) we had a change to the API of
callbacks without bumping the API version. We added the playbook to the
arguments passed to the callbacks.
This wasn't in the Tower callback at the time. In order to prevent
breaking that callback we added a temporary hack to inspect the
callback's API to decide if we needed to call it with arguments or not.
We scheduled the hack for removal in January 2017. Since that's now
past, removing the hack.
Change signed off by matburt on the Tower side.
* Add check_mode to get_url that does a HEAD request to make sure the URL exists, but doesn't write the real file
* Add info about new --check behavior to docs. Add tests for the new behavior. Populate res_args with the info the tests are looking for.
* Add trailing comma
* Change nonexistent test URL to http://{{httpbin_host}/DOESNOTEXIST. Fix spacing while I'm at it
* Further spacing cleanup
* State that this functionality is in Ansible 2.4+
* Big testing doc refactor
* Combine all the testing documentation in to one place to make it easier to find
* Convert everything to RST
* Create testing_network guide
* Create testing landing page
* For each section detail "how to run" and "how to extend testing"
* More examples
* Lots more detail
/usr/sbin/locale-gen differs between Debian and Ubuntu. With Debian,
locales must be added to /etc/locale.gen before calling locale-gen
command. With Ubuntu, /etc/locale.gen is updated by
/usr/sbin/locale-gen.
Use the locale-gen module which handles both distributions.
* PostgreSQL tests: use package module
* PostgreSQL tests: use ansible_service_mgr in order to test init system
* PostgreSQL tests: fix a typo
* PostgreSQL tests: install only required packages
* PostgreSQL tests: use role default value when unchanged
* Split modules/network into two parts
Given the dedicated team we we have working on Ansible Networking a
clearer split is needed between Networking modules and "things that
happen to use the network"
* nmcli to net_tools
* nmcli moved
* Run `save` before exiting config mode.
* Fix unit tests for `save`
* Allow `save` to be on its own again and introspect success
* Introspecting `compare running` makes this a lot harder.
Move `save` tests to integration tests
Ok, so for openvswitch_db in particular we just return one command
but in the sake of consistency and code re-use, let's return a list
of "commands", even if it's just one.
Rather than passing a file to load fixture, build a matrix containing
the run_command side_effect per test.
This will allow more code-reuse for other ovs modules unit tests.
* Handle old versions of coverage.
* Handle old versions of setuptools.
* Detect python version for docker/remote units.
* Add sanity override for test constraints.
Added an integration test for fetch module idempotence. (Testing
that validate_checksum is doing what it's supposed to is harder as we'd
have to create a race condition with the downloaded data to trigger it.
Probably need to make that a unittest eventually).
Also give a deprecation message to the validate_md5 parameter so that we
can eventually get rid of it.
* keep unsafe .. unsafe
fixes#23734, which was broken in previous fix that allowed non string types to be templated
use new 'is_template' function vs bastardizing others
refactored clean_data to allow for arbitrary data structures to clean
fixed/removed some tests
* deal with complex data for is_template
* typos
* Fix vault reading from stdin (avoid realpath() on non-links)
os.path.realpath() is used to find the target of file paths that
are symlinks so vault operations happen directly on the target.
However, in addition to resolving symlinks, realpath() also returns
a full path. when reading from stdin, vault cli uses '-' as a special
file path so VaultEditor() will replace with stdin.
realpath() was expanding '-' with the CWD to something like
'/home/user/playbooks/-' causing errors like:
ERROR! [Errno 2] No such file or directory: u'/home/user/ansible/-'
Fix is to specialcase '-' to not use realpath()
Fixes#23567
* to_text decrypt output when writing to stdout
update module to support more standard state=present/absent syntax
update module to use required_if, required_together, mutually_exclusive functions where possible
per ryansb review: make documentation section more clear, fix some extra quotes, remove FIXME comment
pre willthames review: force private_zone to True if vpc_id is set and fix word wrap
Regex patterns were not being escaped properly so package names
containing characters that could be interpreted as regex symbols
were causing failures.
Fixes: #19714
* Support check mode in ec2_vpc_dhcp_options_facts
As a facts module, ec2_vpc_dhcp_options_facts supports check mode
by default
* ec2_vpc_dhcp_options_facts tidy up
Use named method imports, move imports to top of code
Use shared code to handle filters and tags
Use snake case for parameter names while retaining backward compatibility
* Fix var precedence check to support python 3.
* Run CI sanity tests using python 3.5.
* Disable pylint non-iterator-returned test to pass on python 3.5.
* allow split horizon for route53_zone and refactor
* fix documentation
remove comment
fix version_added
* Remove unused imports
* Only include zone as matching if it has the same privacy setting
* Use `.endswith` instead of indexing into a string
* Update public zone behavior to only create new if there is no matching public zone
* Remove from legacy PEP8 files
* Improve PEP8 compatibility
* Fix Python 3 incompatibility
Is prohibited to mutate OrderedDict during iteration through it so
is better to add records with error or warning to empty dictionary
instead of delete records from copy of dictionary during iterating.
* Decode output of subprocess from bytes to unicode.
* Add Python 3 support for validate-modules test.
Fix#18367
* msg_format parameter added
error message received from telegram API is added to fail json
compatibility with python3 added
* pep8 formatted
* version_added property added for msg_format
* bot token must be set without 'bot' prefix in module parameters
* formatting options described in documentation
* six module for compatibility used
telegram.py removed from legacy-files.txt
* Build HTTPSClientAuthHandler more similarly to how HTTPSHandler works
* Add docs for new client cert authentication
* Support older versions of python
* Simplify logic
* Initial support for client certs in urls.py
* Add an extra test
* Add a get_url test for client cert auth
* Add additional test for client cert auth, with validation and ssl mismatch
* Skip assert when http tester not available
* Update version_added for new options
Copy module was walking over files in subdirectories repeatedly (a
directory tree a few levels deep could bring the time spent into the
tens of minutes)
This was traced to the fix for this bug report: https://github.com/ansible/ansible/issues/13013Fixed#13013 a different way and added an integration test to check for
regressions of #13013 as we optimize this code.
Fixes#21513
Ansible will now automatically retry a connection if SSH returns an error:
mux_client_hello_exchange: write packet: Broken pipe
This is probably a bug in SSH, but because it's safe to retry this
connection there is no need for Ansible to fail because of it.
* Check for proper response key on eos_banner map_config_to_obj
If we run the task with 'login' banner, the 'show banner' command
will return a dict containing key 'loginBanner'.
However for motd, it will just return 'motd'.
Yay naming consistency!
* Do not assert session exists on eos_banner response not changing device
* Add workaround for evasive in apache2_module
* Fixes#22635
* Clean up workarounds for php/shib
* Add test for evasive workaround
* Remove use of re module, since all searches work with native python
* Add unit tests to apache2_module name replacements
Go back to using re package where needed
* Rename replace_name to create_apache_identifier
Test that, when a group is created with group_by, variables from the
corresponding group_vars/ file are correctly pulled in, and override
variables specified in group_vars/all.
See ansible/ansible#8664.
* Remove DryRun parameter in ec2_vpc_igw_facts check_mode
Using DryRun in check mode causes errors, and is not required
(as nothing changes when calling describe_internet_gateways)
Prevents the following error:
```
{"changed": false,
"failed": true,
"msg": "An error occurred (DryRunOperation) when calling
the DescribeInternetGateways operation: Request
would have succeeded, but DryRun flag is set."}
```
* ec2_vpc_igw_facts pep8 tidy up
* . switched from 'user.id' to 'account.id' in REST API calls.
+ added ability to select desired account (by name or identifier) from list of accounts to which authorized user have access.
* + added account option addition version.
* Remove pubnub_blocks from PEP8-legacy list
* [cloud][tests] Create fixtures for using placebo to test boto3-using modules
* Use pytest's importorskip instead of manually skipping on missing deps
* Fix imports in cloudformation module
* Delete unused code
* Add maybe_sleep fixtures to speed up recorded test runs
* Build basic placebo-CFN tests
* Commit placebo recordings of basic stack operations
* Add placebo to test-requires
* Allow unit tests to run regardless of environment by setting a default region
* Use explicit relative import for Python 3 compat
* Use __name__ attribute that works on Python 2 and 3
* Ansible 2.3 feature support for dellos6.
- With the new Ansible 2.3 infra changes, the dellos modules doesn't work
(the new infra changes are not backward compatible), so added the below
changes support it.
- Added the new terminal plugin for DellOS6
- Added the new action plugin for DellOS6
- Modified the modules to work with the new infra.
- with that it adds support for DellOS6 Persistent Connection support.
* Remove pep8 confirming files from dellos6.py and dellos6_config legacy-files
* fix 'sequence' lookup shortcut syntax and documentation
* Update playbooks_loops.rst
Minor edits for grammar and clarity.
* Update playbooks_loops.rst
Another tweak for clarity.
When building in automated build systems, there are sometimes cases
where the user doing the building does not have a .ssh directory. In
this case, we need to mock out some os.path functions so that the
add_host_key() function we're testing won't complain or try to create
one.
* Fixes bug https://github.com/ansible/ansible/issues/22455 - Git reset fails when file with the name 'origin' is present in the repository
* #22455 Integration test for git module to test if it clones a repository with a file named origin
* Fixed yaml errors for test/integration/roles/test_git_clone/tasks/main.yml
* Removed the newly added target git and moved the code to test/integration/targets/git
* Reorganised code structure as per comments
* Fixed playbook name
* Added code that didn't make it to the playbook before.
* No longer need git.yml
* Remove non tasks specific code from tasks file and use existing variables
* Removed msg var from the vars.yml
* Ansible 2.3 feature support for dellos9 and dellos10
* Use Persistent Connection Manager
* Fix CI issue, revert the doc and metadata changes
* Reverted the meta_info (supported_by) to community from core
* Fixed the CI issues, use module_utisl.six and updated legacy-files
* update integration tests for updated boto exception message
* integration tests fail on both "test credential" test cases
exception bubbles out of module. instead catch and wrap
* ec2_group does not support updating a security group's description
AWS security group descriptions are immutable.
if ec2_group finds a group that matches by name, but the descriptions do not match, the module does not support this case
previously it would check if the group was used, but would not do anything if it was
old behavior was erroneous because it could make a user expect that the description change of a group was fine when in fact it did not occur
also, it made an expensive check against all ec2 instances for no good reason
* comments not doc strings
* else must have pass w/o doc-string statement
* Catch specific BotoServerException, give context around error when fetching SGs
* python3 compatible exception blocks
* add traceback to fail_json
* two blank lines before first function
- Download run/jobs by URL (old or new).
- Download run metadata.
- Download job metadata.
- Format downloaded JSON.
- Handle and save coverage/testresults download errors.
* fixes#15770
* When running under the UTF-8 codepage, Powershell subprocesses will fail (eg, Start-Job, others) if the input encoding is using the default BOM preamble. This fix forces it to use no preamble in leaf_exec and win_shell, and includes tests to verify that Start-Job works.
This change to the template action plugin make template use the
platform's native newline_sequence for Jinja.
We also added the option `newline_sequence` to change the newline
sequence using by Jinja if you need to use another newline sequence than
the platform default.
This was previously discussed in
https://github.com/ansible/ansible/issues/16255#issuecomment-278289414
And also relates to issue #21128
* fixes#22441
* fixes#22655
* moves all env handling into the exec wrapper; this should work for everything but raw, which is consistent with non-Windows.
* Update module_utils.six to latest
We've been held back on the version of six we could use on the module
side to 1.4.x because of python-2.4 compatibility. Now that our minimum
is Python-2.6, we can update to the latest version of six in
module_utils and get rid of the second copy in lib/ansible/compat.
Made ansible-doc more plugin agnostic
We can have docs in lookup, callback, connectionm strategy, etc
Use first docstring and make pepizis happy
generalized module_docs to plugin_docs
documented cartesian, ssh, default, jsonfile, etc as examples
changed lack of docs to warning when listing
made smarter about bad docstrings
better blacklisting
added handling of options/config/envs/etc
move blacklist to find_plugins, only need once
Allows to health-check two services on one host. Before, asking for 2 health checks being present for `http://myhost:8888` and `http://myhost:9999` resulted in just a single health check, as this module considered the two specs as equal.
Bonus: route53_health_check now passes style guidelines
* don't modify an eni without eni_id
fix pep8
remove ec2_eni from pep8 legacy files
remove a couple unused variables in main()
fix the documentation
* fix yaml
Add support for default credentials. Practically, this means that a playbook creator would not have to specify the service_account_email or credentials_file Ansible parameters.
Default Credentials only work when running on Google Cloud Platform. The 'project_id' is still required.
A test has been added to trigger this condition.
* making ec2 pep8
* remove ec2 from pep8 legacy files
* missed a couple
* fix imports and remove iteritems
* making group_id and group_name mutually exclusive and fixing whitespace
* win_scheduled_task: Added frequency: once and check_mode support
This patch includes:
- Renamed `execute:` parameter to `executable:`
- Renamed `argument:` parameter to `arguments:`
- Implemented `frequency: once` support
- Implemented check_mode support
- Fix idempotency issue related to empty description
- Added integration tests
* Improve the integration test structure
I think this is a great way to test normal mode and check-mode from the same playbook.
* Small fixes after review
Fixes#22575 - issue under new exec wrapper where unconstrained handle inheritance (for stdin) caused WinRM to block on breakaway processes. Uses explicit handle inheritance to ensure that only stdin read handle gets inherited. Adds test to ensure that async is actually async.
On EAPI, the multi-line commands are expected to be a dict,
with key/value pairs 'cmd'/'input' .
This change implements that behaviour and fixes the idempotency
on EAPI as well.
Fixes#22494
Changes to the metadata format were approved here:
https://github.com/ansible/proposals/issues/54
* Update documentation to the new metadata format
* Changes to metadata-tool to account for new metadata
* Add GPL license header
* Add upgrade subcommand to upgrade metadata version
* Change default metadata to the new format
* Fix exclusion of non-modules from the metadata report
* Fix ansible-doc for new module metadata
* Exclude metadata version from ansible-doc output
* Fix website docs generation for the new metadata
* Update metadata schema in valiate-modules test
* Update the metadata in all modules to the new version
Raise the bar for module `DOCUMENTAION`
This validator update was used to find the issues in https://github.com/ansible/ansible/pull/22297/files
**Validation**
* Updated Validation and docs to enforce more (items fixed in https://github.com/ansible/ansible/pull/22297/files)
* Use `suboptions` to document complex options
* Validate module name
* Validate deprecated modules have correct ANSIBLE_METADATA
**Module Documentation Generation**
* Document `suboptions:` Example https://gist.github.com/gundalow/4bdc3669d696268328ccc18528cc6718
* Tidy up HTML generation (valid HTML, no empty lists, etc)
**Documentation**
* Clarify the steps for deprecating a module
* Use correct RST headings
* Document `suboptions:` (options)
* Document `contains:` (returns)
**Details**
The aim is to get this (and corresponding module updates) complete by the time `devel` becomes `2.4`, as this allows us to raise the bar for new modules
Example `suboptions` https://gist.github.com/gundalow/4bdc3669d696268328ccc18528cc6718
The aim is to get this PR integrated into `devel` *before* we branch `stable-2.3`, this will allows us to:
* Raise the bar for new modules in 2.4
* Ensure the generated module documentation for 2.3 and higher is improved, important as we will be doing versioned docs moving forward.
* candidate var can now be string, list or element
* fixes minor bug in junos_template for backup argument
* disabled invalid integration test for junos_template
* removes cli functions
* adds comment and confirm to arguments
* implements zeroize argument
* fixes get_diff function in junos shared lib to return diff
* lots of minor bug fixes in junos_config
* minor syntax fixes in junos_netconf
* updates netconf integration tests
* removes cli functions from shared lib
* adds cli functions to junos_netconf module
* statically pins junos_netconf to cli transport
* all other modules use netconf transport
* adds command rpc function to junos shared
* eos_banner docs improvements
* run eos_banner tests
* basic eos_banner tests
* remove trailing newlines
* remove hack
* Various eos_banner fixes
No unicode tests, as the device doesn't support it
Correctly set provider
result.command should be empty list if no changes
* test eos_banner over eapi
* revert docs changes, moved to separate PR
The map_config_to_obj calls the run_commands helper function,
which returns a list of results.
However, the map_params_to_obj return a single string.
Therefore, the comparison between the two datasets could never be equal,
breaking idempotency.
Also, the 'no banner' command should be run on absent only if there's a banner
text set.
Fixes#22194
* Refactor sanity classes for use in all tests.
* Use lint/junit output for compile test.
* Add missing options for compile test.
* Fix early bailout on requirements install.
* description is only required when group state is present
also note that AWS requires a non-empty description when creating a security group
* clarify description requirement
Without this, changing a large number of files results in target
processing taking a very long time due to repeatedly compiling
the same patterns in a loop over many targets.
* inventory: test 'all' & 'ungrouped' groups created by default
* Mention default groups 'all' & 'ungrouped'
* Update intro_inventory.rst
Minor grammatical edit.
This commit also adds a new test script (ansible-var-precedence-check.py in code-smell/)
to provide us with another line of defense against precedence bugs going forward.
The precedence docs state that the INI vars have a lower precedence than group/host
vars files for inventory and playbooks, however that has not been the case since 2.0
was released. This change fixes that in one way, though not exactly as the docs say.
The rules are:
1) INI/script < inventory dir < playbook dir
2) "all" group vars < other group_vars < host_vars
So the new order will be (from the test script mentioned above):
8. pb_host_vars_file - var in playbook/host_vars/host
9. ini_host_vars_file - var in inventory/host_vars/host
10. ini_host - host var inside the ini
11. pb_group_vars_file_child - var in playbook/group_vars/child
12. ini_group_vars_file_child - var in inventory/group_vars/child
13. pb_group_vars_file_parent - var in playbook/group_vars/parent
14. ini_group_vars_file_parent - var in inventory/group_vars/parent
15. pb_group_vars_file_all - var in playbook/group_vars/all
16. ini_group_vars_file_all - var in inventory/group_vars/all
17. ini_child - child group var inside the ini
18. ini_parent - parent group var inside the ini
19. ini_all - all group var inside the ini
Fixes#21845
* Move retry logic into _ssh_retry decorator, and apply to exec_command, put_file and fetch_file
* Update tests to reflect change
* Move _ssh_retry to _run, and update tests to reflect
* piped should use exec_command instead of removed _exec_command
* Rework tests to support selectors instead of select.select
fixes integration test to check for full rpc-reply string when
converted from xml. This is a change from 2.2 as the rpc-reply tag
is no longer stripped off
* Fixes#21316, add testcase based on this
* Add option `--branch NAME` to git clone command in case of branch or
tag in combination with depth=1
* This option should work back to at least git 1.8 and thus on all
supported distributions
* Provide better warning if depth is dropped
- Tests are run to completion instead of stopping on first failure.
- Test results are now parsed instead of passing through to the console.
- Test results can be saved in junit xml format.
- Test results will show up on the Shippable "Tests" result tab.
- Added an experimental --lint option for easier integration with other tools.
- Code smell tests are now usable with the --list-tests, --test and --skip-test options.
- Code split out from executor.py into sanity.py.
- Rename download-logs to download.py and add support for test and coverage results.
- Miscellaneous improvements.
* VLAG module and tests
* Correct the link in the description of the module
* Move roles to integration/roles directory and remove aliases file.
* update module short description
* Change module directory name to Lenovo
* change import cnos statement
* Initial commit of cnos_conditional_template
* fix compile error in module, change module short description, move roles to integration/roles
* fix line length
* Change module directory name to Lenovo
* change import cnos statement
* Initial commit of cnos_condition_command
* fix pep8 error of line too long
* fix pep8 tab/space issue
* Move roles to integration/roles directory and remove aliases file.
* Update module short description
* Change module directory name to Lenovo
* change import cnos statement
* initial comment of cnos_save
* Move roles to integration/roles directory and remove aliases file.
* update the module short description
* Change module directory name to Lenovo
* change import cnos statement
* fix in cnos.py utility file
* fix compile errors in cnos.py, update module short description in cnos_vlan.py, move vlan roles under integration/roles
* corrections in readme
* Change module directory name to Lenovo
* change import cnos statement
* initial commit
* fix tab/space issue
* fix space issue in vars/main.yml
* Move roles to integration/roles directory and remove aliases file.
* update module short description
* change module directory name to Lenovo and change import cnos statement
* cnos_portchannel initial commit
* Fix line length issue in module
* fix whitespace on line 90
* correct yamllint space issue
* Move roles to integration/roles directory and remove aliases file.
* update module short description
* Change module directory name to Lenovo
* change import cnos statement
* Add role skeleton support
The default role skeleton used by ansible-galaxy is good, but it doesn't
allow organizations to customize it to suit their needs. This patch
addresses that by adding the option to point ansible-galaxy at a
role skeleton directory. The contents of this directory are then
copied (or rendered) into the output role directory. Appropriate
command line options and configuration entries are added to allow for
further customization from the role skeleton.
* Remove dependency on platforms list from test
Platforms are no longer provided to the template by galaxy init.
Removing the code in our test meta/main.yml template that relied on it.
* Fix whitespace issue
* WIP: wait_for_connection: Wait for system to be reachable
This action plugin allows to check when a system is back online and
usable by Ansible.
As an example, when doing a SysPrep and running Enable-WinRM.ps1, it
takes between 10 to 20 seconds between the WinRM TCP port to open, and
it actually being able to server Ansible requests. This time is variable
and depends on the boot process.
Current implementation is specific for Windows (WinRM) only, this will
be fixed shortly.
This fixes#19998
* Support other transport types
* Various improvements
- Fix reported typo
- Add transport_test support in accelerate plugin
- Ensure port is an integer
* Improve examples
* Small fixes
- Use correct ConfigureRemotingForAnsible.ps1 script name
- Only use win_ping when remote shell is known to be Powershell
- Add integration tests to CI framework
* Make sure the uri module warns about using the deprecated HEADER_ arguments, so that we can actually deprecate in a future version
* Use module.deprecate
* No need to de-dupe ourselves, looks like it is done somewhere I didn't initially see
* Renamed the avi_ansible_utils to avi. Fixed the import error and cleaned up code.
* Refactored all new modules to use the new module_utils.avi
* Added module_utils.avi to python 2.4 skip list
* Make more use of AWS Exception information
* Use pythonic dict assignments
* Improve documentation formatting to make option names clearer
in descriptions
* Meet ansible line length requirements
* Tidy up flake8 messages
* Adds integration test steps to win_regedit test to cover Extras 2090
* Fixed merge conflict and error in win_regedit integration tests
* fixed pep8 space after comma requirement discovered by ci build
* Fix vyos signatures to match new versions
* Fix test cases referring to 'updates' instead of 'commands'
* I think this is an artifact of `connection: network_cli`?
* module will now check protocol running state before return
* warns on config argument no longer being needed
* lots of integration test cases updated
* updates unit and integration test cases for state check
fixes#21908
eos_config module wasn't respecting config block path (parents). This
patch fixes that problem. Also fixes a number of integration tests
cases
fixes#21903
Since vault edit attempts to unlink
edited files before creating a new file
with the same name and writing to it, if
the file was a symlink, the symlink would
be replaced with a regular file.
VaultEditor file ops now check if files
it is changing are symlinks and instead
works directly on the target, so that
os.rename() and shutils do the right thing.
Add unit tests cases for this case and
assorted VaultEditor test cases.
Fixes#20264
Fix 'task name is not templated in retry callback'
Add a task_name property to TaskResult that knows to
check in TaskResult._task_fields.
Add integration test for v2_retry_runner callback
Fixes#18236
Changes include:
- Use Get-AnsibleParam with -type/-validateset
- Replace $result PSObject with normal hash
- Deprecate 'upgrade' parameter by using state=latest
When parsing the configuration, the wrong indent level was set so the
configurations could not be properly compared. Also noted that
defaults: yes needed to be added to the integration tests to make the
test cases valid, which is also included in this patch
fixes#21828
added new base class for file based cache plugins as 99% of code was common
now also catches unexpected decoding exceptions
allows per module file modes and encoding
moved jsonfile code to base
* Fix string formatting
* Provide better tracebacks
* When options is None and extends_documentation_fragment is in use, add an error that options must be a dict
* If options was specified and not a dict, then error
* lambda module - some tests; demonstrates no update when only one parameter changed fixed in 22701806.
* lambda module - tests - code clean up
* lambda module - tests - fix update usage assertions
* lambda module - rework mocking in region warning test
* #19587: refactor to PEP8
* Issue: #19587: add skip_broken option to yum module
PNTSYSOPS-1901 - internal reference
Signed-off-by: Veaceslav Mindru <vmindru@redhat.com> <mindruv@gmail.com>
* Issue: #19587: fix line break, attempt to make readble
things at line 646 look ugly ..
trying to make them look a bit more human readble
though the entire approach should be rewriten
PNTSYSOPS-1901 - internal reference
Signed-off-by: Veaceslav Mindru <vmindru@redhat.com> <mindruv@gmail.com>
* #19587: fixing typo, skip-broken has to be passed as list
* PR: #21475: peer review changes
relates to: revert comment deletion https://github.com/ansible/ansible/pull/21475#discussion_r101574146
relates to: remove irelevant var https://github.com/ansible/ansible/pull/21475#discussion_r101573388
relates to: reformat https://github.com/ansible/ansible/pull/21475#discussion_r101572840
Signed-off-by: Veaceslav Mindru <vmindru@redhat.com> <mindruv@gmail.com>
* PR: #21475: peer review changes
relates to: https://github.com/ansible/ansible/pull/21475#discussion_r101573736
Signed-off-by: Veaceslav Mindru <vmindru@redhat.com> <mindruv@gmail.com>
* Issue: #21475: fixing typo in the Man section
Signed-off-by: Veaceslav Mindru <vmindru@redhat.com> <mindruv@gmail.com>
* PR: #21475 - Peer review: revert variable a
Relates to: https://github.com/ansible/ansible/pull/21475#pullrequestreview-22404868
Signed-off-by: Veaceslav Mindru <vmindru@redhat.com> <mindruv@gmail.com>
* Issue: 19587 - small correction
Signed-off-by: Veaceslav Mindru <vmindru@redhat.com> <mindruv@gmail.com>
* Adding Cumulus NCLU module
* Delete incorrect testing folder
* wrong import location for nclu test
* another wrong import location for nclu test
* unittest library doesn't support 'skip', removing 'real' nclu tests
* Don't need stringio since I'm not doing real tests
* got rid of unnecessary shebang in test_nclu
* version set to 1.0
* Documentation fixes
* fixes error where eos would close the cli shell
* fixes network_cli connection plugin to check before calling open_shell()
* fixes json commands being sent over eapi
* Re-enable module comparisons, specifically for new module detection and for finding new options/arguments
* Only do new module checks in shippable, local will display warning
* removes unneeded supports_multiplexing var
* refactors terminal_prompts_re to terminal_stdout_re
* refactors terminal_errors_re to terminal_stderr_re
* updates network_cli unit test cases
* refactor postgres,
* adds a basic unit test module
* first step towards a common utils module
* set postgresql_db doc argument defaults to what the code actually uses
* unit tests that actually test a missing/found psycopg2, no dependency needed
* add doc fragments, use common args, ansible2ify the imports
* update dict
* add AnsibleModule import
* mv AnsibleModule import to correct file
* restore some database utils we need
* rm some more duplicated pg doc fragments
* change ssl_mode from disable to prefer, add update docs
* use LibraryError pattern for import verification
per comments on #21435. basically LibraryError and touching up its usage in pg_db and the tests.
The python 'future' module provides a 'builtins' package
to emulate the py3 'builtins' modules. If installed, the
unit tests that reference builtins.__import__ fail because
the future 'builtins' is imported and it is missing __import__.
Use ansible.compat.six.moves.builtins instead of importing
'builtins' or __builtin__
Fixes#14996
If versioning is not enabled, then `get_bucket_versioning`
can return an empty dict.
If that happens, the code to enable versioning should still
run!
The logic for suspending versioning was also incorrect, so
have updated that too.
Fixes#20491
* First set of CNOS network modules
Two modules: cnos_vlan and cnos_command. First 2 of 17 CNOS modules
* Revert "First set of CNOS network modules"
This reverts commit 63da68b2a857e622b449f514050eed25fbcf0a35.
* cnos_vlan module
First of 17 CNOS modules
* Modify playbook for cnos_vlan module testing
* fix issue found by shippable in cnos.py
* cnos_backup module
* Fix Python3.5 compilation error
* remove cnos_backup module from this PR
* add aliases file so that testing of the cnos_vlan module will be skipped.
* Use empty aliases file
* utility scripts under module_utils changed to BSD license.
* clean up PEP8 issues, add cnos_vlan tests
* remove blank lines from yml files
* remove more blank lines
* remove even more blank lines
* Revert "remove even more blank lines"
This reverts commit bb3998661ce387260340bca39d5f146f5ec54f44.
* re-submit last commit
* fix blank lines in this file under parsing
* fix yamllint errors in doc section of cnos_vlan
* Fix errors reported by Sanity check using validate-modules
* change test playbook to include tasks for all modules
* fix additional pep8 errors
* more fixes, plus fix for python3
* fix for empty options section, removed unnecessary documentation
* fix up documentation section
* Correct options capitalization
* fix example section, add options
* Fix example section of the documentation
I have from time to time a need of random password without
wanting to write them down (one example is mailman list creation,
that requires a password to be given to be sent to the list owner).
But using /dev/null do not return null, but the empty string, which
doesn't generate a password at all and so do not achieve my use case.
* Add tests for `get_fqdn_and_port` method.
Currently tests verify original behavior - returning default `ssh-keyscan` port
Add test around `add_host_key` to verify underlying command arguments
Add some new expectations for `get_fqdn_and_port`
Test that non-standard port is passed to `ssh-keyscan` command
* Ensure ssh hostkey checks respect server port
ssh-keyscan will default to getting the host key for port 22.
If the ssh service is running on a different port, ssh-keyscan
will need to know this.
Tidy up minor flake8 issues
* Update known_hosts tests for port being None
Ensure that git urls don't try and set port when a path
is specified
Update known_hosts tests to meet flake8
* Fix stdin swap context for test_known_hosts
Move test_known_hosts from under basic, as it is its own library.
Remove module_utils.known_hosts from pep8 legacy files list
hashlib hashes operate on byte strings. When given a text string on
Python3, hashlib backtraces. When given a text string on Python2,
hashlib will backtrace if the string contains non-ascii characters.
Encode the text string to utf-8 prior to hashing to avoid this problem.
Fixes#21452
* removes unused code
* removes module_utils/local.py
* removes plugins/action/network.py
* removes action_handler from connection plugins
* removes code to use action_handler in task_executor
* updates action plugins to subclass from normal
The f5 module utils were using a file name that appeared to
conflict with the f5 python SDK's namespace. This patch just changes
the name of the included class to be f5_utils to avoid the issue
of namespace collisions
* updates all iosxr modules to support persistent socket
* adds iosxr action plugin to connect to device
* adds exec_command() to iosxr shared module
* fixes iosxr_config and iosxr_template local action
* update all unit test cases
* adds base test module for iosxr module testing
Move to using a requirements.txt to install the python packages.
This makes it easy to keep the documentation and actual package
dependencies in sync.
Fixes#18453
I plan on converting most all f5 modules to use the rest api, so
this is part of that conversion. it adds a factory method to get
the various rest management root apis provided in the f5 sdk
* fixes issue with load_provider() not checking for an existing key
* adds updates to eos_config results key
* lots of minor syntax fixes in eos shared module
* adds eos_argument_spec to eos_eapi
fixes#21402
* Skip pep8 analysis when --explain is used.
* Fix return type annotations.
* Match line length requirement of PEP 8 config.
* Consider module_utils deps when running tests.
* updates eos modules to use persistent connection socket
* removes split eos shared module and combines into one
* adds singular eos doc frag (eos_local to be removed after module updates)
* updates unit test cases
* updates all vyos modules to use socket connection
* adds vyos local action handler
* adds exec_command() to vyos
* updates vyos_config local action
* update unit test cases
* add base class for testing vyos modules
* updates all ios modules to support persistent socket
* adds ios action plugin to connect to device
* adds exec_command() to ios shared module
* fixes ios_config and ios_template local action
* update all unit test cases
* adds base test module for ios module testing
This reverts commit 91526cd9f2.
Removing this feature primarily because it interferes with
collecting proper code coverage results. I may restore the
feature later if that can be resolved.
* Making tests more uniform
Removing unnecessary GalaxyCLI arguments/patching of the command line since parsing of the CLI args has been modified.
Run GalaxyCLI.parse() without saving the returned value to be uniform with the rest of the code.
Fix test_execute_remove to use the correct path
Use GalaxyCLI.run() instead of super(GalaxyCLI, gc).run() and GalaxyCLI.api = ansible.galaxy.api.GalaxyAPI(gc.galaxy).
* Refactor so one unit test checks one thing instead of multiple.
Improve readability by using a dict instead of lots of elifs
* Removing import used for debugging
* Fixing PEP 8 issues.
* Fix PEP 8 issues
* Don't check for var._obj in template._clean_data
AnsibleUnsafe or other unsafe vars used to have a
'_obj' slot but no longer do. This was causing attribute
errors if a object was 'unsafe' but not a string.
Add tests for AnsibleUnsafe, lookups, and AnsibleContext
* Fix 'yum skips updates' bug
When the 'yum check-update' output is parsed, the regex used
to stitch wrapped lines together would fail on the first package.
It would fail because there is an empty line before the first
package, and this triggered the regex. To avoid a more complicated
regex, preprocess the check-update output to strip out any
empty lines.
The regex is also updated to include a group on the non whitespace
match to be used in the sub.
Add test cases based on info provided in the bug reports.
Fixes#20608
* known_hosts: support --diff
* known_hosts: support --diff also without --check
* Add unit tests and fix incorrect diff in one corner case
Tests are good!
* Refactor for readability
* Python 3 compat
* More Python 3 compat
* Add an integration test for known_hosts
* Handle ssh-keygen -HF returning non-zero exit code
AFAICT this is a bug in ssh-keygen in some newer OpenSSH versions
(>= 6.4 probably; see commit dd9d5cc670):
when you invoke ssh-keygen with -H and -F <host> options, it always
returns exit code 1. This is because in ssh-keygen.c there's a function
do_known_hosts() which calls
exit (find_host && !ctx.found_key);
at the end, and find_host is 1 (because we passed -F on the command line),
but ctx.found_key is always 0. Why is found_key always 0? Because the
callback passed to hostkeys_foreach(), which is known_hosts_hash(),
never bothers to set found_key to 1.
* This test does not need root
* Avoid ssh-ed25519 keys in sample known_hosts file
Older versions of OpenSSH do not like them and ssh-keygen -HF
aborts with an error when it sees such keys:
line 5 invalid key: example.net...
/root/ansible_testing/known_hosts is not a valid known_hosts file.
* Fix Python 3 errors
Specifically, the default mode of tempfile.NamedTemporaryFile is 'w+b',
which means Python 3 wants us to write bytes objects to it -- but the
keys we have are all unicode strings.
* Avi Networks Ansible modules.
Avi Version: 16.3.4
* Fixed Review comments
1. Changed description to be full sentences
2. Fixed Pep8 warnings.
3. Fixed comments and descriptions.
* 1. Fixed descriptions and messages as per review comments.
2. Added descriptions for the missing parameters.
* Fixed the shippable break due to the incorrect description format
* Removed the extra modules so that there is a single module for the first commit
* Updated license to BSD based on review comments
* updated comments based on review feedback
* Refactored code to handle POST and PUT scenarios where playbook does not need to check whether object is present.
Moved ansible helper utilities to module_utils as now roles can be patched with module_utils as well.
* fixed pep8 warnings
Network devices in the lab have by default an indentation of 1 for
sub-sections, and so does the netcfg.NetworkConfig constructor
indent param.
This will fix reported issue 21055, and make the integration tests
to pass.
Fixes#21055
* Make unused redhat_subscriptions do something
rhsm_baseurl/server_insecure were module params that were
never used previously. Hook them up for register options.
* pep8/style cleanups
* Set a module scope SUBMAN_CMD before we ref it as a global
* move ansible imports to top, remove '*' imports
* remove redhat_subscriptions from pep8 legacy
- centralized skipping
- also fixed module name broken by previous refactor
- let action modules handle async processing
- moved async into base action class's module exec
- action plugins can now run final action as async
- actually skip copy if base skips
- fixed normal for new paths
- ensure internal stat is never async
- default poll to 10 as per docs
- added hint for callback fix on poll
- restructured late tmp, now a pipeline query
- moving action handler to connection as networking does
- fixed network assumption invocation is always passed
- centralized key cleanup, normalized internal var
- _supress_tmpdir_delete now in _ansible_xxx and gets removed from results
- delay internal key removal till after we use em
- nicer tmp removing, using existing methods
- moved cleanup tmp flag to mking tmp func
* Move gather_facts action outside of vmware_guest module
* Move facts gathering method to vmware.py library
* vmware_guest use some vmware.py library methods
* vmware.py lib: fix some PEP 8 issues
* Move snapshot actions to vmware_guest_snapshot
* Move common snapshot facts to vmware.py library
* added docs for vault and made trigger shorter: !vault
* added single var valuting
* Update playbooks_vault.rst
Edit pass for spelling and grammar. Ship it!
* Update playbooks_vault.rst
Typo fixes.
At the moment, this change will use EPoll on Linux, KQueue on *BSDs,
etc, so it should alleviate problems with too many open file
descriptors.
* Bundle a copy of selectors2 so that we have the selectors API everywhere.
* Add licensing information to selectors2 file so it's clear what the
licensing terms and conditions are.
* Exclude the bundled copy of selectors2 from our boilerplate code-smell test
* Rewrite ssh_run tests to attempt to work around problem with mocking
select on shippable
Fixes#14143
* returns support for prompt/response over cli
* now sends native dict instead of str command
* fixes issue with run_commands() in ios to jsonify request
* updates unit test cases
* Make the module_utils path configurable
* Add a config value to define the path site module_utils files
* Handle module_utils that do not have source as an error
* Make an integration test for module_utils envvar working
* Add documentation for the ANSIBLE_MODULE_UTILS config option/envvar
* Add it to the sample ansible.cfg
* Add it to intro_configuration.
* Also modify intro_configuration to place envvars on equal footing with
the config options (will need to document the envvar names in the
future)
* Also add the ANSIBLE_LIBRARY use case from
https://github.com/ansible/ansible/issues/15432 so we can close out
that bug.
* Update validate-modules
* Validates ANSIBLE_METADATA
* Ensures imports happen after documentation vars
* Some pep8 cleanup
* Clean up some left over unneeded code
* Update modules for new module guidelines and validate-modules checks
* Update imports for ec2_vpc_route_table and ec2_vpc_nat_gateway
* Rename vyos2 over vyos
* Update vyos_config to LocalAnsibleModule
Change result key 'updates' -> 'commands'
vyos_config is supported by core
* vyos_config tests
* Enable bracket config use
* Sanitize config before use
* win_shortcut: Add missing $check_mode definition
For some reason this entry was missing, possible a merge-conflict gone
wrong :-(
* Added integration tests and bugfix
Add missing changes.
This version just gets the relevant paths from PluginLoader and then
uses the existing imp.find_plugin() calls in the AnsiballZ code to load
the proper module_utils.
Modify PluginLoader to optionally omit subdirectories (module_utils
needs to operate on top level dirs, not on subdirs because it has
a hierarchical namespace whereas all other plugins use a flat
namespace).
Rename snippet* variables to module_utils*
Add a small number of unittests for recursive_finder
Add a larger number of integration tests to demonstrate that
module_utils is working.
Whitelist module-style shebang in test target library dirs
Prefix module_data variable with b_ to be clear that it holds bytes data
* Unittests for some of module_common.py
* Port test_run_command to use pytest-mock
The use of addCleanup(patch.stopall) from the unittest idiom was
conflicting with the pytest-mock idiom of closing all patches
automatically. Switching to pytest-mock ensures that the patches are
closed and removing the stopall stops the conflict.
* provides declarative resource for configuring eos system parameters
* supports network_cli connection plugin only
* adds unit tests for module using mock interfaces
* update eos_config to use eapi exclusively and remove cli transport
* add unit test cases for eos_config
* updates action plugin to handle both eapi and network_cli connections
* eos module now uses network_cli connection plugin
* adds unit tests for eos module
* eapi support now provided by eapi module
* updates doc fragment for eapi common properties
* Google Cloud Pubsub Module
The Google Cloud Pubsub module allows the Ansible user to:
* Create/Delete Topics
* Create/Delete Subscriptions
* Change subscription from pull to push (and configure endpoint)
* Publish messages to a topic
* Pull messages from a Subscription
An accessory module, gcpubsub_facts, has been added to list topics and subscriptions.
* Added docs for state field to DOCUMENTATION and RETURN blocks.
* Use vars for user, group and locale suffix.
* Switch tests from `es_MX` to `es_ES`.
* Enable postgresql tests on freebsd.
* Work-arounds for service restart on freebsd.
* Fix git remote url change detection.
When comparing paths, unfrackgitpath must be called on both sides
of the comparison. Otherwise comparisons involving symlinks will
return incorrect results.
* Re-enable git test on OS X.
* Return None from unfrackgitpath when path is None.
Per a change in jinja2 2.9, local variables no longer are prefixed
with l_, so this updates AnsibleJ2Vars to pull in all locals (while
excluding some) regardless of name.
Fixes#20063
(cherry picked from commit 4d49b317929b86e1fc1b0cbace825ff73b372dc7)
* Fix `iterators` test for osx.
* Enable `git` test for osx.
* Enable `get_url` test for osx and freebsd.
* Remove unused SNI_URI from uri test.
* Use ansible hosted SNI host oustside docker.
* Enable `uri` test for osx and freebsd.
* Enable `gathering_facts` test for osx.
* Skip single task on osx until test is fixed.
* Add jinja2 groupby filter override to cast namedtuple to tuple. Fixes#20098
* Address some of the requested changes
* Quoting
* Print the python path and version
* Be less explicitly verbose, rely on implicit verbosity
Rather than trying to enumerate tasks or track an ever changing cur_role
flag in PlayIterator, this change simply sets a flag on the last block in
the list of blocks returned by Role.compile(). The PlayIterator then checks
for that flag when the cur_block number is incremented, and marks the role
as complete if the given host had any tasks run in that role.
Fixes#20224
* Only start platform instances with tests selected.
* Enable ios on Shippable.
* Show inventory in explain mode.
* Fix indentation of generated network inventory.
* Update classification of network module_utils.
* Adding support for Amazon ECR
This patch adds a new module named ecr, which can create, update or
destroy Amazon EC2 Container Registries. It also handles the management
of ECR policies.
* ecs_ecr: addressed review feeback
* Renaming ecr to ecs_ecr
* Fixed docs
* Removed bad doc about empty string handling
* Added example of `delete_policy`
* Removed `policy_text` option; switched policy to `json` type so
it can accept string or dict
* Added support for specifying registry_id
* Added explicit else after returned if clauses
* Added `force_set_policy` option
* Improved `set_repository_policy` error handling
* Fixed policy comparisons when AWS doesn't keep the ordering stable
* Moved `boto_exception` into the module
* Conditional include on ansible_network_os
* copy & paste error
* More tests
* More tests
* junos tests (based on vyos)
* remove excessive whitespace
* Pass in ansible_network_os
* net_command for ios
* consistent debug
* wrapp line
* ansible-test changes made in another PR
* ansible-test changes made in another PR
map + extract is the usual way to use it but map isn't available on
older versions of jinja2 that we still work with. Test extract even on
those versions.
* Only add Content-Type if not specified in headers. Fixes#20046
* Update documentation to indicate body_format will not override Content-Type if specified in headers
- Replace nose usage with pytest.
- Remove legacy Shippable integration.sh.
- Update Makefile to use pytest and ansible-test.
- Convert most yield unit tests to pytest parametrize.
* Fix a test failure on Python 3.6
tox -e py36 failed with
======================================================================
ERROR: test_action_base__execute_module (units.plugins.action.test_action.TestActionBase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/mg/src/ansible/test/units/plugins/action/test_action.py", line 507, in test_action_base__execute_module
self.assertEqual(action_base._execute_module(), dict(_ansible_parsed=True, rc=0, stdout="ok", stdout_lines=['ok']))
File "/home/mg/src/ansible/lib/ansible/plugins/action/__init__.py", line 596, in _execute_module
remote_module_path = self._connection._shell.join_path(tmp, remote_module_filename)
File "/home/mg/opt/python36/lib/python3.6/unittest/mock.py", line 939, in __call__
return _mock_self._mock_call(*args, **kwargs)
File "/home/mg/opt/python36/lib/python3.6/unittest/mock.py", line 1005, in _mock_call
ret_val = effect(*args, **kwargs)
File "/home/mg/src/ansible/.tox/py36/lib/python3.6/posixpath.py", line 92, in join
genericpath._check_arg_types('join', a, *p)
File "/home/mg/src/ansible/.tox/py36/lib/python3.6/genericpath.py", line 149, in _check_arg_types
(funcname, s.__class__.__name__)) from None
TypeError: join() argument must be str or bytes, not 'MagicMock'
because os.path.join() now checks argument types since Python 3.6 (due
to pathlib support, I expect).
* Use a more realistic module name in test
When using AWS we have to use the full domain name in the inventory file, which
we rather than the short name. This change avoids that ending up being
set in the tests.
The behavior now matches GNU diff.
Fixes#14094.
Example of output before this change:
TASK [healthchecks.io : hourly healthchecks.io ping] ***************************
changed: [ranka]
--- before: /etc/cron.hourly/mg-healthchecks-dot-io
+++ after: /tmp/tmpOTvXTw
@@ -1,2 +1,2 @@
#!/bin/sh
-curl -sS https://hchk.io/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx > /dev/null+curl -sS https://hchk.io/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx > /dev/null
after this change:
TASK [healthchecks.io : hourly healthchecks.io ping] ***************************
changed: [ranka]
--- before: /etc/cron.hourly/mg-healthchecks-dot-io
+++ after: /tmp/tmpOTvXTw
@@ -1,2 +1,2 @@
#!/bin/sh
-curl -sS https://hchk.io/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx > /dev/null
\ No newline at end of file
+curl -sS https://hchk.io/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx > /dev/null
The added unit tests contain more examples.
This commit also takes care to avoid "no newline at EOF" warnings when
no_log is in effect, and also when modules return dicts rather than
strings. (It also removes trailing whitespace from using json
serialization when diffing dicts, because I hate trailing whitespace in
Python source files, even if they're test files.)
* updates the ios_config module to use the network_cli plugin
* updates the local action plugin to derive from network
* add unit test cases for ios_config
* updates the deprecated ios_template module to use network_cli
* adds unit test cases for ios_template
* adds check for provider argument and displays warning message
* Cleanup git tests
* Split git tests in seperate files
* Remove use of repo_depth_url
* Use native yaml
* Remove unnecessary remote/local clones
* Fix newlines for yamllint
* If the hash is valid (full-length) but doesn't exist, git returns 128 instead of 1.
* Ensure git doesn't use hardlinks for shallow clones
* Reenable yum install root tests
No need for sos to test installroot. Something with less deps works
just as well.
* Fix yum installroot.
Fix module import to use fail_json when the modules aren't installed.
Remove wildcard imports
* Lsat task is supposed to remove sos so make that happen
* Add --installroot to YUM and DNF modules, issue #11310
This continues ansible-modules-core#1558, and
ansible-modules-core#1669
Allow specifying installroot for the yum and dnf modules
to install and remove packages in a location other than /.
* Remove empty aliases
* Simpler installroot set default logic
* Add a encode() to AnsibleVaultEncryptedUnicode
Without it, calling encode() on it results in a bytestring
of the encrypted !vault-encrypted string.
ssh connection plugin triggers this if ansible_password
is from a var using !vault-encrypted. That path ends up
calling .encode() instead of using the __str__.
Fixes#19795
* Fix str.encode() errors on py2.6
py2.6 str.encode() does not take keyword arguments.
* Refactoring: split readkeys() into readfile() and parsekeys()
* Refactoring: split writekeys() into writefile() and serialize()
* authorized_key: support --diff
* Refactoring: remove no-longer used readkeys()/writekeys()
* Integration test for authorized_key in check mode
Support for the Google API and GCloud-Python Clients have been added.
The three libraries:
* GCloud-Python: A new function, get_google_cloud_credentials, should be used. The credentials-object returned can be passed to any gcloud-python client. Using this client library requires in the installation of gcloud-python. This is preferred library for new modules.
* Google API: A new function, gcp_api_auth, should be used to take advantage of services requiring this client. This client library should be used if the desired functionality is not available in GCloud-Python. Using this library requires the installation of google-api-python-client.
* libcloud: Existing function, gcp_connect, should be used. The interface and return values have not changed and existing modules (such as gce, gce_pd and gce_net) should work without modification. Note that the credentials-fetching code has been refactored out of gcp_connect so that can be reused by all connection functions. To use this function, apache-libcloud must be installed.
Import guards have been added and will only be trigger if a user tries to use a function that is missing dependencies.
Credential-specifying mechanisms (i.e, ansible module params, env vars and libcloud secrets.py) have not changed. They have been refactored and unit tests have been added to allow for changes going forward. We are deprecating (and removing in a subsequent release) the ability to specify credentials via the libcloud secrets file. Also, we have deprecated (and also plan to remove in a subsequent release) the ability to use a p12 pem file for a key - the JSON format is strongly preferred. Deprecation warnings have been added for both of these issues (see the Ansible docs on how to disable deprecation warnings).
The gce_tag module can support updating tags on multiple instances via an instance_pattern field. Full Python regex is supported in the instance_pattern field.
'instance_pattern' and 'instance_name' are mutually exclusive and one must be specified.
The integration test for the gce_tag module has been updated to support the instance_pattern parameter. Unit tests have been added to test the list-manipulation functionality.
Run the integration test with:
TEST_FLAGS='--tags "test_gce_tag"' make gce
Run the unit tests with:
python test/units/modules/cloud/google/test_gce_tag.py
This plugin can be used with the lpass cli interface for lastpass.
[lastpass-cli](https://github.com/lastpass/lastpass-cli)
Example:
Add a lookup to your playbooks/variables somewhere:
```
some_variable: "{{ lookup('lastpass','Some Lastpass entry name or ID', field='username') }}"
```
Usage:
* start a lpass session prior to using ansible
* run ansible
* logout when finished
```
lpass login user@domain.com
ansible-playbook foo.yml
lpass logout
```
An inner single-quote pair breaks out of the outer single-quote
pair. Rather than escaping the inner quotes to protect against
this, just use the fact that `str()` is equivalent to `""`.
* Initial Commit for Infinidat Ansible Modules
Skip tests for python 2.4 as infinisdk doesn't support python 2.4
Move common code and arguments into module_utils/infinibox.py
Move common documentation to documentation_fragments. Cleanup Docs and Examples
Fix formating in modules description
Add check mode support for all modules
Import AnsibleModule only from ansible.module_utils.basic in all modules
Skip python 2.4 tests for module_utils/infinibox.py
Documentation and code cleanup
Rewrite examples in multiline format
Misc Changes
Test
* Add Infinibox modules to CHANGELOG.md
* Add ANSIBLE_METADATA to all modules
Since we no longer use a post-validated task in _process_pending_results, we
need to be sure to template fields used in original_task as they are raw and
may contain variables.
This patch also moves the handler tracking to be per-uuid, not per-object.
Doing it per-object had implications for the above due to the fact that the
copy of the original task is now being used, so the only sure way is to track
based on the uuid instead.
Fixes#18289
Print out the data that fails to validate when doing
schema checking on modules
This allows easier interpretation of error messages.
From:
```
ERROR: DOCUMENTATION.notes.2: expected basestring
```
To:
```
ERROR: DOCUMENTATION.notes.2: expected basestring @ data['notes'][2].
Got {"As with C(include) this task can be static or dynamic, If static
it implies that it won't need templating nor loops nor conditionals and
will show included tasks in the --list options. Ansible will try to
autodetect what is needed, but you can set `static": 'yes|no` at task
level to control this.'}
```
* Code smell test for iteritems and itervalues
* Change the keydict object in authorized_keys so it doesn't throw a false postive
keydict is a bad data structure anyway. We don't use the iteritems and
itervalues methods so just disable them so that the code-smell tests do
not trigger on it.
* Change release templates so they work with py3
* Fix UnboundLocalError remote_head in git
Fixes#5505
The use of remote_head was a leftover of #4562.
remote_head is not necessary, since the repo is unchanged anyway and
after is set correctly.
Further changes:
* Set changed=True and msg once local_mods are detected and reset.
* Remove need_fetch that is always True (due to previous if) to improve
clarity
* Don't exit early for local_mods but run submodules update and
switch_version
* Add test for git with local modifications
* Enable tests on python 3 for uri
* Added one more node type to SAFE_NODES into safe_eval module.
ast.USub represents unary operators. This is necessary for
parsing some unusual but still valid JSON files during testing
with Python 3.
* Use native yaml for apache2 test
* Test removal of default modules with force
a2enmod on debian has `-f`, but not on SUSE (runs there without force).
Therefore don't test that option on SUSE.
The docs already specify that the option is intended for Debian systems
only.
* Fix synchronize retries
The synchronize module munges its task args on every invocation of
run(). This was problematic because the munged data was not fit for use
by a second pass of the synchronize module. Correct this by using a copy
of the task args on every invocation of run() so that the original args
are not affected.
Local testing using this playbook seems to confirm that things work as
expected:
- hosts: all
tasks:
- delay: 2
register: task_result
retries: 1
until: task_result.rc == 0
synchronize:
dest: /tmp/out
mode: pull
src: /tmp/nonexistent/
fixes#18281
* Update synchroncization fixture assertions
When we started operating on a copy of the task args the test assertions
were no longer asserting things about the munged state but of the
pristine state. Convert the copy of task args to a class member so that
it can be compared against later in testing and update the assertions to
check this munged copy.
* Shuffle objects around for cleaner testing
Attach the temporary args dict to the task rather than the action as
this makes updating the existing tests cleaner.
This adds back the change to the network_cli plugin. Ths change adds
the ensure_connect decorator to the open_shell() method to make sure
the connection is valid before trying to open a shell.
The issue was due to the addition of the decorator that will call
_connect() when there is no connection. The _connect() method should
have been mocked in the test case. This commit fixes the test
case as well
Change was originally reverted in c414ded69a
* make hash_params more robust in the face of many corner cases
Fixes#18680
Alternative fix to #18681
* add test case for role.hash_params
* Add role.hash_params test for more types
A set, a generator/iterable, and a Container that
is not Iterable.
* Fix regression in jinja2 include search path
Since commit 3c39bb5, the 'ansible_search_path' variable is used to set
jinja2's search path for {% include %} directives. However, this path is
the the proper one because our templates live in 'templates' subdirs in
our search path.
This is a regression because previously, our include search path would
include the dirname of the currently interpreted file, which worked most
of the time.
fixes#18526
* Fix template lookup search path
Improve fix in commit c96c853 so that the search path contain both
template-suffixed paths as well as original paths.
ref PR #18617
* Add integration test for template lookups
Tests regression at #18526
This test fails on current devel branch and succeeds on PR #18617
* wip: add a unit test for playbook/base.py
This commit include a failing test
TestBaseSubClass.test_attr_class_post_validate
It fails with the error:
Traceback (most recent call last):
File "/home/adrian/src/ansible/test/units/playbook/test_base.py", line 264, in test_attr_class_post_validate
bsc = self._base_validate(ds)
File "/home/adrian/src/ansible/test/units/playbook/test_base.py", line 206, in _base_validate
bsc.post_validate(templar)
File "/home/adrian/src/ansible/lib/ansible/playbook/base.py", line 450, in post_validate
" Error was: %s" % (name, value, attribute.isa, e), obj=self.get_ds())
AnsibleParserError: the field 'test_attr_class_post_validate' has an invalid value (<class 'units.playbook.test_base.ExampleSubClass'>), and could not be converted to an class. Error was: test_attr_class_post_validate is not a valid <class 'units.playbook.test_base.ExampleSubClass'> (got a <class 'ansible.playbook.base.BaseMeta'> instead)
* wip, test refactoring
* wip, trying to add a parent->child
* wip, fix isa=class.
the ds the base using needs an instance of the class
(ie, whats normally created by the yaml loaders)
* wip, theres no need to argue, I just dont understand parents
* stub a _preprocess_data for coverage
* cleanup, required, parent, etc
* Make sure include_role inherit variables from parent role
Setting the parent of task blocks generated by include_role after they
have been produced is not sufficient - it means the tasks don't have the
correct dependency chain set afterwards, and therefore, don't properly
inherit variables from outer roles.
In addition to manually setting the parents, pass the dep_chain when
compiling the role, such that variables are correctly imported.
Fixes#18540.
* Add tests for include_role
* Fix include_role variable inheritance for multiple parent levels
* Add test cases for VyOS commands that don't honor paging settings
Testing for issue fixed in PR #18546
* Add provider line and fix indentation
For the way we invoke the tests we need to specify the `provider:`
Also fix the indentation on `register:`
* Replace pipes.quote for shlex_quote
* More migration of pipes.quote to shlex_quote
Note that we cannot yet move module code over. Modules have six-1.4
bundled which does not have shlex_quote. This shouldn't be a problem as
the function is still importable from pipes.quote. It's just that this
has become an implementation detail that makes us want to import from
shlex instead.
Once we get rid of the python2.4 dependency we can update to a newer
version of bundled six module-side and then we're free to use
shlex_quote everywhere.
If the current ansible enviroment has a config setup
that doesn't use 'smart' as the configured transport
test_play_context would fail when it assumes the
transport will be 'smart'.
Previous changes addressed a corner case, which unfortunately introduced
another bug. This patch adds a new flag to the host state (did_rescue) which
is set to true when the rescue portion of a block completes. This flag is
then checked in _check_failed_state() when the fail_state != FAILED_NONE.
This lead to the discovery of another bug - current strategies are not advancing
hosts to ITERATING_COMPLETE after doing a peek at the next task, leaving the
host state in the run_state of the final task. To address this, before gathering
the list of failed hosts in StrategyBase.run(), a final pass through the iterator
for all hosts is done to ensure each host is in its final state. This way, no
strategy derived from StrategyBase has to worry about it and it's handled.
Fixes#17983
This commit extends YAML linting by enabling standard rules from the
`yamllint` tool [1]. Since syntax errors and key duplicates are already
checked since 4d48711, this change only adds detection for cosmetic
problems. It also narrows checks to the test/ dir only.
The main goal is to prevent future problems to enter the code base
without being noticed. While it would be a huge effort to be PEP8
compliant, it is relatively easy to have correct YAML style *now* and
prevent future errors by enabling linting.
Note: for those (like me) caring about code attribution: use `git blame
-w` to ignore whitespace-only changes.
Note: I disabled some linting checks (such as indentation), they can be
enforced in the future if needed. Similarly, current checks can also be
disabled. See the `.yamllint` file.
[1]: https://yamllint.readthedocs.io/
This change corrects problems reported by the `yamllint` linter.
Since key duplication problems were removed in 4d48711, this commit
mainly fixes trailing spaces and extra empty lines at beginning/end of
files.
* Added test for sequenced-name instance generation (num_instances)
* Added param-check tags to tests that only do argument checking
Should be merged AFTER ansible/ansible-modules-core#4276
- Use assertRaisesRegexp to make sure correct exceptions are raised.
- Set docker_command to avoid docker dependency (skips find_executable).
- Use a fake path for docker_command to make sure mock.patch is working.
* Fix bug (#18355) where encrypted inventories fail
This is first part of fix for #18355
* Make DataLoader._get_file_contents return bytes
The issue #18355 is caused by a change to inventory to
stop using _get_file_contents so that it can handle text
encoding itself to better protect against harmless text
encoding errors in ini files (invalid unicode text in
comment fields).
So this makes _get_file_contents return bytes so it and other
callers can handle the to_text().
The data returned by _get_file_contents() is now a bytes object
instead of a text object. The callers of _get_file_contents() have
been updated to call to_text() themselves on the results.
Previously, the ini parser attempted to work around
ini files that potentially include non-vailid unicode
in comment lines. To do this, it stopped using
DataLoader._get_file_contents() which does the decryption of
files if vault encrypted. It didn't use that because _get_file_contents
previously did to_text() on the read data itself.
_get_file_contents() returns a bytestring now, so ini.py
can call it and still special case ini file comments when
converting to_text(). That also means encrypted inventory files
are decrypted first.
Fixes#18355
This allows validate-modules to run in an environment where
python 3 is the default. This will no longer be necessary once
validate-modules is updated to work with both python 2 and 3.
- Remove shebangs from:
- ini files
- unit tests
- module_utils
- plugins
- module_docs_fragments
- non-executable Makefiles
- Change non-modules from '/usr/bin/python' to '/usr/bin/env python'.
- Change '/bin/env' to '/usr/bin/env'.
Also removed main functions from unit tests (since they no longer
have a shebang) and fixed a python 3 compatibility issue with
update_bundled.py so it does not need to specify a python 2 shebang.
A script was added to check for unexpected shebangs in files.
This script is run during CI on Shippable.
- Update import for relocated tests.
- Fix test to expect changed from update_tags.
- Add checks for boto3 and botocore to tests.
- Set check mode with kwarg.
- Python 3 fixes for unit tests.
- Python 2.6 fix for unit tests.
- Add missing meta value for test_create_server
- Add .gitignore for pytest .cache directory
Exclude test_os_server from nose test runs since it was designed
for pytest. The test will work correctly when run using pytest.
This is a temporary issue, as we'll be moving to pytest soon.
This commit adds some unit tests for the `cloud.openstack.os_server`
module. These tests exercise `_network_args` thoroughly and
`_create_server` lightly.
These tests will **fail** until ansible/ansible-modules-core#2275 lands.
To run the tests:
pip install -r test-requirements.txt
PYTHONPATH=$PWD py.test
Originally from ansible/ansible-modules-core@3387526bca
- Correct directory name in test/README.md
- Move code-smell tests to test/sanity/code-smell
- Update code-smell.sh to use new script paths
- Add test/integration/target-prefixes.win for ansible-test
- Move module unit tests to match module directory layout
* Network Test Documentation
Will need improving over time, though this ensure that everything that was in `ansible/test-network-modules` is in `ansible/ansible`
* Update README.md
* Inventory file
* Network module prefixes
In ansible-test we should skip tests for these modules, they will be
tested via another process.
* Update target-prefixes.network
In order to support legacy plugins, the following two method signatures
are allowed for `CallbackBase.v2_playbook_on_start`:
def v2_playbook_on_start(self):
def v2_playbook_on_start(self, playbook):
Previously, the logic to handle this divergence checked to see if the
callback plugin being called supported an argument named `playbook`
in its `v2_playbook_on_start` method. This was fragile in a few ways:
- if a plugin author did not use the literal `playbook` to name their
method argument, their plugin would not be called correctly
- if a plugin author wrapped their `v2_playbook_on_start` method and
by doing so changed the argspec to no longer expose an argument
with that literal name, their plugin would not be called correctly
In order to continue to support both types of callback for backwards
compatibility while making the call more robust for plugin authors,
the logic can be reversed in order to have a positive check for the old
method signature instead of a positive check for the new one.
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>
As neon is derived from Ubuntu, ansible_os_family should have the value
"Debian" instead of "Neon". Add a test case for KDE neon and set
os_family correctly for it.
If the facts returned by setup included strings that
had double quotes in them, the asserts in test_gathering_facts.yml
would fail with errors like:
"The conditional check '\"[{u'mounts': {u'options':
u'rw,context=\"system_u:\"'}}]\" != \"UNDEF_HW\"' failed. The error was:
template error while templating string: expected token 'end of statement
block', got 'system_u'. String: {% if \"[{u'mounts': {u'options':
u'rw,context=\"system_u:\"'}}]\" != \"UNDEF_HW\" %} True {% else %}
False {% endif %}"
For one example, if mount facts returned an 'options' field that
included double quoated selinux context ids, the test would fail.
Fix is removing the double quoting in the assert 'that:' lines,
and removing the unneeded double curly brackets.
Nothing seems to use this now.
Was added originally added in2d11cfab92f9d26448461b4bc81f466d1910a15e
but the code that used it was removed in
e02b98274b
On openSUSE Tumbleweed, lsb-release -a currently reports
the distributor ID as "openSUSE Tumbleweed". On openSUSE
Leap, the distributor ID is "SUSE LINUX".
Add them to the OS_FAMILY dict as Suse family systems.
Also add an entry to TESTSETS in test_distribution_version.py
for openSUSE Tumbleweed.
Since passlib algo sometime takes a bytes, and sometime
not, depending on a internal variable, we have to convert
bnased on it, or it fail with "TypeError: salt must be bytes,
not str" (or unicode instead of bytes)
However, that's not great to use internal structure for that.
* Add tag verification test (ansible-modules-core PR 2654)
* Fix typo
* Use smaller repo for testing, add dependency control
* Test is gpg exists before running git signing tasks
* Correct the test conditionals so that gpg1 is tested
Currently (pre-repomerge) we aren't running sanity.sh from
ansible/ansible, after the merge we will. Therefore I've added the
requirements here, rather than in ansible-modules-*/test/utils/shippable
Implement tag and skip_tag handling in the CLI() class. Change tag and
skip_tag command line options to be accepted multiple times on the CLI
and add them together rather than overwrite.
* Make it configurable whether to merge or overwrite multiple --tags arguments
* Make the base CLI class an abstractbaseclass so we can implement
functionality in parse() but still make subclasses implement it.
* Deprecate the overwrite feature of --tags with a message that the
default will change in 2.4 and go away in 2.5.
* Add documentation for merge_multiple_cli_flags
* Fix galaxy search so its tags argument does not conflict with generic tags
* Unit tests and more integration tests for tags
Prior to this commit, the ini parser would fail if the inventory was
not 100% utf-8. This commit makes this slightly more robust by
omitting full line comments from that requirement.
Fixes#17593
* Remove unicode-escape which is not present on python3
Alternative fix for #17305
* Enable the assemble test on python3
* Fix other problems with assemble on python3
If the sftp fails, roll over to scp by default. This saves users
from having to know about the scp_if_ssh method when sftp is broken
on the remote host.
* Turn mount test back on
* Mount tests need PRIVILEGED so turn that back on
Revert "Revert "Set PRIVILEGED=true for non_destructive tests. (#17733)" (#17738)"
This reverts commit dc0fb1c212.
* Add a needs_privileged tag so that we can skip mount tests on centos6
Some containers timeout on shippable tests when run with privileged.
Unfortunately, some tests require that in order to run. Tagging those
allows us to skip those tests on the platforms that timeout when we get
ready to run the integration test in shippable.
* Centos6 times out with PRIVILEGED set so remove that (will disable the mount tests on centos6)
* Remove false start
Python2 seems to allow any integer. Python3.5 on Linux seems to allow
a 32 bit unsigned int. Python3.5 on El Capitan seems to limit it to
a smaller size... perhaps a 16 bit int.
This adds some test data to test_facts.py that
includes mnt points that have a single quote in
the path.
Ala, https://github.com/ansible/ansible/issues/16855
The bug was already fixed via other changes, but this is
for regression testing.
* Add tests for the mount module
* Switch from unmounted to absent...
the code for mounting always modifies fstab so we need to always modify
fstab to cleanup as well.
* Fix comments and copyright
A parameter of type int should accept int and string, but not float.
A parameter of type float should accept float, int, and string.
Also reset the arguments in another test so that it runs cleanly. This
agrees with what all the other tests are doing.
Includes:
* Tests to create, delete, resize, recreate and configure an Autoscaler
* Tests for parameters only. Use TEST_FLAGS="--tags 'param-check'" to execute only these tests.
Requires:
* An instance template to be configured in your project.
* Improve unit testing of 'password' lookup
The tests showed some UnicodeErrors for the
cases where the 'chars' param include unicode,
causing the 'getattr(string, c, c)' to fail.
So the candidate char generation code try/excepts
UnicodeErrors there now.
Some refactoring of the password.py module to make
it easier to test, and some new tests that cover more
of the password and salt generation.
* More refactoring and fixes.
* manual merge of text enc fixes from pr17475
* moving methods to module scope
* more refactoring
* A few more text encoding fixes/merges
* remove now unused code
* Add test cases and data for _gen_candidate_chars
* more test coverage for password lookup
* wip
* More text encoding fixes and test coverage
* cleanups
* reenable text_type assert
* Remove unneeded conditional in _random_password
* Add docstring for _gen_candidate_chars
* remove redundant to_text and list comphenesion
* Move set of 'chars' default in _random_password
on py2, C.DEFAULT_PASSWORD_CHARS is a regular str
type, so the assert here fails. Move setting the
default into the method and to_text(DEFAULT_PASSWORD_CHARS)
if it's needed.
* combine _random_password and _gen_password
* s/_create_password_file/_create_password_file_dir
* native strings for exception msgs
* move password to_text to _read_password_file
* move to_bytes(content) to _write_password_file
* add more test assertions about genned pw's
* Some cleanups to alikins and abadger's password lookup refactoring:
* Make DEFAULT_PASSWORD_CHARS into a text string in constants.py
- Move this into the nonconfigurable section of constants.
* Make utils.encrypt.do_encrypt() return a text string because all the
hashes in passlib should be returning ascii-only strings and they are
text strings in python3.
* Make the split up of functions more sane:
- Don't split such that conditionals have to occur in two separate functions.
- Don't go overboard: Good to split file system manipulation from parsing
but we don't need to do every file manipulation in a separate
function.
- Don't split so that creation of the password store happens in two
parts.
- Don't split in such a way that no decisions are made in run.
* Organize functions by when it gets called from run().
* Run all potential characters through the gen_candidate_chars function
because it does both normalization and validation.
* docstrings for functions
* Change when we store salt slightly. Store it whenever it was already
present in the file as well as when encrypt is requested. This will
head of potential idempotence bugs where a user has two playbook tasks
using the same password and in one they need it encrypted but in the
other they need it plaintext.
* Reorganize tests to follow the order of the functions so it's easier
to figure out if/where a function has been tested.
* Add tests for the functions that read and write the password file.
* Add tests of run() when the password has already been created.
* Test coverage currently at 100%
As suggested in feedback on
https://github.com/ansible/ansible/pull/17575, add
os_family to test_distribution_version. Add the
correct os_family to the existing testcase data
entries.
Also add os_family to the output of
gen_distribution_version_testcase.py so any new
generated entries will contain this data.
* Make is_encrypted_file handle both files opened in text and binary mode
On python3, by default files are opened in text mode. Since we know
the encoding of vault files (and especially the header which is the
first set of bytes) we can decide whether the file is an encrypted
vault file in either case.
* Fix is_encrypted_file not resetting the file position
* Update is_encrypted_file to check that all the data in the file is ascii
* For is_encrypted_file(), add start_pos and count parameters
This allows callers to specify reading vaulttext from the middle of
a file if necessary.
* Combine VaultLib.encrypt() and VaultLib.encrypt_bytestring()
* Change vault's is_encrypted() to take either text or byte strings and to return False if any part of the data is non-ascii.
* Remove unnecessary use of six.b
* Vault Cipher: mark a few methods as private.
* VaultAES256._is_equal throws a TypeError if given non byte strings
* Make VaultAES256 methods that don't need self staticmethods and classmethods
* Mark VaultAES and is_encrypted as deprecated
* Get rid of VaultFile (unused and feature implemented in a different way)
* Normalize variable and parameter names on plaintext, ciphertext, vaulttext
* Normalize variable and parameter names on "b_" prefix when dealing with bytes
* Test changes:
* Remove redundant tests( both checking the same byte string)
* Fix use of format string without format operator
* Enable vault editor tests on python3
* Initialize the vault_cipher for VaultAES256 testing in setUp()
* Make assertTrue and assertFalse take the actual method calls for
better error messages.
* Test that non-ascii byte strings compare correctly.
* Test that unicode strings and ints raise TypeError
* Test-specific:
* Removed test_methods_exist(). We only have one VaultLib so the
implementation is the assurance that the methods exist. (Can use an abc for
this if it changes).
* Add tests for both byte string and text string input where the API takes either.
* Convert "assert" to unittest assert functions or add a custom message where
that will make failures easier to debug.
* Move instantiating the VaultLib into setUp().
* Added aws_retry decorator function with unit tests
* Restructured the code to be used with a base class.
This base class CloudRetry can be reused by any other cloud provider.
This decorator should be used in situations, where you need to implement
a backoff algorithm and want to retry based on the status code from the
exception.
* updated documentation
* fixed tabs
* added botocore and boto3 to requirements.txt
* removed cloud.py from py24 tests, as it depends on boto3
* fix relative imports
* updated test to be 2.6 compat
* updated method name from retry to backoff
* readded lxd
* Updated default backoff from 2 seconds to 1.1s.
This will be about a total of 48 seconds in 10 tries. This is
configurable.
* Fixes to the controller text model
* Change command line args to text type
* Make display replace undecodable bytes with replacement chars. This
is only a problem on pyhton3 where surrogates can enter into the msg
but sys.stdout doesn't know how to handle them.
* Remove a deprecated playbook syntax in unicode.yml
* Fix up run_cmd to change its parameters to byte string at appropriate times.
* Enable more integration tests for python 3.
* Split out python 3 integration tests.
Now that we're running more integration tests on python 3, the
tests are taking long enough that they warrant splitting out in
the same way the python 2 tests are split.
We couldn't copy to_unicode, to_bytes, to_str into module_utils because
of licensing. So once created it we had two sets of functions that did
the same things but had different implementations. To remedy that, this
change removes the ansible.utils.unicode versions of those functions.
The test_async test target was updated to accommodate changes in
output buffering behavior in python 3. This change in behavior
may need to be addressed in the future.
* Fix paramiko's exec_command() to return bytes on python3
* Run test_connection for python3 now too
* Fix atomic_move for problem in shippable's testing
* Python-2.4 needs to use b()
for `VariableManager._get_magic_variables()`.
This saves a lot of time re-iterating the nearly always constant global
list of groups and their members.
Generate once and cache, and invalidate cache in case `add_host:` or
`group_by:` are used.
* Port set_*_if_different functions to python3
* Add surrogate_or_strict and surrogate_or_replace error handlers for
to_text, to_bytes, to_native
* Set default error handler to surrogate_or_replace
* Make use of the new error handlers in the already ported code
* Move the unittests for module_utils._text as they aren't in basic.py
* Cleanup around SEQUENCETYPE. On python2.6+ SEQUENCETYPE includes
strings so make sure code omits those explicitly if necessary
* Allow arg_spec aliases to be other sequence types
* New features for include_vars
include_vars.py now allows you to include an entire directory and its nested directories of variable files.
Added Features..
* Ignore by default *.md, *.py, and *.pyc
* Ignore any list of files.
* Only include files nested by depth (default=unlimited)
* Match only files matching (valid regex)
* Sort files alphabetically and load in that order.
* Sort directories alphabetically and load in that order.
```
- include_vars: 'vars/all.yml'
- name: include all.yml
include_vars:
file: 'vars/all.yml'
- name: include all yml files in vars/all and all nested directories
include_vars:
dir: 'vars/all'
- name: include all yml files in vars/all and all nested directories and save the output in test.
include_vars:
dir: 'vars/all'
name: test
- name: include all yml files in vars/services
include_vars:
dir: 'vars/services'
depth: 1
- name: include only bastion.yml files
include_vars:
dir: 'vars'
files_matching: 'bastion.yml'
- name: include only all yml files exception bastion.yml
include_vars:
dir: 'vars'
ignore_files: 'bastion.yml'
```
* Added whitelist for file extensisions (yaml, yml, json)
* Removed unit tests in favor of integration tests
While trying to fix the test suite on python3, I noticed
this test fail due to to_json adding more whitespace in
python3 than in python2. So -w should ignored those
differences.
* Fix to_native call in selinux_context and selinux_default_context to
use the error handler correctly.
* Port set_mode_if_different to work on python3
* Port atomic_move to work on python3
* Fix check_password_prompt variable which wasn't renamed properly
tempfile.NamedTemporaryFile keeps a file handle causing os.rename() to fail with windows based vboxfs: [Errno 26] Text file busy.
Changed NamedTemporaryFile to mkstemp() and added a finally block to unlink the temp file in each and every case.
* run_command needed a bit of tweaking to its string handling of
arguments.
* The run_command change fixes the last bit of lineinfile so we can
enable its tests
- Fix octal formatting of file mode in module response on py3.
- Convert file path to unicode in copy action.
- Enable file and copy module tests for py3 now that they pass.
Make some python3 fixes to make the unittests pass:
* galaxy imports
* dictionary iteration in role requirements
* swap_stdout helper for unittests
* Normalize to text string in a facts.py function
Fixes for these are either rewriting to get rid of the need for the
functions or using six.moves to get equivalent functions for both
python2 and python3
Make !vault-encrypted create a AnsibleVaultUnicode
yaml object that can be used as a regular string object.
This allows a playbook to include a encrypted vault
blob for the value of a yaml variable. A 'secret_password'
variable can have it's value encrypted instead of having
to vault encrypt an entire vars file.
Add __ENCRYPTED__ to the vault yaml types so
template.Template can treat it similar
to __UNSAFE__ flags.
vault.VaultLib api changes:
- Split VaultLib.encrypt to encrypt and encrypt_bytestring
- VaultLib.encrypt() previously accepted the plaintext data
as either a byte string or a unicode string.
Doing the right thing based on the input type would fail
on py3 if given a arg of type 'bytes'. To simplify the
API, vaultlib.encrypt() now assumes input plaintext is a
py2 unicode or py3 str. It will encode to utf-8 then call
the new encrypt_bytestring(). The new methods are less
ambiguous.
- moved VaultLib.is_encrypted logic to vault module scope
and split to is_encrypted() and is_encrypted_file().
Add a test/unit/mock/yaml_helper.py
It has some helpers for testing parsing/yaml
Integration tests added as roles test_vault and test_vault_embedded
* Give native strings to selinux library functions.
SELinux takes pathnames as native strings. That means we need to
convert to bytes on python2 and convert to text on python3.
Fixes#17155
* Read kitchen documentation, make module_utils params more like kitchen API
* Remove none nonstring strategy and add strict
* Raise TypeError on invalid nonstring strategy
* Document to_native()
* Make unittests for testing module_utils.text
* Rm py2.7+ code in docker connection plugin
The docker connection plugin was using subprocess.check_output
which only exists in python 2.7 and later. Connection plugins
need to support python2.6 so this replaces it with Popen/communicate()
* Handle docker ver errors in docker connection
Add unit tests for DockerConnection
Fixes#16971
test/units/plugins/action/test_action.py had code
for handling a bug in python 3.4's mock_open that
causes errors when reading binary data.
Moved to compat/tests/mock.py so other tests can
use it by default.
* actions/unarchive: fix unarchive from remote url
Currently unarchive from remote url does not work because the core
unarchive module was updated to support 'remote_src' [1], but the
unarchive action plugin was not updated for this. This causes failures
because the action plugin assumes it needs to copy a file to the
remote server, but in the case of downloading a file from a remote
url a local file does not exist, so an error occurs when the file is
not found.
[1] https://github.com/ansible/ansible-modules-core/commit/467516e
* test_unarchive: fix test with wrong remote_src use
The non-ascii filenames test had improperly set remote_src=yes even
though it was actually copying the file from the local machine (i.e.
the file did not already exist remotely). This test was passing
until the remote_src behavior of unarchive was fixed in 276550f.
Fixes#10779
Refactor some of the block device, mount point, and
mtab/fstab facts collection for linux for better
performance on systems with lots of block devices.
Instead of invoking 'lsblk' for every entry in mtab,
invoke it once, then map the results to mtab entries.
Change the args used for invoking 'findmnt' since the
previous combination of args conflicts, so this would
always fail on some systems depending on version.
Add test cases for facts Hardware()/Network()/Virtual() classes
__new__ method and verify they create the proper subclass based
on the platform.system() results.
Split out all the 'invoke some command and grab it's output'
bits related to linux mount paths into their own methods so
it is easier to mock them in unit tests.
Fix the DragonFly* classes that did not defined a 'platform'
class attribute. This caused FreeBSD systems to potentially
get the DragonFly* subclasses incorrectly. In practice it
didnt matter much since the DragonFly* subclasses duplicated
the FreeBSD ones. Actual DragonFly systems would end up with
the generic Hardware() etc instead of the DragonFly* classes.
Fix Hardware.__new__() on PY3, passing args to __new__
would cause "object() takes no parameters" errors. So
check for PY3 and just call __new__ without the args
See
https://hg.python.org/cpython/file/44ed0cd3dc6d/Objects/typeobject.c#l2818
for some explaination.
Instead of immediately returning a failed code (indicating a break in
the play execution), we internally 'or' that failure code with the result
(now an integer flag instead of a boolean) so that we can properly handle
the rescue/always portions of blocks and still remember that the break
condition was hit.
Fixes#16937
Rather than repeatedly searching for tasks by uuid via iterating over
all known blocks, cache the tasks when they are added to the PlayIterator
so the lookup becomes a simple key check in a dict.
* Query lookup plugin
* Add license and docstrings
* Add python3-ish imports
* Change query plugin type from lookup to filter
* Switch from dq to jsonpath_rw
* Add integration test for query filter
* Rename query filter to json_query
* Add jsonpath-rw
* Rename query filter to json_query
* Switch query implementation from jsonpath-rw to jmespath
Run setfacl/chown/chmod on each temp dir and file.
This fixes temp file permissions handling on platforms such as FreeBSD
which always return success when using find -exec. This is done by
eliminating the use of find when setting up temp files and directories.
Additionally, tests that now pass on FreeBSD have been enabled for CI.
When a task result has an empty results list, the
list should be ignored when determining the results
of `_check_key`. Here the empty list is treated the
same as a non-existent list.
This fixes a bug that manifests itself with squashed
items - namely the task result contains the correct
value for the key, but an empty results list. The
empty results list was treated as zero failures
when deciding which handler to call - so the task
show as a success in the output, but is deemed to
have failed when deciding whether to continue.
This also demonstrates a mismatch between task
result processing and play iteration.
A test is also added for this case, but it would not
have caught the bug - because the bug is really in
the display, and not the success/failure of the
task (visually the test is more accurate).
Fixesansible/ansible-modules-core#4214
This feature changes the scalar value of `serial:` to a list, which
allows users to specify a list of values, so batches can be ramped
up (commonly called "canary" setups):
- hosts: all
serial: [1, 5, 10, "100%"]
tasks:
...
Binary module tests now download pre-compiled binaries from S3
instead of downloading go and compiling the modules.
Files downloaded form SourceForge are now downloaded from S3.
* Enabled more tests in test_service on systems with systemd.
* Fix inconsistency in cleaning up test service. The conditions for service setup and service cleanup were different.
* Add check mode run for service enable.
This is a refactoring of the existing GCE utility module to support other projects on Google Cloud Platform.
The previous gce.py module was hard-coded specifically for GCE, and attempting to use it with other projects in GCP failed.
See https://github.com/ansible/ansible/pull/15918#issuecomment-220165913 for more detail.
This has also been an issue for others in the past, although they've handled it by simply
duplicating some of the logic of gce.py in their own modules.
- The existing gce.py module was renamed to gcp.py, and modified to remove any
imports or other code that refers to libcloud.compute or GCE (the GCE_* params were
retained for compatibility). I also renamed the gce_connect function to gcp_connect,
and modified the function signature to make supplying a provider, driver, and agent
information mandatory.
- A new gce.py module was created to handle connectivity to GCE. It imports the
appropriate libcloud.compute providers and drivers, and then passes them on
to gcp_connect in gcp.py. The constants and function signatures are the same
as the old gce.py, so compatibility with existing modules is retained.
- A new gcdns.py module was created to support PR ansible/ansible-modules-extras#2252
for two new Google Cloud DNS modules, and to demonstrate support for a non-GCE
Google Cloud service. It follows the same basic structure as the new gce.py module,
but imports from libcloud.dns instead.
Enable color output from integration tests in Docker containers:
* In run_tests.sh when output is attached to a terminal.
* In shippable/integration.sh using force mode (can be disabled).
Also fix blocks tests to work with or without color output
* switch cwd to basedir of task
This restores previous behaviour in pre 2.0 and allows for 'local type' plugins
and actions to have a more predictable relative path.
fixes#14489
* removed FIXME since prev commit 'fixes' this
* fix tests, now they need a loader (thanks jimi!)
* add check_mode option for tasks
includes example testcases for the template module
* extend check_mode option
* replace always_run, see also proposal rename_always_run
* rename always_run where used and add deprecation warning
* add some documentation
* have check_mode overwrite always_run
* use unique template name to prevent conflicts
test_check_mode was right before, but failed due to using the same filename as other roles
* still mention always_run in the docs
* set deprecation of always_run to version 2.4
* fix rst style
* expand documentation on per-task check mode
Adds a new test for GalaxyCLI. Tests cases when:
- no actions are provided
- invalid actions are provided
- each valid action is provided
When parser() runs successfully the method returns True, creates a
SortedOptParser instance, a Galaxy instance, and updates certain class data.
A little unittest refactoring
* Add a class decorator to generate tests when using a unittest.TestCase base class
* Add a TestCase subclass with setUp() and tearDown() that sets up
module parameter parsing
* Move test_safe_eval to use the class decorator and ModuleTestCase base
class
* Move testing of set_mode_if_different into its own file and separate
some test methods out so we get better errors and more coverage in
case of errors.
* Naming convention for test cases doesn't need to duplicate information
that's already in the file path.
* Adding a test for GalaxyCLI and two methods on which it depends. Tests execute_info method.
* Adding a test for GalaxyCLI and two methods on which it depends. Tests execute_remove method.
* Update test_galaxy.py
* Revising test for GalaxyCLI execute_remove.
* Removing mocks
I suspect this problem was masked previously as older versions of pywinrm
where significantly slower, allowing more time for the windows installer service to complete.
6eefc11c converted task.loop_control into an object, but while the other
callers were updated to use .loop_var instead of .get('loop_var'), this
site was overlooked.
This can be reproduced by including with loop_control a file that does
set_fact; a simple regression test along these lines is included.
* Lookup unencrypted password must not include salt
* Integration test lookup: remove previous directory
* Test that lookup password doesn't return salt
* Lookup password: test behavior with empty encrypt parameter
Closes#16189
- Do not update base image packages unnecessarily.
- Fix incorrect package names.
- Remove commented out RUN statements.
- Sort list of packages to install.
Due to the fact that roles may be instantiated with different sets of
params (multiple inclusions of the same role or via role dependencies),
simply tracking notified handlers by name does not work. This patch
changes the way we track handler notifications by using the handler
object itself instead of just the name, allowing for multiple internal
instances. Normally this would be bad, but we also modify the way we
search for handlers by first looking at the notifying tasks dependency
chain (ensuring that roles find their own handlers first) and then at
the main list of handlers, using the first match it finds.
This patch also modifies the way we setup the internal list of handlers,
which should allow us to correctly identify if a notified handler exists
more easily.
Fixes#15084
- Reduce image size by skipping weak package references.
- Consolidate and sort packages to install.
- Improve consistency between versions.
- Combine yum/dnf install and clean to avoid unwanted caching.
- Don't update existing packages from base image unnecessarily.
- Reduce image size by skipping recommended packages.
- Consolidate and sort packages to install.
- Improve consistency between Ubuntu versions.
- Combine apt-get update and install to avoid caching stale updates.
* add git shallow fetch test
covers https://github.com/ansible/ansible-modules-core/issues/3782
updating a repo with depth=1 fails silently if version==HEAD
* disable git shallow tests for old git versions
Older git versions don't treat the --depth option correctly.
While the git module tried to work around this and introduced subtle
bugs, ansible/ansible-modules-core#3794 falls back to full checkouts.
Don't run the tests then.
This allows the PlaybookExecutor to receive more information regarding
what happened internal to the TaskQueueManager and strategy, to determine
things like whether or not the play iteration should stop.
Fixes#15523
* Port urls.py to python3
Fixes (largely normalizing byte vs text strings) for python3
* Rework what we do with attributes that aren't set already.
* Comments
* reduce async sleep time in test
* make zypper test less destructive (don't break following uses of zypper)
* fix ca cert on suse
* fix/enable postgres/mysql on opensuseleap
* fix mysql test for mysql versions 5.7.6 and newer
* skip sni_host check on ubuntu1604
* add HTTPTESTER flag for test_uri
ubuntu 16.04 uses dash which drops env variables containing a dot
we work around this by adding an explicit env variable to enable httptester
The output of 'ansible-galaxy info' was formatting the
'galaxy_info' key with one char per line.
Previously, when building the output string, items in
role_info that had a dict for value, the label for
it's key ('galaxy_info' for ex) was being added to
the text list in addition to being appended. Only
the append is needed.
Also added a unit test in test/units/cli/test_galaxy.py,
but skip it on py3 until galaxy is py3 compatible.
fixes#15177
* Merge conflicts:
* [skip ci] Revert changes to run_tests.sh
gundalow will update this in a different PR
* [skip ci] Add in ubuntu1604 and opensuseleap
NOTE: We are not configuring anything to use these new images yet.
Therefore no impact on Travis performance
* python-mysql for opensuse
* It's mysql-server on centos6
* alias module is very basic and removing it leads to the suse default
config failing
* future improvements might test different modules and the effect of
them being removed
* When var name is the same as var content, try to template it before reporting that var is not defined.
Add asserts in test_var_blending to check this special corner case.
* Fix integration tests when using debug with list or dict.
* add tests for centos6, rhel6 and rhel7
* gen_distribution_version_testcase with python2.6
* remove unused imports
* fix redhat/vmware/... parsing
* add centos7 test case
* Make ziploader's ansible and ansible.module_utils libraries into
namespace packages.
* Move __version__ and __author__ from ansible/__init__ to
ansible/release.py. This is because namespace packages only load one
__init__.py. If that is not the __init__.py with the author and
version info then those won't be available.
* In ziplaoder, move the version ito ANSIBLE_CONSTANTS.
* Change PluginLoader to properly construct the path to the plugins even
when namespace packages are present.
* extend zypper integration tests
* fix caching issue of local RPMS on openSUSE 42.1 Leap
* add tests for simultaneous install/remove via prefixes +-
* test fail cases (rm patch or URL)
* test patch install (succes, unchanged second run, fail on wrong name)
* add test for pattern install
* include options refresh and priority introduced in ansible/ansible-modules-extras#1990
* test repo update both based on name and URL
* run on all suse flavors
* test overwriting repos by url or name
Updated python module wrapper explode method to drop 'args' file next to module.
Both execute() and excommunicate() debug methods now pass the module args via file to enable debuggers that are picky about stdin.
Updated unit tests to use a context manager for masking/restoring default streams and argv.
* add integration test for apache2_module
* fix assert syntax
* check os_family for this test
* add comment why only debian and suse are tested
* move apache2 test to destructive
* install apache2 package
- get_real_file will decrypt vault encrypted files and return a path to
a temporary file.
- cleanup_real_file will remove a temporary file created previously with
get_real_file
Ansible when there was a percentage that was calculated to be less than
1.0 would run all hosts as the value for a rolling update.
The error is due to the fact that Python will round a
float that is under 1.0 to 0, which will trigger the case of
0 hosts. The 0 host case tells ansible to run all hosts.
The fix will see if the percentage calculation after int
conversion is 0 and will else to 1 host.
This makes our recursive, ast.parse performance measures as fast as
pre-ziploader baseline.
Since this unittest isn't testing that the returned module data is
correct we don't need to worry about os.rename not having any module
data. Should devise a separate test for the module and caching code
* Ziploader proof of concept (jimi-c)
* Cleanups to proof of concept ziploader branch:
* python3 compatible base64 encoding
* zipfile compression (still need to enable toggling this off for
systems without zlib support in python)
* Allow non-wildcard imports (still need to make this recusrsive so that
we can have module_utils code that imports other module_utils code.)
* Better tracebacks: module filename is kept and module_utils directory
is kept so that tracebacks show the real filenames that the errors
appear in.
* Make sure we import modules that are used into the module_utils files that they are used in.
* Set ansible version in a more pythonic way for ziploader than we were doing in module replacer
* Make it possible to set the module compression as an inventory var
This may be necessary on systems where python has been compiled without
zlib compression.
* Refactoring of module_common code:
* module replacer only replaces values that make sense for that type of
file (example: don't attempt to replace python imports if we're in
a powershell module).
* Implement configurable shebang support for ziploader wrapper
* Implement client-side constants (for SELINUX_SPECIAL_FS and SYSLOG)
via environment variable.
* Remove strip_comments param as we're never going to use it (ruins line
numbering)
* Don't repeat ourselves about detecting REPLACER
* Add an easy way to debug
* Port test-module to the ziploader-aware modify_module()
* strip comments and blank lines from the wrapper so we send less over the wire.
* Comments cleanup
* Remember to output write the module line itself in powershell modules
* for line in lines strips the newlines so we have to add them back in
Before this patch, if config was ['/some/path'] then it would enter the
else block and config would be set to [].
The regression this patch fixes was introduced by 700db154.
* Unit tests exposed a problem where nested blocks did not correctly
hit rescue/always portions of parent blocks
* Cleaned up logic in PlayIterator
* Unfortunately fixing the above exposed a potential problem in the
block integration tests, where a failure in an "always" section may
always lead to a failed state and the termination of execution
beyond that point, so certain parts of the block integration test
were disabled.
* fetch_url shouldn't both accept follow_redirects and support follow_redircts via module.params
* Default follow_redirects for open_url should be 'urllib2'
* Add redirect test for get_url
- now workers passes queue to task_executor so it can send back events per item and on retry attempt
- updated result class to pass along events to strategy
- base strategy updated to forward new events to callback
- callbacks now remove 'items' on final result but process them directly when invoked per item
- new callback method to deal with retry attempt messages (also now obeys nolog)
- updated tests to match new signature of task_executor
fixes#14558fixes#14072
The dep chain for roles created during the compile step had bugs, in
which the dep chain was overwriten and the original tasks in the role
were not assigned a dep chain. This lead to problems in determining
whether roles had already run when in a "diamond" structure, and in
some cases roles were not correctly getting variables from parents.
Fixes#14046
this was taken out in an effort to default to the user's shell but creates issues as this is not known ahead of time
and its painful to set executable and shell_type for all servers, it should only be needed for those that restrict the user
to specific shells and when /bin/sh is not available. raw and command may still bypass this by explicitly passing None.
fixes#13882
still conditional
commit 24efa310b58c431b4d888a6315d1285da918f670
Author: James Cammarata <jimi@sngx.net>
Date: Tue Dec 29 11:23:52 2015 -0500
Adding an additional test for copy exclusion
Adds a negative test for the situation when an exclusion doesn't
exist in the target to be copied.
commit 643ba054877cf042177d65e6e2958178bdd2fe88
Merge: e6ee59f 66a8f7e
Author: James Cammarata <jimi@sngx.net>
Date: Tue Dec 29 10:59:18 2015 -0500
Merge branch 'speedup' of https://github.com/chrismeyersfsu/ansible into chrismeyersfsu-speedup
commit 66a8f7e873ca90f7848e47b04d9b62aed23a45df
Author: Chris Meyers <chris.meyers.fsu@gmail.com>
Date: Mon Dec 28 09:47:00 2015 -0500
better api and tests added
* _copy_results = deepcopy for better performance
* _copy_results_exclude to deepcopy but exclude certain fields. Pop
fields that do not need to be deep copied. Re-assign popped fields
after deep copy so we don't modify the original, to be copied, object.
* _copy_results_exclude unit tests
commit 93490960ff4e75f38a7cc6f6d49f10f949f1a7da
Author: Chris Meyers <chris.meyers.fsu@gmail.com>
Date: Fri Dec 25 23:17:26 2015 -0600
remove uneeded deepcopy fields
Also adds a new var, used by the prepare_tests role, to prevent it from
deleting the temp test directory at the start of each play to avoid any
potential race conditions
* Changed parse_addresses to throw exceptions instead of passing None
* Switched callers to trap and pass through the original values.
* Added very verbose notice
* Look at deprecating this and possibly validate at plugin instead
fixes#13608
For the deps setup of integration tests, as we sometimes see odd
errors we can't reproduce, which may be related to slightly out of
date package dependencies.
