* Update vmware.py
Incorrect chain of attributes to obtain snapshots from virtual machine. Module 'vmware_guest_facts' uses method 'gather_vm_facts' of vmware.py and always gets empty list of snapshots.
* Added test for module vmware_guest_facts. Checked VM snapshots.
* Fixed tests for vmware_guest_facts. Ignored certificate for vmware_guest_snapshot module.
* fixes asa action plugin for connection=local
This change fixes asa modules when using connection=local to load the
provider values.
* fix up pep8 issues
* Remove uses of assert in production code
* Fix assertion
* Add code smell test for assertions, currently limited to lib/ansible
* Fix assertion
* Add docs for no-assert
* Remove new assert from enos
* Fix assert in module_utils.connection
* nxos_inteface TypeError fix
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
* Make sure that run_commands does not list of strings for json output
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
* return default value to handle exception
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
* When splitting, use rsplit and 1 just in case the string has more pipe
characters than anticipated
* When converting to text, make decoding errors an error instead of
silently corrupting data
* When converting to text for message strings, use the default error
handler as that is already surrogate_then_replace
* implements jsonrpc message passing for ansible-connection
* implements more generic mechanism for persistent connections
* starts persistent connection in task_executor if enabled and supported
* supports using network_cli as top level connection plugin
* enhances logging for persistent connection to stdout
* Update action plugins
* Fix Python3 RPC
* Fix Junos bytes<-->str issues
* supports using netconf as top level connection plugin
* Error message when running netconf on an unsupported platform
* Update tests
* Fix `authorize: yes` for `connection: local`
* Handle potentially JSON data in terminal
* Add clarifying detail if possible on ConnectionError
Platform agnostic action plugin (net_base) calls
`get_provider_argspec()` to fetch the provider specific
details for each platform. This fix adds the function in
eos module_utils and retuns a dict of provider spec.
This fix adds additional error handling for vmware connect
method, where username provided user does not have required
permissions to use/login ESXi.
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Squashing all commits to one as suggested by John
* Adding Unit test method for the module enos_facts.py
* Pep8 and Ylint issues addressed
* Trying again to remove blank line. Some scripts are required for this.
* Bug Fixing for interfaces
* Editing for over indenting issue
* E203 whitespace before ','
* Update enos.py
Added warnings argument as to check_args method
* Update enos_facts.py
Added warnings to check_args method
* Move profile and region checking to module_utils.ec2
Remove ProfileNotFound checking from individual modules
There are plenty of `if not region:` checks that could be removed,
once more thorough testing of this change has occured
The ec2_asg, iam_managed_policy and ec2_vpc_subnet_facts modules
would also benefit from this change but as they do not have tests
and are marked stableinterface, they do not get this change.
eapi transport was not passing the auth_pass to the remote device with
it was provided. this fix will now insert the correct command hash into
the jsonrpc request.
fixes#30802
This patch changes the base parameters class to segment out the
update method. This is done often throughout the f5 modules, so
this just bakes it into the mod utils so that I can delete it
from each module
The platform/distro/etc facts were being passed in
correctly, but service_mgr.py was looking up the
wrong names ('system' vs 'ansible_system') resulting
in service_mgr falling back to default 'service' result.
Fixes#30753, #31095
We do not go through the effort of finding the right PROTOCOL setting if
we have SSLContext in the stdlib. So we do not want to hit the code
that uses PROTOCOL to set the urllib3-provided ssl context when
SSLContext is available. Also, the urllib3 implementation appears to
have a bug in some recent versions. Preferring the stdlib version will
work around that for those with Python-2.7.9+ as well.
Fixes#26235Fixes#25402Fixes#31998
* Move compare_policies and hashable_policy functions into module_utils/ec2
* Use compare_policies which is compatible with python 2 and 3.
* rename function to indicate internal use
* s3_bucket: don't set changed to false if it has had the chance to be changed to true already.
This code originated in module_utils/basic.py which was BSD licensed.
In moving it and making it aplicable to other pieces of code that were
using similar functions, I added onto it a little.
* Adding a cli transport option for the bigip_command module.
* Fixing keyerror when using other f5 modules. Adding version_added for new option in bigip_command.
* Removing local connection check because the F5 tasks can be delegated to any host that has the libraries for REST.
* Using the network_common load_provider.
* Adding unit test to cover cli transport and updating previous unit test to ensure cli was not called.
* Fix rollback in junos_config
Fixes#30778
* Call `load_configuration` with rollback id in case
the id is given as input
* Pass rollback id to `get_diff()` to fetch diff from device
* Fix unit test
* Make ansible_selinux facts a consistent type
Rather than returning a bool if the Python library is missing, return a dict with one key containing a message explaining there is no way to tell the status of SELinux on the system becasue the Python library is not present.
* Fix unit test
* Add new lines to end of config file lines
* Properly write out selinux config file
Change module behavior to not always report a change but warn if a reboot is needed and return reboot_required.
Improve the output messages.
Add strip parameter to get_file_lines utility to help with parsing the selinux config file.
* Add return documentation
* Add integration tests for selinux module
* Use consistent capitalization for SELinux
* Use atomic_move in selinux module
* Don't copy the config file initially
There's no need to make a copy just for reading.
* Put message after set_config_policy in case the change fails
* Add aliases to selinux tests
The /etc/os-release based distro detection doesn't
seem to work for Ubuntu 10.04 (no /etc/os-release?).
So it was testing the next case which was /etc/lsb-release to
see if it is 'Mandriva'. Since the check for existence of
(/etc/lsb-release, Mandrive) was the first non-empty dist
file match, 'ansible_distribution' was being set to 'Mandriva'
expecting to be corrected by the data from the dist file content.
But since the dist file parsing for Mandriva didn't match for
Ubuntu 10.04 /etc/lsb-release _and_ there is no Debian specific
lsb-release check, 'ansible_distribution' stayed at 'Mandriva'
and the dist file checking loop keeps going and eventually off
the end of the list before finding a better match.
Adding a debian/ubuntu specific check for /etc/lsb-release after
the debian os-release sets the info correctly and stops further
checking of dist files.
Fixes#30693
'distribution' facts were being set after checking
the existence of the dist file, and then being set
again with more detail after they were succesfully parsed.
But if the dist file was not succesfully parsed and
matched the required names, the loop continues
without resetting the earlier set facts. This is
how 'Mandriva' would end up being the 'distribution'
file for unrelated cases (it would find /etc/lsb-release,
set distro to 'Mandriva', then fail to parse/match and
continue the loop. If no other checks worked, 'Mandriva'
would stick).
* parse_dist_file_NA should check 'name' not distro for NA
parse_distribution_file_NA was checking the incoming
'distribution' fact to be 'NA', but the fact itself can
be specific at that point ('KDE Neon', for ex) but the
check is really if the 'name' it was passed is NA.
* for matches on OS_RELEASE_ALIAS (ie, 'Archlinux') do
not continue if the dist file content doesn't match. Previously
it had to because of the 'Mandriva' bug mentioned above.
This is a more general fix for #30693 than #30723
Fixes #30693
Related to #30600
Fix adds a new module 'vmware_guest_powerstate' to manage
power states of virtual machine.
Fixes: #30371
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* module_utils.urls - Encode the proxy connect as binary
Under Python3 the sendall method expects binary not a string.
Prior to this change the below exception was being thrown;
Traceback (most recent call last):
File "/tmp/ansible_umxox7_x/ansible_modlib.zip/ansible/module_utils/urls.py", line 1044, in fetch_url
client_key=client_key, cookies=cookies)
File "/tmp/ansible_umxox7_x/ansible_modlib.zip/ansible/module_utils/urls.py", line 951, in open_url
r = urllib_request.urlopen(*urlopen_args)
File "/opt/blue-python/3.6/lib/python3.6/urllib/request.py", line 223, in urlopen
return opener.open(url, data, timeout)
File "/opt/blue-python/3.6/lib/python3.6/urllib/request.py", line 524, in open
req = meth(req)
File "/tmp/ansible_umxox7_x/ansible_modlib.zip/ansible/module_utils/urls.py", line 729, in http_request
s.sendall((self.CONNECT_COMMAND % (self.hostname, self.port)).decode())
AttributeError: 'str' object has no attribute 'decode'
Encoding the value is inline with the lines below (Proxy-Authorization etc) which are being sent as binary.
Code like this:
if cond1 and cond2:
pass
elif cond1:
pass
Has a hidden dependency on the order that the conditions are checked.
This makes them fragile and subject to breakage during refactors.
Rewrite the code like this:
if cond1:
if cond2:
pass
else:
pass
The nested structure makes the ordering explicit and less likely for
someone to break the code when they refactor.
* Fix fact failures cause by ordering of collectors
Some fact collectors need info collected by other facts.
(for ex, service_mgr needs to know 'ansible_system').
This info is passed to the Collector.collect method via
the 'collected_facts' info.
But, the order the fact collectors were running in is
not a set order, so collectors like service_mgr could
run before the PlatformFactCollect ('ansible_system', etc),
so the 'ansible_system' fact would not exist yet.
Depending on the collector and the deps, this can result
in incorrect behavior and wrong or missing facts.
To make the ordering of the collectors more consistent
and predictable, the code that builds that list is now
driven by the order of collectors in default_collectors.py,
and the rest of the code tries to preserve it.
* Flip the loops when building collector names
iterate over the ordered default_collectors list
selecting them for the final list in order instead
of driving it from the unordered collector_names set.
This lets the list returned by select_collector_classes
to stay in the same order as default_collectors.collectors
For collectors that have implicit deps on other fact collectors,
the default collectors can be ordered to include those early.
* default_collectors.py now uses a handful of sub lists of
collectors that can be ordered in default_collectors.collectors.
fixes#30753fixes#30623
* Save the serialized values instead of their types
* Add tests for creating and modifying VMs without using a template
* Remove blank line
* Add tests for vm deletion
* Fix 'distribution' fact for ArchLinux
Allow empty wasn't breaking out of the process_dist_files
loop, so a empty /etc/arch-release would continue searching
and eventually try /etc/os-release. The os-release parsing
works, but the distro name there is 'Arch Linux' which does
not match the 2.3 behavior of 'Archlinux'
Add a OS_RELEASE_ALIAS map for the cases where we need to get
the distro name from os-release but use an alias.
We can't include 'Archlinux' in SEARCH_STRING because a name match on its keys
but without a match on the content causes a fallback to using the first
whitespace seperated item from the file content as the name.
For os-release, that is in form 'NAME=Arch Linux'
With os-release returning the right name, this also supports the
case where there is no /etc/arch-release, but there is a /etc/os-release
Fixes#30600
* pep8 and comment cleanup
On sparc64, /proc/cpuinfo has no usual 'model name', 'Processor', 'vendor_id', 'Vendor',
as a result "ansible_processor_vcpus" is always 1.
Add check element "ncpus active" to fix the issue.
* Fix pkg_mgr fact on OpenBSD
Add a OpenBSDPkgMgrFactCollector that hardcodes pkg_mgr
to 'openbsd_pkg'. The ansible collector will choose the
OpenBSD collector if the system is OpenBSD and the 'Generic'
one otherwise.
This removes PkgMgrFactCollectors depenency on the
'system' fact being in collected_facts, which also
avoids ordering issues (if the pkg mgr fact is collected
before the system fact...)
Fixes#30623
* Feature to Specify AZURE blob storage type
* Feature to Specify AZURE blob storage type
* Feature to Specify AZURE blob storage type
* Revert "Feature to Specify AZURE blob storage type"
This reverts commit 1d33997769ef3763a2eb434404c918134761635f.
modified: lib/ansible/module_utils/azure_rm_common.py
* Feature to Specify AZURE blob storage type
* Fix nxos provider transport warning issue
* Add default value of transport arg in provider spec
* Remove default value if transport arg in top level spec
This ensure deprecation warning is seen only in case transport
is given as a top level arg in task
* Refactor nxos modules to reference transport value from provider
spec
* Fix unit test
* Remove transport arg assignment in nxos action plugin
* As assigning transport value is handled in provider spec
top level task arg assignment is no longer required
* windows: fix list type in legacy module utils
* only change the return for the list type instead of affecting it all
* additional null check when using an array
* changed RunCommand result from Tuple to CommandResult for easier future extensibility
* moved Win32 Dictionary->multi-null-string environment munging into C#
As-merged, had several issues that prevented idempotent usage. Some args were defined at the wrong UI level. Dual-state args didn't match up with typical Ansible UI.
keyUsage and extendedKeyUsage are currently statically limited via a
static dict defined in modules_utils/crypto.py. If one specify a value
that isn't in there, idempotency won't work.
Instead of having static dict, we uses keyUsage and extendedKyeUsage
values OpenSSL NID and compare those rather than comparing strings.
Fixes: https://github.com/ansible/ansible/issues/30316
* fixes#26623
* Test-Path (and thus `-type path` in Get-AnsibleParam) fail on a nonexistent drive letter, since it can't be mapped to a PSProvider.
* added support and basic smoke tests for
* Fix unwanted deprecation message in network module argspec
Fixes#25663Fixes#24537
* segregate provider spec and top level arg spec
* add deprecation key in top level arg spec
* remove action plugin code to load provider and add
that logic at a common place in network_common.py file
* Fix CI issue
* Minor change
* Fix digital_ocean module_util api_token bug
* Included environment variables also
* Removed try/catch and added a check on self.oauth_token
Modules using the DigitalOceanHelper would expect the module to handle any api key resolution.
* Add EOS provider options as subspec
* Add IOS provider options as subspec
* Add IOS XR provider options as subspec
* Add Junos provider options as subspec
* Add NX-OS provider options as subspec
* Add Vyos provider options as subspec
* Remove password checks from check_args
* Do the same to aireos, aruba, ce, dellos*, & sros, as they work the same way
* VyOS does not support `transport`
* Fix junos_user pruge option failures
Fixes#25989
Add seperate handling for purge option which
fetches configured users on remote device
and delete the one not present in aggregate
list.
* Minor changes
CloudLinux and OpenVZ have common roots, but CloudLinux does not really
provide OS virtualization so it should not be regarded as a 'openvz'
system. This change adds a check for the existance of the LVE kernel
module which only exists on CloudLinux.
Fixes: https://github.com/ansible/ansible/issues/26424
* Add NetworkSetFactsModule for retrieving HPE OneView Network Sets
- Allow querying for Network Set resources in HPE OneView
- Adds unit tests to new module
- Updates oneview_module_loader copyright header to short GPL3 version
* Adding possibility to pass in credentials as parameters
* Removed required false and changed format of filter_by_name declaration
* Updated examples in docs to reflect new way to pass in credentials
- All examples of the oneview_network_set_facts updated to use
credential parameters
- All required=False from oneview base module removed
- Shared docs updated to bring attention to API version being used
* New Model manageiq manageiq_provider
* update docs, port is not required, region is provider-region
* add example of using token
* loop on endpoints instead of creating them one by one
* add alerts endpoint
* Simplify boilerplate and reorganize docs
Tried to make it clearer how the suboptions are laid out.
* Flatten out suboptions in order to make tests pass
These will not render properly in the HTML docs. Simplifying how this module accepts options should be addressed in a future PR.
* add template for az func
* (wip) add basic azure functions support
* add support to add app settings to azure function
* add support for updating based off of app settings
* add integration tests and refactor required param
* support check mode and add facts module
* add test for azure functions facts module
* add necessary checks and registrations for web client
* fix documentation
* change return type from complex to dict
* disable azure_rm_functionapp tests until stable
* remove dict comprehension for py2.6
* pepe has whitespace tumor
* Adding acs module
* linter issue
* Reduce the VM Size for the int. tests
* Short the name of the cluster
* Fixing the asserts and title
* Fixing VM Scale count in int. tests
* Changing the location of the tests
* trying eastus2
* disabling acs CI tests until stabilized
* Can be set via env, credential profile, or module arg
* Valid values defined by Azure Python SDK, currently `AzureCloud`,`AzureChinaCloud`,`AzureUSGovernment`,`AzureGermanCloud` or any Azure Stack metadata discovery URL.
Fix adds refactor of code for find_host_by_cluster_datacenter
API in vmware_host and vmware_resource_pool.
Also, adds strict check for cluster and datacenter object find.
Testcase for vmware_resource_pool.
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
If a PV hasn't been added to a VG i.e.:
[pmcclory@box ~]$ sudo pvs --noheadings --nosuffix --units g
/dev/xvdb lvm2 ---- 10.00 10.00
/dev/xvdv1 vg0 lvm2 a--u 24.99 0
Than ansible_facts.ansible_lvm will be unset after running setup module.
The issue is that the module splits on whitespace, which causes an
indexing error when the VG column is empty.
Fix is to add the separator field and safely split on that.
* Adding module to manage ethernet network on HPE OneView
* Adding unit tests to EthernetNetwork module
* Added OneViewModuleException custom exceptions to module
- Removed exception imports from hpOneView
- Updated unit tests
* Fixing mock import inside ethernet network module unit test
* Fixing issues found in METADATA by CI
* Updated paths to use solution name instead of vendor name
* Fixed documentation, removed redundant if and improved readability
* Updated _bulk_present to use and return `result`, same way as _present
* Changed __ to _ in private methods following ansible style
* Fixed some example inconsistencies and turned states doc into a list
* Added adriane-cardozo to list of maintainers
* amazon: extract boto_exception to ec2 module
This function was copy/pasted throughout several Amazon modules. This
causes a consistency problem, since some improvements to message
formatting were applied to some modules but not others. Now all modules
use the same, improved function.
* Rebase and make requested changes
* Rebase and make requested changes
* ovirt_templates: Update the argument spec of templates.
Add id of template since it is needed for register.
* ovirt_vms: Register unregistered VM.
Use register of VM with id instead of name since an
unregitered entity can be registered also without name attribute.
* ovirt_hosts: Add iscsidiscover to ovirt_hosts
Adding functionality of iscsidiscover to be used to discover iscsi
targets.
* ovirt_storage_domains: Add support for import block storage domain.
* Add functionality of partial import to unregistered VMs.
* Add functionality of partial import to unregistered Templates.
* ovirt_hosts: Add iscsilogin to ovirt hosts.
Add functionality of iscsi login to ovirt hosts to be used to connect to
iscsi targets and to be able to import iSCSI storage domain eventually.
* Add ovirt_storage_templates_facts
Adding fact module for storage templates.
The module should help with registering unregistered templates.
* Add ovirt_storage_vms_facts
Adding fact module for storage VMs.
The module should help with registering unregistered VMs.
* Add module_utils/aws/direct_connect.py for frequently used functions
* new AWS Direct Connect link aggregation group module with tests and placebo recordings
* remove extra argument
* Remove use of undefined var
* Fix param name for extra exception codes for AWSRetry to use.
* Fix undefined var and line length and metadata version number
* Fix copyright headers
* Added the ability to extend the exception list in CloudRetry
* AWSRetry boto and boto compatible
* Updated tests to reflect boto/boto3
* Added boto to shippable requirements
* Have base_class and added_exceptions default to None in CloudRetry
AWSRetry - only retry on boto3 exceptions and remove boto requirement from tests
* Make requested changes.
* openssl_privatekey: Extend test coverage
Extend the coverage of the integration test for the module
openssl_privatekey.
New tests have been added:
* passphrase
* idempotence
* removal
Co-Authored-By: Pierre-Louis Bonicoli <pierre-louis.bonicoli@gmx.fr>
* openssl_publickey: Extend test coverage
Extend the coverage on the integration test for the module
openssl_publickey.
New tests have been added:
* OpenSSH format
* passphrase
* idempotence
* removal
vca module utility uses response object instead of
response.content which raises exception in while fail_json
call. Use content attribute from response object instead which
is exact description of HTTP Response error.
Fixes#25378
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Adapt azure_rm_resource_group to azure 2.0.0 + azure Cli support
* Fix exceptions in Azure ARM plugins
* update azure_rm_networkinterface documention to reflect required params
* change state param to not required for docs in azure_rm_subnet
* fix import to reflect azure==2.0.0 changes
* add aliases and fix docs for azure_rm_storageblob
* add resource_group_name alias to azure_rm_storageaccount_facts
* fix import bug due to change in azure==2.0.0
* fix args bug and enum modules issue
* update docs to reflect azure==2.0.0
* pin management clients to a specific api_version
* update docs to reflect the new azure-ansible-base python package
* add fallback for older api resource group listing
* rework azure dependencies installation
* refactor path joining to a cross-plat solution
This PR includes:
- Removal of get_exception (sadly)
- Avoid deprecating 'state' parameter with aci_rest
- Small fix for querying using aci_rest
Signed-off-by: @bcoca
* Return the request dictionary in the results
It's sometimes useful to have access to the request params in a k8s
style. The dictionary returned by the request_params call can be
serialized into YAML to produce a k8s like file.
* Add dry_run option to skip module execution
By having support for dry_run executions, it'll be possible to generate
YAML files from the results dictionary by using the data in the
`requests` key.
We use ansible+cloudformation pretty heavily in our CI environments and occasionally bump into throttling issues. By adding this error code to the retry system we should be able to achieve better stability.
* Fix for issue ansible/ansible#27715
* Also fixing mutually exclusive check
* Updating subspec checks
These changes take into account a spec with all features enabled and do
the following tests for subspecs:
1. Test proper specs
2. Test Alias
3. Test missing required param
4. Test mutually exclusive params
5. Test required if params
6. Test required one of params
7. Test required together params
8. Test required if params with a default value
9. Test basis subspec params
10. Test invalid subsec params
* adds new filter plugins for network use cases
* adds parse_cli filter
* adds parse_cli_textfsm filter
* adds Template class to network_common
* adds conditional function to network_common
* fix up PEP8 issues
* Add aggregate for junos modules and sub spec validation
* aggregate support of junos modules
* aggregate sub spec validation
* relevant changes to junos integration test
* junos module boilerplate changes
* Add new boilerplate for junos modules
* Fix CI issues
* ManageIQ: manageiq_user module, module utils and doc_fragment
ManageIQ is an open source management platform for Hybrid IT.
This change is adding:
- manageiq_user module, responsible for user management in ManageIQ
- manageiq utils
- manageiq doc_fragment
* Handle import error
* Use formatting options
* group parameter is required
* changed doesn't need to be an attribute
* resource dictionary should contain values which isn't None
* move from monitoring to remote-management
* Use ManageIQ nameing convention
* Do not set defauts in arguments
* Use idempotent state parameter instead of action
* Check import error in the manageiq util class
* Update the miq documentation
* rename the connection configuration from miq to manageiq_connection
* All messeges start with non cap, fix typos, add examples, rename vars
* more typos fixes
* Make sure we insert only strings to logs by using % formating
* use suboptions keyword for the manageiq connection
* do not log the managiq connection struct (it include sensitive information like username and password)
* add missing from __future__
* ahh, wrong no-log line
* Use sub options
* openssl_csr: make subjectAltNames a list
* csr module now uses the new standard way to build openssl crypto modules
* add check functions for subject and subjectAltNames
* added support for keyUsage and extendedKeyUsage
* check if CSR signature is correct (aka the privatekey belongs to the CSR)
* fixes for first PR review
* fixes for second PR review
* openssl_csr: there is no need to pass on privatekey as it can be accessed directly
* openssl_csr: documentation fixes
* Initial commit for integration of HPE OneView resources with Ansible Core. Adding FC Network and FC Network Fact modules and unit tests, and OneView base class for all OV resources.
previously gather_subset=['!all'] would still gather the
min set of facts, and there was no way to collect no facts.
The 'min' specifier in gather_subset is equilivent to
exclude the minimal_gather_subset facts as well.
gather_subset=['!all', '!min'] will collect no facts
This also lets explicitly added gather_subsets override excludes.
gather_subset=['pkg_mgr', '!all', '!min'] will collect only the pkg_mgr
fact.
Create preserved_copy function in basic.py to perserve file ownership.
* Add a test for template preserved backup
* Use a script to get the random names
* bytes to strings
* Remove dump of hostvars
* Stop being fancy and create a testuser instead
* Fix pep8
* set file attributes
* Pass the correct data to set_attributes_if_different
* Use -j instead -b and pass the attributes as a string instead of a list
* remove debugging message
* Use shell to softly set the attr
Fixes#24408
* Add command_timeout timer that defines the amount
of time to wait for a command or RPC call before
timing out.
* Remove connect_retries and connect_interval configuration
varaible and replace it with connect_retry_timeout to control
the timeout value of connection to local scoket.
* Make required changes to netowrk action plugins and relevant
network files in module_utils.
* Required documentation changes.
* Add 2.0-2.3 facts api compat (ansible_facts(), get_all_facts())
These are intended to provide compatibilty for modules that
use 'ansible.module_utils.facts.ansible_facts' and
'ansible.module_utils.facts.get_all_facts' from 2.0-2.3 facts
API.
Fixes#25686
Some related changes/fixes needed to provide the compat api:
* rm ansible.constants import from module_utils.facts.compat
Just use a hard coded default for gather_subset/gather_timeout
instead of trying to load it from non existent config if the
module params dont include it.
* include 'external' collectors in compat ansible_facts()
* Add facter/ohai back to the valid collector classes
facter/ohai had gotten removed from the default_collectors
class used as the default list for all_collector_classes by
setup.py and compat.py
That made gather_subset['facter'] fail.
* Add aggregate parameter validation
aggregate parameter validation will support checking each individual dict
to resolve conditions for aliases, no_log, mutually_exclusive,
required, type check, values, required_together, required_one_of
and required_if conditions in argspec. It will also set default values.
eg:
tasks:
- name: Configure interface attribute with aggregate
net_interface:
aggregate:
- {name: ge-0/0/1, description: test-interface-1, duplex: full, state: present}
- {name: ge-0/0/2, description: test-interface-2, active: False}
register: response
purge: Yes
Usage:
```
from ansible.module_utils.network_common import AggregateCollection
transform = AggregateCollection(module)
param = transform(module.params.get('aggregate'))
```
Aggregate allows supports for `purge` parameter, it will instruct the module
to remove resources from remote device that hasn’t been explicitly
defined in aggregate. This is not supported by with_* iterators
Also, it improves performace as compared to with_* iterator for network device
that has seperate candidate and running datastore.
For with_* iteration the sequence of operartion is
load-config-1 (candidate db) -> commit (running db) -> load_config-2
(candidate db) -> commit (running db) ...
With aggregate the sequence of operation is
load-config-1 (candidate db) -> load-config-2 (candidate db) -> commit
(running db)
As commit is executed only once per task for aggregate it has
huge perfomance benefit for large configurations.
* Fix CI issues
* Fix review comments
* Add support for options validation for aliases, no_log,
mutually_exclusive, required, type check, value check,
required_together, required_one_of and required_if
conditions in sub-argspec.
* Add unit test for options in argspec.
* Reverted aggregate implementaion.
* Minor change
* Add multi-level argspec support
* Multi-level argspec support with module's top most
conditionals options.
* Fix unit test failure
* Add parent context in errors for sub options
* Resolve merge conflict
* Fix CI issue
* Make camel_to_snake work on capitalized plurals
`TargetGroupARNs` should become `target_group_arns`, not
`target_group_ar_ns`
Promote `camel_to_snake` to top layer function but prefix
it with an underscore.
Add tests for improved `_camel_to_snake` function.
Reduce use of `re.compile` as it makes no sense when the
compilation result is not reused.
* Remove unused LooseVersion check
* Fix PLURALs case for camel_to_snake
Also renamed EXPECTED_CAMELIZATION to EXPECTED_SNAKIFICATION
* ACI module_utils library for ACI modules
This PR includes:
- the ACI argument_spec
- an aci_login function
- an experimental aci_request function
- an aci_response function
- included the ACI team
* New prototype using ACIModule
This PR includes:
- A new ACIModule object with various useful methods
Made the following changes:
* Removed wildcard imports
* Replaced long form of GPL header with short form
* Removed get_exception usage
* Added from __future__ boilerplate
* Adjust division operator to // where necessary
For the following files:
* web_infrastructure modules
* system modules
* linode, lxc, lxd, atomic, cloudscale, dimensiondata, ovh, packet,
profitbricks, pubnub, smartos, softlayer, univention modules
* compat dirs (disabled as its used intentionally)
* Fix fetch configuration in junos_command
Fixes#26358 Add support to fetch configuration
from device in `xml`, `text`, `json`, `set` display
format.
* Add error message if set display is not supported
* Throw error in case of warning for show commands
* Module argument_spec now accepts a callable for the type argument, which is passed through and called with the value when appropriate. On validation/conversion failure, the name of the callable (or its type as a fallback) is used in the error message.
* adds basic smoke tests for custom callable validator functionality
* Enable the pylint no-name-in-module check. Checks that identifiers in
imports actually exist. When we do this, we also have to ignore
_MovedItems used in our bundled six. This means pylint won't check
for bad imports below ansible.module_utils.six.moves but that's
something that pylint punts on with a system copy of six so this is
still an improvement.
* Remove automatic use of system six. The exec in the six code which
tried to use a system library if available destroyed pylint's ability
to check for imports of identifiers which did not exist (the
no-name-in-module check). That test is important enough that we
should sacrifice the bundling detection in favour of the test.
Distributions that want to unbundle six can replace the bundled six in
ansible/module_utils/six/__init__.py to unbundle. however, be aware
that six is tricky to unbundle. They may want to base their efforts
off the code we were using:
2fff690caa/lib/ansible/module_utils/six/__init__.py
* Update tests for new location of bundled six Several code-smell tests
whitelist the bundled six library. Update the path to the library so
that they work.
* Also check for basestring in modules as the enabled pylint tests will
also point out basestring usage for us.
* Fix undefined vars on python3 and a whole bunch of other cleanup.
References #27193
* No need to catch exception and reraise. This just obfuscates the traceback
* Build up a list and then join at the end instead of building up a string. list.append() is faster than string concatenation
* No need to extract k, v pairs from one dict to make a second dict and then extract k, v pairs from the second dict. Iterate over the k, v pairs extracted from the first dict directly instead of building the second dict.
* No need to check if the dict is empty before iterating on it. Iterating on an empty dict will automatically go to the end of the loop
* Use isinstance instead of type(obj) is class, handles inheritance and is better style
* use to_native instead of v.encode(). We can use the surrogate_or_strict error handler to deal with more potential tracebacks. Does the right conversion on both Py2 and Py3.
* Convert bool to string before combining it with the string we're building.
* Don't reference unicode directly as unicode does not exist in Python3
* The string resulting from this function will not have a trailing comma
* Simplify the conversion to string int and bool values are now used in string formatting which will use str to transform them without an explicit invocation.
This patch adds cookie parsing to the fetch_url/open_url module_utils
method. The overall result will still contain the key `set_cookie`, however
an additional key (`cookies`) will also be present. This new field is a
dictionary of values. Overall, this should make looking for individual
cookies in the response much easier, as currently the `set_cookie` field
is an amalgamation of the returned set-cookie headers and can be somewhat
difficult to parse.
The OpenSSLObject class has been merged[1]. This commit makes the
openssl_privatekey rely on this class and standardize the way openssl
module should be written.
Co-Authored-By: Christian Pointner <cpointner@mgit.at>
[1] https://github.com/ansible/ansible/pull/26945
Crypto namespace contains the openssl modules. It has no integration
testing as of now.
This commits aims to add integration tests for the crypto namespace.
This will make it easier to spot breaking changes in the future.
This tests currently apply to:
* openssl_privatekey
* openssl_publickey
* openssl_csr
Fixing undefined vars across the codebase so that we can have pylint
catch them on every commit.
Some of the changes to this module_utils file are Python3 related => The
identifiers exist on python2 but not on Python3. The changes should be
portable to both py2 (2.6+) and py3.
References #27193
* Mark _symbolic_mode_to_octal and helper functions as classmethod and staticmethod
These helpers should be made toplevel functions in their own module.
For now, make them staticmethod/classmethod so that they can be used
(and tested) without instantiating an AnsibleModule.
* Move regex compilation out of loops
* Get rid of python-2.4 compat
* surrogate_then_strict doesn't exist. Switch to surrogate_or_strict
instead.
* Found some bugs in the _text.py implementation
* The composed error handlers (error handlers which are made up of two
or more python encoding error handlers) had a wrong string in it,
'surrogate_or_escape' doesn't exist. Replaced that with
'surrogate_or_replace' which is the correct handler name.
* Left comment about the implicit conditions that are part of the
surrogate_then_replace code path
Fixes#23865Fixes#23861
This fixes the symbolic notation of the chmod modes, as stated in the man page of chmod (in Linux). This also takes into account that chmod a+x is different from chmod +x. As the second one should take the current umask into account.
Fixes#14634
* Use atomic_move rather than writing in place in redhat_subscription and redhat utils
* Change status to curated
* Put curated in the proper field
* Add RETURN doc section
* Disable yum plugins when unregistering
* Change back to community supported
* Alphabetize imports
More openssl modules are about to be made, each of them rewriting
some pieces of code that can be refactored and used via a common
library.
This commit aims to create this "base" object and the common functions
one might want to reuse in order to avoid duplication.
unsafe_writes currently allows updating a file that can be updated but
not removed (for instance, when docker mounted). This change also
allows unsafe_writes to write to writable files in unwritable dirs. For
instance, if a system has made a single file inside of /etc/ writable to
a specific normal user.
Fixes#14961
The HTTP User-Agent "ansible-httpget" is already kind of the default,
it being the default value provided by the `url_argument_spec` helper
method. Yet, it may not be practical for all modules to get their
argument_spec that way.
Without a default User-Agent we fall back on the library
User-Agent. That being something like "Python-urllib/2.7".
While I'm no big fan of web servers making decisions based on the
provided User-Agent I still think that part of being a well-behaved
HTTP client is to provide an informative User-Agent. Not to mention
that it's a good thing for Ansible to behave consistently.
Indirectly fixes#26239
Fix IOS TypeError
* if flags are None, then ' '.join(flags) fails
* fixed get_defaults_flag so that it returns a list, and ignores lines with white space
Fixes#26918
* Revert change to docker_common as it's not as good as the try: except fix
* limit docker_volume fix to ImportErrors
* fix docker_secret i nthe same way
* Remove docker_secret from import tests
* Adding ciscowlc_command module and unit tests.
* Adding __init__.py for unit test.
* Fixing PEP8 W503.
* Renaming module from ciscowlc_command to aire_command.
* Renaming aire_command to aireos_command.
* Adding aruba_command module along with unit tests.
* Fixing PEP8 E303 too many blank lines.
* Adding default for timeout.
* Removing unused arguments. Moving default for timeout argument. Fixing cliconf to find hostname.
* Fixing PEP8 E302.
This fix corrects the usage of function FindByUuid by
specifying correct parameter 'uuid' and 'instanceUuid'
as documentation of VMWare's API.
Fixes: #24398, #24835, #25713
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
socket.create_connection is a higher-level function, which tries to
establish a socket connection using both AF_INET and AF_INET6. It got
introduced in Python 2.6, which ought to be fine with Ansible 2.4.
Fixes#26740
* Re-introduce relative paths to vmware_guest folder:
Move compile_folder_path_for_object function from vmware_guest_find to
utilities
Allow full path or relative path to be specified for the folder
parameter. We will build the full path to the new VM.
* Remove duplicate check
* PEP8 Fixes
* WIP update asa to use network_cli connection plugin
* add asa.py to cliconf plugins
* update asa.py terminal plugin to support regexp and events
* update constants to map asa modules to asa action handler
* update asa action handler to implement persistent connections
* update asa shared module to use persistent connections
* update asa_command module to use new connection
* fixed pep8 issues
Consolidate the module_utils, constants, and config functions that
convert values into booleans into a single function in module_utils.
Port code to use the module_utils.validate.convert_bool.boolean function
isntead of mk_boolean.
* junos implementation for net_l3_interface module
* junos_l3_interface implementation
* junos_l3_interface integration test
* net_l3_interface integration test for junos
* Fix module name typo
These integration tests were used for testing the exact behaviour of
Ansible for YAML-style syntax and key=value syntax.
This includes fixes to win_shortcut (as `src` can be a URL too)
When the file is opened with rwa+ and the update file size is smaller than the original the ini file can become corrupt. The issue was noticed when we had comments at the top of /etc/yum/pluginconf.d/rhnplugin.conf after using the rhn_register module the file became correct.
rwa+ also make no sense as the file is only written too and why would any appending need to happen?
* vyos implementation for net_interface module
* vyos_interface implementation module
* vyos_interface integration test
* net_interface integration test for vyos
* Change collection to aggregate
* Slight refactor on vmware_guest to fix path searching and vcsim compatibility.
* Clean up pep8 errors
* Fix more pep8 issues
* Add assertions
* Remove debug failure
* Update docsting for folder with examples
* Revise _get_vm_prop based on suggestions
* Implement folder walker to find a folder path
* More enhancements for datacenter->datacenter cloning
* Fix a few pep8 issues
* Remove useless check for subclass of None
* Fix a few suggestions
* Move serialize_spec to a util function
* Group exceptions together
Fixes#25011
* adds more intelligent save logic and diff to network config modules
* adds sha1 property to NetworkConfig
* adds new argument save_when to argument_spec
* adds new argument diff_against to argument_spec
* adds new argument intended_config to argument_spec
* renames config argument to running_config with alias to config
* deprecates the use of the save argument
* before and after now work with src argument
* misc module clean
Modules updated
* nxos_config
* ios_config
* eos_config
Most notably this makes the save mechanism more intelligent for config
modules for devices that need to copy the ephemeral config to
non-volatile storage.
The diff_against argument allows the playbook task to control what the
device's running-config is diff'ed against. By default it will return
the diff of the startup-config.
* removes ios_config from pep8/legacy_files.txt
* extends the ignore lines argument to the module
* clean up CI errors
* add missing list brackets
* fixes typo
* fixes unit test cases
* remove last line break when returning config contents
* encode config string to bytes before hashing
* fix typo
* addresses feedback in PR
* update unit test cases
Based on issue 23642, add some info about the used python
executable and version to the error message when ssl connection
fail in a way that may be related to the version.
It was in lib/ansible/modules/system/setup.py since it
was the only thing using it, but move it back to module_utils
and add a ansible_collector.get_ansible_collector() to build
a facts collector just like the one used by setup.py
mv test_setup.py -> test_ansible_collector.py
All the code it was testing is now in ansible_collector
rm code to create 'ansible_facts' subkey from namespace
Just leave it up to the caller to do, and just return a
flat dictionary from AnsibleFactCollector.collect()
* junos_linkagg implementation and junos modules refactor
* junos_linkagg implementation
* junos_linkagg integration test
* net_linkagg integration test for junos
* decouple `load_config` and `commit` operations,
to allow single commit (in case on confirm commit) and
to perform batch commit (multiple `load_config` followed by single
`commit`)
* Other related refactor
* Fix CI issues
* Fix unit test failure
VMware provides a different DMI product name for VMs booted via UEFI vs BIOS.
VMware provides a different DMI product name for VMs booted via UEFI ('VMware7,1') vs BIOS ('VMware Virtual Platform')
Fixes#26517
During the writing of Windows path integration tests we discovered that
incorrect paths (including escape sequences) cause very cryptic error
messages.
This fix ensures that invalid paths cause a proper error message.
We also had to fix the following modules:
- win_shortcut: `src` can be a URL
restored 'rc' inspection but only when failed is not specified
removed redundant changed from basic.py as task_executor already adds
removed redundant filters, they are tests
added aliases to tests removed from filters
fixed test to new rc handling
There are too many possible special cases for Ansible to be able to
precheck known_hosts files without introducing all kinds of false
failures.
* Alternative known_hosts paths
* Alternative host name aliases
* ssh host certificates
* SSHFP + DNSSEC
Fixes#24860
* Add junos_system declartive module and other related change
* junos_system declartive module
* integration test for junos_system
* integration test for net_system (junos platform)
* pep8 fixes for junos modules
* move to lxml from elementree for xml parsing as it support
complete set of xpath api's
* other minor changes
* Fix CI and doc changes
* Fix unit test failures
* Fix typo in import
* Fix import issue for py2.6
* Add missed Element in import
This is a cleanup of the win_uri module to make it feature-complete.
This PR includes:
- Added check-mode support
- Add as many options from the uri module as possible
- Added creates
- Added follow_redirects
- Added maximum_redirection
- Added password
- Added removes
- Added return_content
- Added status_code
- Added timeout
- Added user
- Added validate_certs
- Fixed list-handling for comma-separated strings
- Added basic integration tests (should come from uri module)
refactors the Connection class to use the top level function. This will
make the request_builder() function useful for other components such as
action handlers.
* Add junos_banner declartive module
* junos_banner implementation
* Integration test for junos_banner
* Integration test for net_banner (junos)
* Minor fixes
* Minor doc change
The method name was missing a 'd'. The method was not used anywhere
however, so no other code needs to be changed. Neither 'has_chilren'
nor 'has_children' are used in the codebase.
* Add net_interface declartive module
* Add net_interface module
* Add junos_interface implementation module
* Other minor changes
* Add integration test
* Integration test for net_interface
* Integration test for junos_interface
* Fix CI failures
* Documentation changes
* adds new common functions for declarative intent modules
* adds Entity and EntityCollection
* adds dict_diff and dict_combine
* update for CI PEP8 compliance
* more CI PEP8 fixes
* more PEP8 CI clean up
* refactors the lambda assignments into top level classes
this is to be in compliant the PEP8 CI sanity checks
* one last pep8 ci fix
* Initial commit for Pure Storage Ansible module
* Initial commit for Pure Storage Ansible module
* Initial commit for Pure Storage Ansible module
* Fix import issues as required by post-2.2
* Move last import to top
* Follow suggestions and only implement one module per PR
Fix documentation changes requested
* Documentation and formatting changes
* Various fixes to VM customizations (from template)
This patch implements:
- New find_obj() function from vmware.py replacing get_obj()
- Implement proper resource_pool selection
- Fix productId implementation (was not working)
- Ensure that we are not changing anything that is not mandatory (hostName, orgName, fullName)
This is an alternative proposal to #24283
This does not fix#19860 yet though.
For our use-case, we do not want to customize the network information (or any information in fact).
What is used in the template should remain intact.
* Added find_obj() function
* Fix the returned object-list (unused yet)
* Small improvement
* Support DHCP type and fix customizations
* Small fix
* Support resource_pool also for reconfiguring VM
* Remove redundant
* Fix short hostname, specific resource_pool, PEP8
* Improve docs and examples
* Fix missing hostsystem
* Make folder absolute path
* Improve docs, add missing 'mac'
At present, the available facts around block devices are not sufficient to be able to find stable names guaranteed to work across reboots, or to identify block devices by label (UUID, etc).
This patch provides a list of observed links for each device. It relies on functionality specific to Linux (as does the existing sysfs-based code which it extends), but should not cause issues on other platforms.
Moreover, it prevents virtual devices from being excluded, and links such devices to the physical devices to which they are attached.
* Add more mount point statvfs info including sizes
Based on https://github.com/ansible/ansible/pull/12073
facts.utils.get_mount_size() now returns a dict of most
of the posix statvfs data, including block_size and inode
counts.
Update the facts.hardware classes that use get_mount_size() to
use the new info by mount_info.update(mount_statvfs_inof) to merge.
* add back unit tests for LinuxHardware mount/fs facts
* add test cases for facts.utils.get_mount_size
* Support NetBSD 7.1+ style ifconfig -a output
network facts on NetBSD after 7.1 cvs would fail
because of format changes in 'ifconfig -a' output.
update code to support new and old format.
add unit tests for both based on
examples from Bruce V Chiarelli.
* wrap use of interfaces.keys() in list() for py3 compat
* sort interface ids for stability
* Fix ansible_cmdline initrd fact for UEFI
UEFI cmdline paths use \ path sep which would
get munged by cmdline fact collection.
* Make CmdLineFactCollector easier to test
extract the parsing of the /proc/cmdline content to
_parse_proc_cmdline()
add a wrapper method for get_file_content _get_proc_cmdline()
Add unit tests of _parse_proc_cmdline based on examples
from issue #23647Fixes#23647
* netscaler_service initial implementation
* Changes as requested by reviewers
* Skip some tests if under python2.6 and importing requests library
* Change option "operation" to "state"
* Remove print statements from netscaler module utils
* Catch all exceptions during login
* Fix fail message
* Add common option save_config
* Fix boto3 tags to remove; unable to remove a list of strings, need list of {'Key': key}
* pep8 fix
* cannot bugfix due to differences in AWS APIs - just document
* add more info to guidelines for using compare_aws_tags
* Add net_user platform agnostic module
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
* Integration test for net_user
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
* modify eos_user module to support name param as alias to username
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
* Test collection of users
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
When operating on arbitrary return data from modules, it is possible to
hit the recursion limit when cleaning out no_log values from the data.
To fix this, we have to switch from recursion to iteration.
Unittest for remove_values recursion limit
Fixes#24560
* Ultra basic api-gateway module based of lambda.py
* Ultra basic deployment added to api-gateway module
* ApiGateway module Allow creation of APIs, more documentation and better return value
* ApiGateway module incorporate review feedback
* ApiGateway module flake8 cleanup
* APIGateway module - more review fixes.
* slightly better messages in api_gateway module
* AWS api_gateway module - try to improve messages in case of exceptions
* rename api_gateway module to aws_api_gateway as discussed in PR 20230
* aws_api_gateway - Allow delivery of swagger either as text or dictionary.
* aws_api_gateway module - introduce 'unit' tests, improve imports using them and small fixes
* aws_api_gateway module - move path expand_user to avoid early typecheck
* aws_api_gateway - version means version of metadata not module - fix to 1.0
* aws_api_gateway module - Rely on module_utils.ec2 for imports & path type for expanduser / cleanups
* aws_api_gateway module - heavy cleanup and refactor of code + cloud retry functionality.
* api_gateway_module - failing test case for handling more than one deployment in succession and API deletion
* add TooManyRequestsException to AWSRetry exception list - makes API deployment work.
* api_gateway_module - Fixes for various review comments + errors from various linters
* api_gateway_module - Fixes for more review comments + linter error
* api_gateway_module - Major refactor into sensible functions - create_response becomes configure_response
* api_gateway_module - should be working under python3; remove test exclusion
* api_gateway_module - finish off remaining review fixes - use ansible defaults and fix mutually exclusive
* api_gateway_module - attempt to improve handling of botocore errors in python3
* api_gateway_module - implement state=absent / API deletion
Fix 'hostname' module Facts is not defined by updating
'hostname' module to use it.
is_systemd_managed() was previously on the module_utils.facts.Facts
class that no longer exists.
Fixes#25289
* Get AWS access and secret keys from boto config
* matching the elif pattern
* avoid setting credentials as empty strings by checking os.environ.get('cred')
put comments back
* remove None from os.environ.get()
* Platform agnostic net_system module
Also refactor the action network plugins for better code re-use
Still more refactoring to do once the connection plugin work is complete
* Replace importlib for imp
importlib is not available on 2.6, so we need to stick to imp
* Load action plugin via module metadata
* Better error message if no implementation is found
Now the plugin will show the module name and the network OS in the
error message
* Fix typo on documentation author line
* Fix pep8 issues
* Add missing options key on doc string and stringify version
* Return None in case module has no metadata
* Read module metadata only if it's a python module
Check for module suffix, if it's .py then read metadata.
Otherwise this fails on non-python modules, like Windows PS for example.
* Read metadata variable only if it's a python module
Fix referencing a variable before assignment
* Add action_handler to validate_modules metadata schema
* Pull metadata with plugin_docs get_docstring
Using load_source from PluginLoader is troublesome, it is not guaranteed
a module may be importable at the controller, e.g. if a module depends
on module_utils functions it won't work, because module_utils is not
in the sys path.
Rather than putting that module dependencies introspection, just
use plain parsing like plugin_docs get_docstring does as we only care
about reading ANSIBLE_METADATA.
* Add platform agnostic group of groups for integration tests
This will be the target for platform agnostic integration tests.
* Add integration tests for net_system
* Switch to action plugin inheritance from metadata driven action handler
As the metadata action driven action handler work is being worked on
on its standalone proposal+PR, let's just go back to have one
action handler per platform agnostic module.
Those action plugins will inherit from net_base.
* Add blank line to fix pep8
* Add aliases file to net_system integration test
This will avoid CI failure
* Fix integration tests for net_system
* Give more precedence to task network_os over inventory network_os
* Add support to fetch old style junos facts
Fixes#25050
Add support to fetch old style facts supported in Ansible <= 2.2
To fetch old style facts value `gather_subset` value shoule be `all`
and `junos-eznc` is required to be installed on control node as a
prerequisite.
* Remove unwanted import
Facts Refresh (2.4 roadmap)
This commit implements most of the 2.4 roadmap 'Facts Refresh'
- move facts.py to facts/__init__.py
- move facts Distribution() to its own class
- add a facts/utils.py
- move get_file_content and get_uname_version to facts/utils.py
- move Facts() class from facts/__init__ to facts/facts.py
- mv get_file_lines to facts/utils.py
- mv Ohai()/Facter() class to facts/ohai.py and facter.py
- Start moving fact Hardware() classes to facts/hardware/*.py
- mv HPUX() hardware class to facts/hardware/hpux.py
- move SunOSHardware() fact class to facts/hardware/sunos.py
- move OpenBSDHardware() class to facts/hardware/openbsd.py
- mv FreeBsdHardware() and DragonFlyHardware() to facts/hardware/
- mv NetBSDHardware() to facts/hardware/netbsd.py
- mv Darwin() hardware class to facts/hardware/darwin.py
- pep8/etc cleanups on facts/hardware/*.py
- Mv network facts classes to facts/network/*.py
- mv Virtual fact classes to facts/virtual
- mv Hardware.get_sysctl to facts/sysctl.py:get_sysctl
- Also mv get_uname_version from facts/utils.py -> distribution.py
since distribution.py is the only thing using it.
- add collector.py with new BaseFactCollector
- add a subclass for AnsibleFactCollector
- hook up dict key munging FactNamespaces
- add some test cases for testing the names of facts
- mv timeout stuff to facts.timeout
- rm ansible_facts()/get_all_facts() etc
- Instead of calling facts.ansible_facts(), fact collection
api used by setup.py is now to create an AnsibleFactCollector()
and call it's collect method.
- replace Facts.get_user_facts with UserFactCollector
- add a 'systems' facts package, mv UserFactCollector there
- mv get_dns_facts to DnsFactCollector
- mv get_env_facts to EnvFactCollector
- include the timeout length in exception message
- modules and module_utils that use AnsibleFactCollector
can now theoretically set the 'valid_subsets'
May be useful for network facts module that currently have
to reimplement a good chunk of facts.py to get gather_subsets
to work.
- get_local_facts -> system/LocalFactCollector
- get_date_time -> system/date_time.py
- get_fips_facts -> system/fips.py
- get_caps_facts() -> system/caps.py
- get_apparmor_facts -> system/apparmor.py
- get_selinux_facts -> system/selinux.py
- get_lsb_facts -> system/lsb.py
- get_service_mgr_facts -> system/service_mgr.py
- Facts.is_systemd_managed -> system/service_mgr.py
- get_pkg_mgr_facts -> system/pkg_mgr.py
- Facts()._get_mount_size_facts() -> facts.utils.get_mount_size()
- add unit test for EnvFactCollector
- add a test case for minimal gather_subsets
- add test case for collect_ids
- Make gather_subset match existing behavior or '!all'
If 'gather_subset' is provided as '!all', the existing behavior
(in 2.2/2.3) is that means 'dont collect any facts except those
from the Facts() class'. So 'skip everything except
'apparmor', 'caps', 'date_time', 'env', 'fips', 'local', 'lsb',
'pkg_mgr', 'python', 'selinux', 'service_mgr', 'user', 'platform', etc.
The new facts setup was making '!all' mean no facts at all, since
it can add/exclude at a finer granularity. Since that makes more
sense for the ansible collector, and the set of minimal facts to
collect is really more up to setup.py to decide we do just that.
So if setup.py needs to always collect some gather_subset, even
on !all, setup.py needs to have the that subset added to the
list it passes as minimal_gather_subset.
This should fix some intg tests that assume '!all' means that
some facts are still collected (user info and env for example).
If we want to make setup.py collect a more minimal set, we can do that.
- force facts_dicts.keys() to a list so py3 works
- split fact collector tests to test_collectors.py
- convert Facter(Facts) -> other/facter.py:FacterFactCollector
- add FactCollector.collect_with_namespace()
regular .collect() will return a dict with the key names
using the base names ('ip_address', 'service_mgr' etc)
.collect_with_namespace() will return a dict where the key names
have been transformed with the collectors namespace, if there is
one. For most, this means a namespace that adds 'ansible_' to the
start of the key name.
For 'FacterFactCollector', the namespace transforms the key to
'facter_*'.
- add test cases for collect_with_namespace
- move all the concrete 'which facts does setup.py' stuff to setup.py
The caller of AnsibleFactCollector.from_gather_subset() needs to
pass in the list of collector classes now.
- update system/setup.py to import all of the fact classes and pass
in that list.
- split the Distribution fact class up a bit
extracted the 'distro release' file handling (ie, linux
boxes with /etc/release, /etc/os-release etc) into its
own class.
- extract get_cmdline_facts -> cmdline.py
- extract get_public_ssh_host_keys -> system/ssh_pub_keys.py
- extract get_platform_facts -> system/platform.py
platform.py may be a good candidate for further splitting.
- rm test for plain Facts() base class
- let the base class for Collector unit tests provide collected_facts
some Collectors and/or their migrated Facts() subsclasses need
to look at facts collected by other modules ('ansible_architecture'
the main one...).
Collector.collect() has the collected_facts arg for this, so add
a class variable to BaseFactsTest so we can specify it.
- mv Ohai to other/ohai.py and convert to Collector
- update hardware/*.py to return facts (no side effects)
- mv AnsibleFactCollector to setup.py
- extra collector class gathering to module method in
facts/__init__.py (collector_classes_from_gather_subset)
- add a CollectorMetaDataCollector collector used to provide
the 'gather_setup' fact
- add unit test module for 'setup' module
(test/units/modules/system/setup.py)
- Collector init now doesnt need a module, but collect does
An instance of a FactCollector() isnt tied to a AnsibleModule
instance, but the collect() method can be, so optionally pass
in module to FactCollector.collect() (everywhere)
- add a default_collectors for list of default collectors
import and use it from setup.py module
eventually, would like to replace this with a plugin loader
style class finder/loader
- unit tests for module_utils/facts/__init__.py
- add unit tests for ohai facts collector
- remove self.facts side effect on populate() in hardware/sunos.py
- convert OpenBSDHardware() to rm side effects on self.facts
- try to rm some self.facts side effects in Network()
plumb in collected_facts from populate() where it is needed.
stop passing collected_facts into Network() [via cached_facts=,
where it eventually becomes self.facts]
- nothing provides Fact() cached_facts arg now, rm it
Facts() should be internal only implementation so nothing
should be using it.
Of course, now someone will.
- add a Collector.name attr to build a map of name->_fact_ids
To properly exclude a gather_subset spec like '!hardware', we
need to know that 'hardware' also means 'devices', 'dmi', etc.
Before, '!hardware' would remove the 'hardware' collector name
but not 'devices'. Since both would end up in id_collector_map,
we would still end up with the HardwareCollector in the collector
list. End result being that '!hardware' wouldn't stop hardware
from being collected.
So we need to be able to build that map, so add the Collector.name
attribute that is the primary name (like 'hardware') and let
Collector._fact_ids be the other fact ids that a collector is
responsible for.
Construct the aliases_map of Collector.name -> set of _fact_ids
in fact/__init__.py get_collector_names, and use it when we are
populating the exclude set.
- refactor of distribution.py
make the big OS_FAMILY literal a little easier to read
Also keys can now be any string instead of python literals
99% sure the test for 'KDE Neon' was wrong
I don't see how/where it should or could get 'Neon' instead
of 'KDE Neon' as provided in os-release NAME=
Use 'distribution' string for key to OS_MAP
ie, we dont need to make it a valid python label anymore so dont.
move _has_dist_file to module as _file_exists
easier to mock without mucking with os.path
mv platform.system() calls to within get_distribution_facts() instead
of Distribution() init.
- remove _json compat module
The code in here was to support:
-a 'json' python module that was not the standard one included
with python since 2.6.
- potentially fallback to simplejson if 'json' was not available.
'json' is available for all supported python versions now so
no longer needed.
- mv get_collector_names -> facts.collector
- mv collector_classes_from_gather_subset -> facts.collector
- mv collector tests from test_facts -> test_collector
- Use six's reduce() in sunos/netbsd hardware facts
- rm extraneous get_uname_version in utils
only system/distribution.py uses it
- Remove Facts() subclass metaclass usage
- using fact_id and a platform id for matching collectors
gut most of Facts() subclasses
rm Facts() subclasses with weird metaclass
only add collectors that match the fact_ids and the platform_info
to the list of collectors used.
atm, a collectors platform_id will default to 'Generic', and
any platform matches 'Generic'
goal is to select collector classes including matching the
systems platform in collector.py, instead of relying on
metaclasses in hardware/*. To finish this, the various
Facts() subclasses will need to be replaced entirely with
Collector() subclasses.
use collector classmethod platform_match() to match the platform
This lets the particular class decide if it is compatible with
a given platform_info. platform_info is a dict like obj, so it could be
expanded in the future.
Add a default platform_match to BaseFactCollector that matches
platform_info['system'] == cls._platform
They were needed previously to trigger a module
load on all the collector classes when we import
facts/hardare so that the Hardware() and related
classes that used __new__ and find_all_subclasses()
would work.
Now that is done in collectors based on platform matching
at runtime we dont need to do it py module import/parse
time. So the non empty __init__.pys are no longer needed
and their is a more flexible mechanism for selection
platform specific stuff.
facts/facts.py is no longer used, rm'ed
- if we dont find an implement class for gather spec.. just ignore it.
Would be useful to add a warn to warn about this case.
- Fix SD-UX typo (should be HP-UX)
- Port fix for #21893 (0 sockets) to this branch
This readds the change from 8ad182059d
that got lost in merge/rebase
Fixes#21893
- port sunos fact locale fix for #24542 to this branch
based on e558ec19cdFixes#24542
Solaris fact fix (#24793)
ensure locale for solaris fact gathering
fixes issue with locale interfering with proper reading of decimals
- raise exceptions in the air like we just dont care.
Pretty much ignore any not exit exception in facts
collection. And add some test cases.
- added new selinux fact to clarify python lib
the selinux fact is boolean false when the library is not installed,
a dictionary/hash otherwise, but this is ambigous
added new fact so we can eventually remove the type dichtomy and normalize it as a dict
Re-add of devel commit 85c7a7b844 to
the new code layout, since it got removed in merge/rebase
* Import from urllib and not url
In Python 3, the correct way to import
urlparse is through urllib.parse.
* Use six module to import urlparse
Import urlparse from ansible.module_utils.six.
This way, the import statement is compatible both
with Python 3 and Python 2.
* Fix urlparse import using six module
The correct import is:
from six.moves.urllib.parse import urlparse
* Import six from ansible.compat
* Import from six.moves. Don't make six global.
Prevent code smell by not making six available
in the global namespace and instead importing from
one of its subpackages.
This is required for modules that may return a non-zero `rc` value for a
successful run, similar to #24865 for Windows fixing **win_chocolatey**.
We also disable the dependency on `rc` value only, even if `failed` was
set.
Adapted unit and integration tests to the new scheme.
Updated raw, shell, script, expect to take `rc` into account.
* Prevent IndexError when deleting multiple lines
The old code will raise `IndexError: list assignment index out of range` when deleting multiple lines because the indexes of the original and the copy get out of sync. Solved by deleting from the high indexes first so the lower ones remain stable.
* Don't load configuration if nothing to load
Instead of sending an empty candidate config (for example because the candidate only consisted of `delete` lines, and all of them were filtered out by `filter_delete_statements`) just return. JunOS seems to get confused by empty changes, and if the candidate config is empty then it's a no-op anyway.
the selinux fact is boolean false when the library is not installed,
a dictionary/hash otherwise, but this is ambigous
added new fact so we can eventually remove the type dichtomy and normalize it as a dict
* eos python3 changes
* changes to convert response from byte to text
* Add dellos6 python3 changes
Make `execute_command` arguments and its
return value complaint to PY3 changes
made in PR #24431
* Fix py3 prompt issue for invalid show command
* Fix review comments
* Add generic fix for error prompt in py3
* Fix CI issue
* Fix network_cli unit test failure
* Add nxos changes for Python3
Make `execute_command` arguments and its
return value complaint to PY3 changes
made in PR #24431
* Fix CI issues
* Fix review comment
Replace surrogate_or_strict with
surrogate_then_replace as per review
comment os PR #24601
Fix converts commit_timeout to string as
Elementree.SubElement requires text as string.
Fixes#24611
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Add ios changes for Python3
Make `execute_command` arguments and its
return value complaint to PY3 changes
made in PR #24431
pep8 fixes
* Fix CI issues
* Fix review comment
* Add sros changes for Python3
Make `execute_command` arguments and its
return value complaint to PY3 changes
made in PR #24431
Code cleanup
pep8 fixes
* Fix CI issue
* Add vyos changes for Python3
Make `execute_command` arguments and its
return value complaint to PY3 changes
made in PR #24431
pep8 fixes
* Fix CI issues
On Python3 and Python2 use pickle slightly differently so we need to be
explicit about some things.
If pickles could be shared between python2 and python3, as in
ansible-connection and the pickle cache, we need to specify the protocol
to use when dumping and the encoding to use for byte strings when
loading.
The dumping protocol needs to be no higher than 2 as python-2 only
supports up to protocol 2. The encoding should usually be 'bytes' so
that python2 str type becomes python3 bytes type. However, doing this
means that we must make sure that the objects being serialized properly
make their strings into text strings except when they're supposed to be
bytes. If strings are improperly byte strings, they may cause
tracebacks on the receiving end
* [GCP] UrlMap module
This module provides support for UrlMaps on Google Cloud Platform. UrlMaps allow users to segment requests by hostname and path and direct those requests to Backend Services.
UrlMaps are a powerful and necessary part of HTTP(S) Global Load Balancing on Google Cloud Platform.
UrlMap takes advantage of the python-api so the appropriate infrastructure has been added to module_utils.
More about UrlMaps can be found at:
https://cloud.google.com/compute/docs/load-balancing/http/url-map
UrlMap API:
https://cloud.google.com/compute/docs/reference/latest/
Google Cloud Platform HTTP(S) Cross-Region Load Balancer:
https://cloud.google.com/compute/docs/load-balancing/http/
* updated documentation, remmoved parens
* fixed tabs
* Add new helper function for comparing AWS tag key pair dicts. Also modify boto3_tag_list_to_ansible_dict function to be more generic when looking for key names because AWS sometimes uses 'Key', sometimes 'TagKey' and who knows what the future holds! Fixed modules to work with changes.
* Review changes
* Add some more doc to GUIDELINES for tags and fix var name for snaked values in ec2_group_facts
Simplified the logic, and fix the issue of having a nested list as
entry.
Also indicate that we deliberately want to retain $null values
(and i.e. do not want them to be turned into a boolean).
* WIP file_mode
* WIP
* Add file_mode + integration tests
* fix pep8
* Update doc fragments
Create mutualy_exclusive param
Fix yamllint problem in tests
* Add aliases file + main playbook for fortios
* Install pyfg before running tests
* Install pyfg before running tests in role
* Remove pre_task as it's done in roles
* Force pyFG minimal version for python3
* role_path not role_dir :(
* Change requirements
* Specify Error type when error on import
* Bug in pygf library with python 2.5 (PR is waiting https://github.com/spotify/pyfg/pull/19)
* Bad requirement format
* still bad format -_-'
* remove test/integration/fortios.py (auto generated by tests)
missing new lines at end of file
* pyFG is now fixed in 0.50
ssh-keyscan isn't very verbose about errors. Give the user whatever
information we have available even if it isn't much. At least they will
know how we were running ssh-keyscan and why there's an error now.
Fixes#19440
When unittesting, the framework creates a pipes module that is picked up
by the basic module_utils test. Switch to using shlex_quote as that is
the right thing to use for portability anyway.
* Added new common argument api_version for all Avi modules
* Updated dependency for the new modules to use avi sdk 17.1 and above
* Updated Avi 17.1.1 modules with descriptions and moddule dependencies
* Updated version_added for new options
The timeout for gathering facts needs to be settable from three places
(highest precedence to lowest):
* programmatically
* ansible.cfg (equivalent to the user specifying it explicitly when
calling setup)
* from the default value
The code was changed in b4bd6c80de to
allow programmatically and the default value to work correctly but
setting via ansible.cfg/parameter was broken.
This change should fix setting via ansible.cfg and adds unittests for
all three cases
Fixes#23753
ClearLinux has changed its pretty name in os-release file
from: 'Clear Linux Software for Intel Architecture'
to: 'Clear Linux OS for Intel Architecture'
This patch makes the SEARCH_STRING 'Clear Linux' rather than
the full name to make it compatible with the old and new name.
Signed-off-by: Alberto Murillo Silva <alberto.murillo.silva@intel.com>
* Make warning logs consistent
Arguments outside provider with default
value should not log as warning in case
it is not mentioned in play.
* Make nxos timeout default consistent and add comments
* Make comments more verbose
* Code cleanup
Removed 'add' method from CustomNetworkConfig. It is identical
to the one inherited from NetworkConfig
* Removed unused CustomNetworkConfig import
* Replaced
```
def get_existing(module, args):
existing = {}
netcfg = get_config(module)
config = netcfg.get_section(parents)
```
with
```
netcfg = CustomNetworkConfig(indent=2, contents=get_config(module))
```
get_config returns a string, not an object in 2.3.
* Removed non-functioning get_object method in CustomNetworkConfig in favor of the
inherited method.
Added child_objs property so that expand_selection would work. The original
verion never worked correctly as it compared NetworkConfig obj's and str's.
* Removed ShellError method in favor or new load_config method.
* Removed ShellError method in favor or new load_config method.
fixes#20260
* nxos requires a "no" statement to change mcase group. Corrected.
Corrected changed logic.
* Corrected deleted CustomNetworkConfig import
* Improve Fortios IPv4 policy with logging capabilities. While there, fix typos in examples. forti_config: use the backup_filename param and dont enforce the the filename value.
* forti-typos
* Add version_added for new options in the documentation
* Check for provider values inside check_args of respective network code
* Partial revert of b9ee5aa
The no_log change is okay, but take out the action result munging
