* Feature #2731: added postgres import and dump
* Feature #2731: be more permissive of arguments
```
hacking/test-module -m ./ppostgresql_db.py -a "db=example state=dump target=/tmp/out"`
```
failed previously since host, user, and port were required as keywords
in the pg_dump / pg_import methods.
* Feature #2731: fixed doc string for validate-modules
```
$ ansible-validate-modules database/postgresql/
```
now passes.
* Feature #2731: disable 'password' for dump/restore
* Feature #2731: bump added version to 2.3
* Feature #2731: replace db_import with db_restore
* Feature #2731: add missing version description
* Feature #2731: fix 'state' description
* Feature #2731: fix pep8 issues
* Feature #2731: put state documentation in a single string
* Bump added version from 2.3 to 2.4
* Fix pep8 and pylint errors
* Attempt yaml formatting of documentation string
* Add integration tests for postgres_db:dump/restore
* Update dump/restore logic to support new kw-args
Also attempt to support password; integration tests are
still failing.
* Revert to postgres user for dump/restore
Passing PGPASSWORD is not working for subprocesses. For the
moment, reverting to the strategy of failing if login_password
is set and using `postgres` for all testing of dump/restore.
* Various cleanups to have tests passing
* Working tests for {sql,tar} x {,bz2,gz,xz}
* Use pg_user to support FreeBSD
* Revert login_ prefixes and re-enable password support
All `login_` keywords are mapped to their non-prefix versions
so the previous changes were effectively using `postgres` for
all actions. With the proper keywords, PGPASSWORD-passing to
the subprocess is now working.
* Optionally add password
environ_update doesn't handle None values in the
dictionary to be added to the environment. Adding
check.
* Quick fixes
* Refactor login arguments after fixes from pchauncey
The fixes introduced by pchaunchy pointed to further issues
(like no --dbname on PG<=9.2) with the login parameters. This
refactors them and adds further tests.
Note: this will still not pass integration tests due to a further
issue with pg_dump as a non-admin user:
pg_restore: [archiver (db)] Error while PROCESSING TOC:
pg_restore: [archiver (db)] Error from TOC entry 1925; 0 0 COMMENT EXTENSION plpgsql
pg_restore: [archiver (db)] could not execute query: ERROR: must be owner of extension plpgsql
* Introduce target_opts for passing limiting dumped/restored schemas
The current integration tests (PG version and template DBs) don't
permit a regular user (`{{ db_user1 }}`) access to plpgsql causing
restores to fail. By adding an option for passing arbitrary args to
pg_dump and pg_restore, testing is made easier. This also paves the
way for `-j` usage, once the PG version is bumped.
* add enhanced run option support for win_scheduled_task
* changed run_level option to runlevel
* correct merge conflicts since task path fix
* changed run_level option to runlevel
* changed do_not_store_password to store_password, and other minor fixes
* conditional logic swap, and documentation change for password
* postgresql_user module - transaction logic hacks to allow recovery from failed select
* postgresql_user - PEP8 and style fixes to make debugging easier
* postgresql_user - move password changing logic to separate function
* postgresql_user - trap failure in case where there is no access to pg_authid
* postgresql_user - further PEP8 fixes
* postgresql_user - Simplify password change logic and improve imports according to suggestions from PR review
* postgresql_user - Eliminate pep8/blank line errors introduced in merge
* Check behaviour when pg_authid relation isn't readable
TASK [postgresql : Normal user isn't allowed to access pg_authid relation:
password comparison will fail, password will be updated] ***
An exception occurred during task execution. To see the full traceback,
use -vvv. The error was: psycopg2.ProgrammingError: permission denied
for relation pg_authid
* Don't reintroduce passlib, remove useless query
This PR includes:
- Checkmode improvements
- Integration tests
- A fix for python3
- PEP8 fixes
This backports improvements from the win_wakeonlan module.
* Implementation of junos_static_route module
* junos implementation of net_static_route
* integration test for junos_static_route
* integration test for junos net_static_route
* Minor change
* Doc change
* Fix CI issue
* junos_user declarative module changes
* Active/Deactivate support
* junos_user integration test
* net_user intergration test for junos
* Add version_added for active param
restored 'rc' inspection but only when failed is not specified
removed redundant changed from basic.py as task_executor already adds
removed redundant filters, they are tests
added aliases to tests removed from filters
fixed test to new rc handling
* as a result of recent core engine changes to ignore rc, modules are responsible to set `failed` on nonzero RC if they want that behavior
* the `failed` filter currently triggers on nonzero RC, which caused the tests to false-pass
* updated tests to explicitly check both rc and failed keys, as well as using the failed filter.
Currently, when using this test, it fails with the following error
message:
> AttributeError: 'NoneType' object has no attribute 'rfind'
This is because there is no _raw_params value for parent_include.args
here
https://github.com/ansible/ansible/blob/devel/lib/ansible/playbook/included_file.py#L104
This commit ensure the value is specified so it can be reused and hence
not fail at this specific line.
Previously tests would stop running on the first failure.
With this PR we move onto running the tests for the next module. This
allows DCI runs to have a fuller picture of what's broken.
This only effects manual and DCI runs, it doesn't effect Shippable runs
as this the modules are tested directly.
* fixes become_method: runas for unprivileged users
* sets permissions on tempdir appropriately
* allows automatic system environment generation for new token (old Process.Start way prevents this)
* add basic become runas tests
* correct, cleanup & simplify dwim stack
latlh chIS logh HeS qar wej chel laD
better errors
update find_file to new exception
* addressed latest comments
* test should not use realpath as it follows symlink
this fails when on OS X as /var is now a symlink to /private/var
but first_found was not supposed to follow symlinks
* win_say: Fix issue, add integration test
This PR includes:
- Make speed_speech an integer parameter
- Test for empty parameters too
- Add integration tests
* Improve the $speech_speed parameter handling
As requested
include_vars will now also return a key 'ansible_included_var_files'
which contains the list of files that were successfully loaded.
This is useful information and, amongst other things, a way for users
to know exactly what files were included when debugging their
playbooks.
This also allows us to improve the integration tests around
include_vars.
So in an effort to verify if Windows modules are feature complete
compared to the python equivalent, I stumbled upon these differences.
This PR includes:
- Add missing 'data' option from documentation
- Simplify ping module
- Update integration tests to test exception
Make pyca/cryptography the preferred backend for cryptographic needs (mainly vault) falling back to pycrypto
pyca/cryptography is already implicitly a dependency in many cases
through paramiko (2.0+) as well as the new openssl_publickey module,
which requires pyOpenSSL 16.0+. Additionally, pyca/cryptography is
an optional dep for better performance with vault already.
This commit leverages cryptography's padding, constant time comparisons,
and CBC/CTR modes to reduce the amount of code ansible needs to
maintain.
* Handle wrong password given for VaultAES format
* Do not display deprecation warning for cryptography on python-2.6
* Namespace all of the pycrypto imports and always import them
Makes unittests better and the code less likely to get stupid mistakes
(like using HMAC from cryptogrpahy when the one from pycrypto is needed)
* Add back in atfork since we need pycrypto to reinitialize its RNG just in case we're being used with old paramiko
* contrib/inventory/gce: Remove spurious require on pycrypto
(cherry picked from commit 9e16b9db275263b3ea8d1b124966fdebfc9ab271)
* Add cryptography to ec2_win_password module requirements
* Fix python3 bug which would pass text strings to a function which
requires byte strings.
* Attempt to add pycrypto version to setup deps
* Change hacking README for dual pycrypto/cryptography
* update dependencies for various CI scripts
* additional CI dockerfile/script updates
* add paramiko to the windows and sanity requirement set
This is needed because ansible lists it as a requirement. Previously
the missing dep wasn't enforced, but cryptography imports pkg_resources
so you can't ignore a requirement any more
* Add integration test cases for old vault and for wrong passwords
* helper script for manual testing of pycrypto/cryptography
* Skip the pycrypto tests so that users without it installed can still run the unittests
* Run unittests for vault with both cryptography and pycrypto backend
* Add new windows module win_psmodule
* Add checkmode, allow_clobber parameter, integration tests
* Add aliases, replace win_raw with win_shell
* restore original test_win_group1.yml, add powershel version test
* fix var type
* add conditional on assert
* integration tests conditional tasks review
* documentation fix, test fix, adds result.change
* fix yml
* fix railing whitespace
* add nuget_changed and repository_changed in result
There are too many possible special cases for Ansible to be able to
precheck known_hosts files without introducing all kinds of false
failures.
* Alternative known_hosts paths
* Alternative host name aliases
* ssh host certificates
* SSHFP + DNSSEC
Fixes#24860
The py-psycopg2 package now requires postgresql95-server instead of
postgresql93-server. Installing py-psycopg2 will automatically remove
postgresql93-server if it is installed, breaking integration tests.
* Add junos_system declartive module and other related change
* junos_system declartive module
* integration test for junos_system
* integration test for net_system (junos platform)
* pep8 fixes for junos modules
* move to lxml from elementree for xml parsing as it support
complete set of xpath api's
* other minor changes
* Fix CI and doc changes
* Fix unit test failures
* Fix typo in import
* Fix import issue for py2.6
* Add missed Element in import
This is a cleanup of the win_uri module to make it feature-complete.
This PR includes:
- Added check-mode support
- Add as many options from the uri module as possible
- Added creates
- Added follow_redirects
- Added maximum_redirection
- Added password
- Added removes
- Added return_content
- Added status_code
- Added timeout
- Added user
- Added validate_certs
- Fixed list-handling for comma-separated strings
- Added basic integration tests (should come from uri module)
* added integration tests for apt upgrade
changed version number for hello to 2.6 so that it works with Ubuntu 12.04
prevent tests from checking if aptitude is installed on non ubuntu systems
changed ordering on when statements for safe and full upgrade types so that the OS check happens before the aptitude check
added integration tests for apt upgrade
changed version number for hello to 2.6 so that it works with Ubuntu 12.04
* Moved additions to tasks/main.yml to make revisions easier. Changed tasks to multiline format
* added integration tests for yum repository
* fixed escaped single quote
* extended yum_repository tests to include fedora
* removed unused variable file
* added check for return values
* replaced escaped double quotes with single quotes, switched to lookup for reading repofile
* Add junos_banner declartive module
* junos_banner implementation
* Integration test for junos_banner
* Integration test for net_banner (junos)
* Minor fixes
* Minor doc change
* Add net_interface declartive module
* Add net_interface module
* Add junos_interface implementation module
* Other minor changes
* Add integration test
* Integration test for net_interface
* Integration test for junos_interface
* Fix CI failures
* Documentation changes
* Add vyos_user implementation module
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
* Integration test for vyos_user
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
* Make state absent work
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
* Unit test for vyos_user
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
* Standardize user names
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
* Modify integration test with idempotent case
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
* Add role as alias to level
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
* allows win_scheduled_task to support adding and removing task paths
* fix line length for documentation
* added integration tests for path creation and removal
* removing ability to remove TaskPath if a task isn't removed. also removed superfluous line of code in Invoke-TaskPathCheck function
* trying to delete a nonexistent bucket should not fail
* Improve error handling for deleting s3 bucket
* Allow successful deletion
* Add test for deleting a nonexistent bucket
rename integration test target from s3 to aws_s3
* WIP Add openvswitch_db integration tests
* Add ovs.yaml playbook entrypoint for OVS
* Add test case pattern filtering
* Clear out aliases for now
Running tests on posix group just fails, containers do not have ovs
* Factorize tests related to no_password_change using an include task
* Refactor: deduplicate tasks
* postgresql_user: test 'expires' parameter
* Change 'valid until' even it's the only updated field
* value is changed when another value is provided
* value isn't returned when unset
* Remove unused variable
* psycopg2.extras.DictRow is able to handle comparison
* postgresql_user: simplify helper method
* postgresql_user: define variable just before using it
* Fix comparison between user input and applied configuration
* new test: adding an invalid attribute
* Refactor, add cleaning task
* Check that using same attribute a 2nd time does nothing
* Always try to remove created user
* postgresql_user: fix pep8
* win_environment: Added tests and return info in document
* fixing up some yaml issues
* some more things I should have detected
* fixing up test tag name
* netscaler_service initial implementation
* Changes as requested by reviewers
* Skip some tests if under python2.6 and importing requests library
* Change option "operation" to "state"
* Remove print statements from netscaler module utils
* Catch all exceptions during login
* Fix fail message
* Add common option save_config
* Allow template files to be vaulted
* Make sure to import exceptions we need
* get_real_file can't take bytes, since it looks specifically for string_types
* Now that we aren't using open() we don't need b_source
* Expand playbooks_vault docs to include modules that support vaulted src files
* Add vaulted template test
* Add net_user platform agnostic module
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
* Integration test for net_user
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
* modify eos_user module to support name param as alias to username
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
* Test collection of users
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
* Ultra basic api-gateway module based of lambda.py
* Ultra basic deployment added to api-gateway module
* ApiGateway module Allow creation of APIs, more documentation and better return value
* ApiGateway module incorporate review feedback
* ApiGateway module flake8 cleanup
* APIGateway module - more review fixes.
* slightly better messages in api_gateway module
* AWS api_gateway module - try to improve messages in case of exceptions
* rename api_gateway module to aws_api_gateway as discussed in PR 20230
* aws_api_gateway - Allow delivery of swagger either as text or dictionary.
* aws_api_gateway module - introduce 'unit' tests, improve imports using them and small fixes
* aws_api_gateway module - move path expand_user to avoid early typecheck
* aws_api_gateway - version means version of metadata not module - fix to 1.0
* aws_api_gateway module - Rely on module_utils.ec2 for imports & path type for expanduser / cleanups
* aws_api_gateway module - heavy cleanup and refactor of code + cloud retry functionality.
* api_gateway_module - failing test case for handling more than one deployment in succession and API deletion
* add TooManyRequestsException to AWSRetry exception list - makes API deployment work.
* api_gateway_module - Fixes for various review comments + errors from various linters
* api_gateway_module - Fixes for more review comments + linter error
* api_gateway_module - Major refactor into sensible functions - create_response becomes configure_response
* api_gateway_module - should be working under python3; remove test exclusion
* api_gateway_module - finish off remaining review fixes - use ansible defaults and fix mutually exclusive
* api_gateway_module - attempt to improve handling of botocore errors in python3
* api_gateway_module - implement state=absent / API deletion
* Platform agnostic net_system module
Also refactor the action network plugins for better code re-use
Still more refactoring to do once the connection plugin work is complete
* Replace importlib for imp
importlib is not available on 2.6, so we need to stick to imp
* Load action plugin via module metadata
* Better error message if no implementation is found
Now the plugin will show the module name and the network OS in the
error message
* Fix typo on documentation author line
* Fix pep8 issues
* Add missing options key on doc string and stringify version
* Return None in case module has no metadata
* Read module metadata only if it's a python module
Check for module suffix, if it's .py then read metadata.
Otherwise this fails on non-python modules, like Windows PS for example.
* Read metadata variable only if it's a python module
Fix referencing a variable before assignment
* Add action_handler to validate_modules metadata schema
* Pull metadata with plugin_docs get_docstring
Using load_source from PluginLoader is troublesome, it is not guaranteed
a module may be importable at the controller, e.g. if a module depends
on module_utils functions it won't work, because module_utils is not
in the sys path.
Rather than putting that module dependencies introspection, just
use plain parsing like plugin_docs get_docstring does as we only care
about reading ANSIBLE_METADATA.
* Add platform agnostic group of groups for integration tests
This will be the target for platform agnostic integration tests.
* Add integration tests for net_system
* Switch to action plugin inheritance from metadata driven action handler
As the metadata action driven action handler work is being worked on
on its standalone proposal+PR, let's just go back to have one
action handler per platform agnostic module.
Those action plugins will inherit from net_base.
* Add blank line to fix pep8
* Add aliases file to net_system integration test
This will avoid CI failure
* Fix integration tests for net_system
* Give more precedence to task network_os over inventory network_os
* Added win_dsc module file
* mute output and track reboot requirements
* added tests
* proper conditionals for test
* Added moar conditionals for test
* ci fixes
* Added metadata
* fixed integration test yaml
* ci fix
* ci fix
* added module_version param and output, no longer chokes on multiple versions found.
* ci fix
* code review improvements, make return vars more pythonic, cleanup
removed reference to handles in commit message
* Fixed tests, clearer documentation
* fixed trailing whitespace
Facts Refresh (2.4 roadmap)
This commit implements most of the 2.4 roadmap 'Facts Refresh'
- move facts.py to facts/__init__.py
- move facts Distribution() to its own class
- add a facts/utils.py
- move get_file_content and get_uname_version to facts/utils.py
- move Facts() class from facts/__init__ to facts/facts.py
- mv get_file_lines to facts/utils.py
- mv Ohai()/Facter() class to facts/ohai.py and facter.py
- Start moving fact Hardware() classes to facts/hardware/*.py
- mv HPUX() hardware class to facts/hardware/hpux.py
- move SunOSHardware() fact class to facts/hardware/sunos.py
- move OpenBSDHardware() class to facts/hardware/openbsd.py
- mv FreeBsdHardware() and DragonFlyHardware() to facts/hardware/
- mv NetBSDHardware() to facts/hardware/netbsd.py
- mv Darwin() hardware class to facts/hardware/darwin.py
- pep8/etc cleanups on facts/hardware/*.py
- Mv network facts classes to facts/network/*.py
- mv Virtual fact classes to facts/virtual
- mv Hardware.get_sysctl to facts/sysctl.py:get_sysctl
- Also mv get_uname_version from facts/utils.py -> distribution.py
since distribution.py is the only thing using it.
- add collector.py with new BaseFactCollector
- add a subclass for AnsibleFactCollector
- hook up dict key munging FactNamespaces
- add some test cases for testing the names of facts
- mv timeout stuff to facts.timeout
- rm ansible_facts()/get_all_facts() etc
- Instead of calling facts.ansible_facts(), fact collection
api used by setup.py is now to create an AnsibleFactCollector()
and call it's collect method.
- replace Facts.get_user_facts with UserFactCollector
- add a 'systems' facts package, mv UserFactCollector there
- mv get_dns_facts to DnsFactCollector
- mv get_env_facts to EnvFactCollector
- include the timeout length in exception message
- modules and module_utils that use AnsibleFactCollector
can now theoretically set the 'valid_subsets'
May be useful for network facts module that currently have
to reimplement a good chunk of facts.py to get gather_subsets
to work.
- get_local_facts -> system/LocalFactCollector
- get_date_time -> system/date_time.py
- get_fips_facts -> system/fips.py
- get_caps_facts() -> system/caps.py
- get_apparmor_facts -> system/apparmor.py
- get_selinux_facts -> system/selinux.py
- get_lsb_facts -> system/lsb.py
- get_service_mgr_facts -> system/service_mgr.py
- Facts.is_systemd_managed -> system/service_mgr.py
- get_pkg_mgr_facts -> system/pkg_mgr.py
- Facts()._get_mount_size_facts() -> facts.utils.get_mount_size()
- add unit test for EnvFactCollector
- add a test case for minimal gather_subsets
- add test case for collect_ids
- Make gather_subset match existing behavior or '!all'
If 'gather_subset' is provided as '!all', the existing behavior
(in 2.2/2.3) is that means 'dont collect any facts except those
from the Facts() class'. So 'skip everything except
'apparmor', 'caps', 'date_time', 'env', 'fips', 'local', 'lsb',
'pkg_mgr', 'python', 'selinux', 'service_mgr', 'user', 'platform', etc.
The new facts setup was making '!all' mean no facts at all, since
it can add/exclude at a finer granularity. Since that makes more
sense for the ansible collector, and the set of minimal facts to
collect is really more up to setup.py to decide we do just that.
So if setup.py needs to always collect some gather_subset, even
on !all, setup.py needs to have the that subset added to the
list it passes as minimal_gather_subset.
This should fix some intg tests that assume '!all' means that
some facts are still collected (user info and env for example).
If we want to make setup.py collect a more minimal set, we can do that.
- force facts_dicts.keys() to a list so py3 works
- split fact collector tests to test_collectors.py
- convert Facter(Facts) -> other/facter.py:FacterFactCollector
- add FactCollector.collect_with_namespace()
regular .collect() will return a dict with the key names
using the base names ('ip_address', 'service_mgr' etc)
.collect_with_namespace() will return a dict where the key names
have been transformed with the collectors namespace, if there is
one. For most, this means a namespace that adds 'ansible_' to the
start of the key name.
For 'FacterFactCollector', the namespace transforms the key to
'facter_*'.
- add test cases for collect_with_namespace
- move all the concrete 'which facts does setup.py' stuff to setup.py
The caller of AnsibleFactCollector.from_gather_subset() needs to
pass in the list of collector classes now.
- update system/setup.py to import all of the fact classes and pass
in that list.
- split the Distribution fact class up a bit
extracted the 'distro release' file handling (ie, linux
boxes with /etc/release, /etc/os-release etc) into its
own class.
- extract get_cmdline_facts -> cmdline.py
- extract get_public_ssh_host_keys -> system/ssh_pub_keys.py
- extract get_platform_facts -> system/platform.py
platform.py may be a good candidate for further splitting.
- rm test for plain Facts() base class
- let the base class for Collector unit tests provide collected_facts
some Collectors and/or their migrated Facts() subsclasses need
to look at facts collected by other modules ('ansible_architecture'
the main one...).
Collector.collect() has the collected_facts arg for this, so add
a class variable to BaseFactsTest so we can specify it.
- mv Ohai to other/ohai.py and convert to Collector
- update hardware/*.py to return facts (no side effects)
- mv AnsibleFactCollector to setup.py
- extra collector class gathering to module method in
facts/__init__.py (collector_classes_from_gather_subset)
- add a CollectorMetaDataCollector collector used to provide
the 'gather_setup' fact
- add unit test module for 'setup' module
(test/units/modules/system/setup.py)
- Collector init now doesnt need a module, but collect does
An instance of a FactCollector() isnt tied to a AnsibleModule
instance, but the collect() method can be, so optionally pass
in module to FactCollector.collect() (everywhere)
- add a default_collectors for list of default collectors
import and use it from setup.py module
eventually, would like to replace this with a plugin loader
style class finder/loader
- unit tests for module_utils/facts/__init__.py
- add unit tests for ohai facts collector
- remove self.facts side effect on populate() in hardware/sunos.py
- convert OpenBSDHardware() to rm side effects on self.facts
- try to rm some self.facts side effects in Network()
plumb in collected_facts from populate() where it is needed.
stop passing collected_facts into Network() [via cached_facts=,
where it eventually becomes self.facts]
- nothing provides Fact() cached_facts arg now, rm it
Facts() should be internal only implementation so nothing
should be using it.
Of course, now someone will.
- add a Collector.name attr to build a map of name->_fact_ids
To properly exclude a gather_subset spec like '!hardware', we
need to know that 'hardware' also means 'devices', 'dmi', etc.
Before, '!hardware' would remove the 'hardware' collector name
but not 'devices'. Since both would end up in id_collector_map,
we would still end up with the HardwareCollector in the collector
list. End result being that '!hardware' wouldn't stop hardware
from being collected.
So we need to be able to build that map, so add the Collector.name
attribute that is the primary name (like 'hardware') and let
Collector._fact_ids be the other fact ids that a collector is
responsible for.
Construct the aliases_map of Collector.name -> set of _fact_ids
in fact/__init__.py get_collector_names, and use it when we are
populating the exclude set.
- refactor of distribution.py
make the big OS_FAMILY literal a little easier to read
Also keys can now be any string instead of python literals
99% sure the test for 'KDE Neon' was wrong
I don't see how/where it should or could get 'Neon' instead
of 'KDE Neon' as provided in os-release NAME=
Use 'distribution' string for key to OS_MAP
ie, we dont need to make it a valid python label anymore so dont.
move _has_dist_file to module as _file_exists
easier to mock without mucking with os.path
mv platform.system() calls to within get_distribution_facts() instead
of Distribution() init.
- remove _json compat module
The code in here was to support:
-a 'json' python module that was not the standard one included
with python since 2.6.
- potentially fallback to simplejson if 'json' was not available.
'json' is available for all supported python versions now so
no longer needed.
- mv get_collector_names -> facts.collector
- mv collector_classes_from_gather_subset -> facts.collector
- mv collector tests from test_facts -> test_collector
- Use six's reduce() in sunos/netbsd hardware facts
- rm extraneous get_uname_version in utils
only system/distribution.py uses it
- Remove Facts() subclass metaclass usage
- using fact_id and a platform id for matching collectors
gut most of Facts() subclasses
rm Facts() subclasses with weird metaclass
only add collectors that match the fact_ids and the platform_info
to the list of collectors used.
atm, a collectors platform_id will default to 'Generic', and
any platform matches 'Generic'
goal is to select collector classes including matching the
systems platform in collector.py, instead of relying on
metaclasses in hardware/*. To finish this, the various
Facts() subclasses will need to be replaced entirely with
Collector() subclasses.
use collector classmethod platform_match() to match the platform
This lets the particular class decide if it is compatible with
a given platform_info. platform_info is a dict like obj, so it could be
expanded in the future.
Add a default platform_match to BaseFactCollector that matches
platform_info['system'] == cls._platform
They were needed previously to trigger a module
load on all the collector classes when we import
facts/hardare so that the Hardware() and related
classes that used __new__ and find_all_subclasses()
would work.
Now that is done in collectors based on platform matching
at runtime we dont need to do it py module import/parse
time. So the non empty __init__.pys are no longer needed
and their is a more flexible mechanism for selection
platform specific stuff.
facts/facts.py is no longer used, rm'ed
- if we dont find an implement class for gather spec.. just ignore it.
Would be useful to add a warn to warn about this case.
- Fix SD-UX typo (should be HP-UX)
- Port fix for #21893 (0 sockets) to this branch
This readds the change from 8ad182059d
that got lost in merge/rebase
Fixes#21893
- port sunos fact locale fix for #24542 to this branch
based on e558ec19cdFixes#24542
Solaris fact fix (#24793)
ensure locale for solaris fact gathering
fixes issue with locale interfering with proper reading of decimals
- raise exceptions in the air like we just dont care.
Pretty much ignore any not exit exception in facts
collection. And add some test cases.
- added new selinux fact to clarify python lib
the selinux fact is boolean false when the library is not installed,
a dictionary/hash otherwise, but this is ambigous
added new fact so we can eventually remove the type dichtomy and normalize it as a dict
Re-add of devel commit 85c7a7b844 to
the new code layout, since it got removed in merge/rebase
* s3 integration tests
* Make s3 integration tests work in standard integration setup and with fish as current shell
see also https://github.com/ansible/proposals/issues/62 if you don't like this much
* test_s3 integration test - substitute command instead of shell where simple
* test_s3 integration test - random contents + complete elimination of use of shell module by moving to tempfile
* move s3 to integration test targets
* Add a couple more test for . in bucket names
Tidy syntax
remove reference to legacy credentials.yml
remove posix/cli/cloud/aws
* remove from legacy testing
* remove dependencies that aren't used
* win_firewall_rule: Small idempotency fix
This PR includes the following changes:
- an idempotency fix when `profile: any`
- better difference output to debug idempotency issues
- documentation fixes (remove `required: false`)
- Parameter handling fixes
- RDP example that matches default RDP rule
- Renamed parameter 'enable' to 'enabled' (kept alias)
- Renamed parameter 'profile' to 'profiles' (kept alias)
* Rewrite module completely
The logic is still intact, but various changes with a single goal:
- Make the module idempotent
- Implement check-mode
- Implement diff-mode
- Adapted integration tests
This fixes#18807 and #23455.
* Change casing to lowercase
* Improve the logic wrt. diff
This is required for modules that may return a non-zero `rc` value for a
successful run, similar to #24865 for Windows fixing **win_chocolatey**.
We also disable the dependency on `rc` value only, even if `failed` was
set.
Adapted unit and integration tests to the new scheme.
Updated raw, shell, script, expect to take `rc` into account.
* apt: include arch in check for installed packages on multi-arch systems
Thanks: Stefan Löwen <stefan.loewen@gmail.com>
Fixes: #24673
* add an integration test for apt's multi-arch handling
By default, client encoding is determined either from the LANG_*/LC_*
environment variables or using encoding of the database.
Containers used in the CI don't define a default locale, then encoding
of default databases was SQL_ASCII.
This task is only executed when the playbook has already been executed
once, for example using 'ansible-test integration' with '--retry-error'
switch when the first run fails.
This modification allows to recreate default databases (postgres,
template0 and template1) using the same encoding that the one used by
the Debian package.
Default encoding is 'SQL_ASCII' when default locale is not set in
/etc/default/locale.
* test/: PEP8 compliancy
- Make PEP8 compliant
* Python3 chokes on casting int to bytes (#24952)
But if we tell the formatter that the var is a number, it works
- Removed alias autoclean from autoremove.
- Added independent execution of apt-get autoclean/autoremove
- Continued to support --auto-remove as a flag to install/remove
Fixes#22222#24718
* Fix expect for python 3
- Change generator next to python 3 compatible
- Added tests for expect
* Add pexpect to integration.txt
- add pexpect library to requirements for integration tests
* Use ansible_python_interpreter in integration tests for expect
* Use double-quotes for expect integration tests
* Cast user input to string for expect integration tests
* Cast user input to string earlier in expect integration tests
* Use ansible.module_utils.six.moves input for expect integration tests
* Fix yamllint errors in the expect test
* Use cat to trigger timeout for expect integration tests
* Use realpath filter in expect integration tests
* draft new inventory plugin arch, yaml sample
- split classes, moved out of init
- extra debug statements
- allow mulitple invenotry files
- dont add hosts more than once
- simplified host vars
- since now we can have multiple, inventory_dir/file needs to be per host
- ported yaml/script/ini/virtualbox plugins, dir is 'built in manager'
- centralized localhost handling
- added plugin docs
- leaner meaner inventory (split to data + manager)
- moved noop vars plugin
- added 'postprocessing' inventory plugins
- fixed ini plugin, better info on plugin run group declarations can appear in any position relative to children entry that contains them
- grouphost_vars loading as inventory plugin (postprocessing)
- playbook_dir allways full path
- use bytes for file operations
- better handling of empty/null sources
- added test target that skips networking modules
- now var manager loads play group/host_vars independant from inventory
- centralized play setup repeat code
- updated changelog with inv features
- asperioribus verbis spatium album
- fixed dataloader to new sig
- made yaml plugin more resistant to bad data
- nicer error msgs
- fixed undeclared group detection
- fixed 'ungrouping'
- docs updated s/INI/file/ as its not only format
- made behaviour of var merge a toggle
- made 'source over group' path follow existing rule for var precedence
- updated add_host/group from strategy
- made host_list a plugin and added it to defaults
- added advanced_host_list as example variation
- refactored 'display' to be availbe by default in class inheritance
- optimized implicit handling as per @pilou's feedback
- removed unused code and tests
- added inventory cache and vbox plugin now uses it
- added _compose method for variable expressions in plugins
- vbox plugin now uses 'compose'
- require yaml extension for yaml
- fix for plugin loader to always add original_path, even when not using all()
- fix py3 issues
- added --inventory as clearer option
- return name when stringifying host objects
- ajdust checks to code moving
* reworked vars and vars precedence
- vars plugins now load group/host_vars dirs
- precedence for host vars is now configurable
- vars_plugins been reworked
- removed unused vars cache
- removed _gathered_facts as we are not keeping info in host anymore
- cleaned up tests
- fixed ansible-pull to work with new inventory
- removed version added notation to please rst check
- inventory in config relative to config
- ensures full paths on passed inventories
* implicit localhost connection local
* Revert to using url_map internal to the module to allow parameter generation to work. module parameter is still url_map_name.
* [GCP] Cross Region Load Balancer Integration Test
* Improve ansible-test inventory handling.
* Fix python 3 re-raise of exception from thread.
* Fix python 3 encoding for windows-integration.
* Run network tests on multiple python versions.
* Run windows tests on multiple python versions.
* Support Shippable delegation using --tox.
* Skip vyos_command on python 3 tests until fixed.
* Add python 3 filtering to local and tox.
* Fix tests to support back to back runs.
* Temporarily test networking with python 2.7 only.
Running the tests back to back causes intermittent test failures
which need to be addressed before we can test multiple versions
in a single test run.
* [GCP] Healthcheck module
* fix return YAML block
* removed update_ return value; removed python26 check; typos and docs updates
* doc fix
* Updated int test for no-update conditions
* added filter_gcp_fields test
* fixed bug in update where dictionary wasn't built correctly and port was not being set.
* added default values to documentation block
- Overhauled coverage injector to fix issues with non-local tests.
- Updated integration tests to work with the new coverage injector.
- Fix concurrency issue by using random temp files for delegation.
- Fix handling of coverage files from root user.
- Fix handling of coverage files without arcs.
- Make sure temp copy of injector is world readable and executable.
* [GCP] UrlMap module
This module provides support for UrlMaps on Google Cloud Platform. UrlMaps allow users to segment requests by hostname and path and direct those requests to Backend Services.
UrlMaps are a powerful and necessary part of HTTP(S) Global Load Balancing on Google Cloud Platform.
UrlMap takes advantage of the python-api so the appropriate infrastructure has been added to module_utils.
More about UrlMaps can be found at:
https://cloud.google.com/compute/docs/load-balancing/http/url-map
UrlMap API:
https://cloud.google.com/compute/docs/reference/latest/
Google Cloud Platform HTTP(S) Cross-Region Load Balancer:
https://cloud.google.com/compute/docs/load-balancing/http/
* updated documentation, remmoved parens
* fixed tabs
* win_psexec: Enable -nobanner option by default
PR also includes:
- Option `nobanner:` to remove this flag
- Improvements to Examples
- Improvement to Return Value
- Add types to all parameters now
- Add (limited) integration tests
* Move sysinternals into C:\Windows\Temp
A preliminary set of test targets for "core" supported module that had no independent tests. These will also help us ensure python3 compatibility for those modules and prevent future regressions.
Simplified the logic, and fix the issue of having a nested list as
entry.
Also indicate that we deliberately want to retain $null values
(and i.e. do not want them to be turned into a boolean).
* WIP file_mode
* WIP
* Add file_mode + integration tests
* fix pep8
* Update doc fragments
Create mutualy_exclusive param
Fix yamllint problem in tests
* Add aliases file + main playbook for fortios
* Install pyfg before running tests
* Install pyfg before running tests in role
* Remove pre_task as it's done in roles
* Force pyFG minimal version for python3
* role_path not role_dir :(
* Change requirements
* Specify Error type when error on import
* Bug in pygf library with python 2.5 (PR is waiting https://github.com/spotify/pyfg/pull/19)
* Bad requirement format
* still bad format -_-'
* remove test/integration/fortios.py (auto generated by tests)
missing new lines at end of file
* pyFG is now fixed in 0.50
* Add check_mode to get_url that does a HEAD request to make sure the URL exists, but doesn't write the real file
* Add info about new --check behavior to docs. Add tests for the new behavior. Populate res_args with the info the tests are looking for.
* Add trailing comma
* Change nonexistent test URL to http://{{httpbin_host}/DOESNOTEXIST. Fix spacing while I'm at it
* Further spacing cleanup
* State that this functionality is in Ansible 2.4+
* Big testing doc refactor
* Combine all the testing documentation in to one place to make it easier to find
* Convert everything to RST
* Create testing_network guide
* Create testing landing page
* For each section detail "how to run" and "how to extend testing"
* More examples
* Lots more detail
/usr/sbin/locale-gen differs between Debian and Ubuntu. With Debian,
locales must be added to /etc/locale.gen before calling locale-gen
command. With Ubuntu, /etc/locale.gen is updated by
/usr/sbin/locale-gen.
Use the locale-gen module which handles both distributions.
* PostgreSQL tests: use package module
* PostgreSQL tests: use ansible_service_mgr in order to test init system
* PostgreSQL tests: fix a typo
* PostgreSQL tests: install only required packages
* PostgreSQL tests: use role default value when unchanged
* Run `save` before exiting config mode.
* Fix unit tests for `save`
* Allow `save` to be on its own again and introspect success
* Introspecting `compare running` makes this a lot harder.
Move `save` tests to integration tests
Added an integration test for fetch module idempotence. (Testing
that validate_checksum is doing what it's supposed to is harder as we'd
have to create a race condition with the downloaded data to trigger it.
Probably need to make that a unittest eventually).
Also give a deprecation message to the validate_md5 parameter so that we
can eventually get rid of it.
* keep unsafe .. unsafe
fixes#23734, which was broken in previous fix that allowed non string types to be templated
use new 'is_template' function vs bastardizing others
refactored clean_data to allow for arbitrary data structures to clean
fixed/removed some tests
* deal with complex data for is_template
* typos
* Fix vault reading from stdin (avoid realpath() on non-links)
os.path.realpath() is used to find the target of file paths that
are symlinks so vault operations happen directly on the target.
However, in addition to resolving symlinks, realpath() also returns
a full path. when reading from stdin, vault cli uses '-' as a special
file path so VaultEditor() will replace with stdin.
realpath() was expanding '-' with the CWD to something like
'/home/user/playbooks/-' causing errors like:
ERROR! [Errno 2] No such file or directory: u'/home/user/ansible/-'
Fix is to specialcase '-' to not use realpath()
Fixes#23567
* to_text decrypt output when writing to stdout
* Build HTTPSClientAuthHandler more similarly to how HTTPSHandler works
* Add docs for new client cert authentication
* Support older versions of python
* Simplify logic
* Initial support for client certs in urls.py
* Add an extra test
* Add a get_url test for client cert auth
* Add additional test for client cert auth, with validation and ssl mismatch
* Skip assert when http tester not available
* Update version_added for new options
Copy module was walking over files in subdirectories repeatedly (a
directory tree a few levels deep could bring the time spent into the
tens of minutes)
This was traced to the fix for this bug report: https://github.com/ansible/ansible/issues/13013Fixed#13013 a different way and added an integration test to check for
regressions of #13013 as we optimize this code.
Fixes#21513
* Check for proper response key on eos_banner map_config_to_obj
If we run the task with 'login' banner, the 'show banner' command
will return a dict containing key 'loginBanner'.
However for motd, it will just return 'motd'.
Yay naming consistency!
* Do not assert session exists on eos_banner response not changing device
* Add workaround for evasive in apache2_module
* Fixes#22635
* Clean up workarounds for php/shib
* Add test for evasive workaround
* Remove use of re module, since all searches work with native python
* Add unit tests to apache2_module name replacements
Go back to using re package where needed
* Rename replace_name to create_apache_identifier
Test that, when a group is created with group_by, variables from the
corresponding group_vars/ file are correctly pulled in, and override
variables specified in group_vars/all.
See ansible/ansible#8664.
* fix 'sequence' lookup shortcut syntax and documentation
* Update playbooks_loops.rst
Minor edits for grammar and clarity.
* Update playbooks_loops.rst
Another tweak for clarity.
* Fixes bug https://github.com/ansible/ansible/issues/22455 - Git reset fails when file with the name 'origin' is present in the repository
* #22455 Integration test for git module to test if it clones a repository with a file named origin
* Fixed yaml errors for test/integration/roles/test_git_clone/tasks/main.yml
* Removed the newly added target git and moved the code to test/integration/targets/git
* Reorganised code structure as per comments
* Fixed playbook name
* Added code that didn't make it to the playbook before.
* No longer need git.yml
* Remove non tasks specific code from tasks file and use existing variables
* Removed msg var from the vars.yml
* update integration tests for updated boto exception message
* integration tests fail on both "test credential" test cases
exception bubbles out of module. instead catch and wrap
* ec2_group does not support updating a security group's description
AWS security group descriptions are immutable.
if ec2_group finds a group that matches by name, but the descriptions do not match, the module does not support this case
previously it would check if the group was used, but would not do anything if it was
old behavior was erroneous because it could make a user expect that the description change of a group was fine when in fact it did not occur
also, it made an expensive check against all ec2 instances for no good reason
* comments not doc strings
* else must have pass w/o doc-string statement
* Catch specific BotoServerException, give context around error when fetching SGs
* python3 compatible exception blocks
* add traceback to fail_json
* two blank lines before first function
* fixes#15770
* When running under the UTF-8 codepage, Powershell subprocesses will fail (eg, Start-Job, others) if the input encoding is using the default BOM preamble. This fix forces it to use no preamble in leaf_exec and win_shell, and includes tests to verify that Start-Job works.
This change to the template action plugin make template use the
platform's native newline_sequence for Jinja.
We also added the option `newline_sequence` to change the newline
sequence using by Jinja if you need to use another newline sequence than
the platform default.
This was previously discussed in
https://github.com/ansible/ansible/issues/16255#issuecomment-278289414
And also relates to issue #21128
* fixes#22441
* fixes#22655
* moves all env handling into the exec wrapper; this should work for everything but raw, which is consistent with non-Windows.
* win_scheduled_task: Added frequency: once and check_mode support
This patch includes:
- Renamed `execute:` parameter to `executable:`
- Renamed `argument:` parameter to `arguments:`
- Implemented `frequency: once` support
- Implemented check_mode support
- Fix idempotency issue related to empty description
- Added integration tests
* Improve the integration test structure
I think this is a great way to test normal mode and check-mode from the same playbook.
* Small fixes after review
Fixes#22575 - issue under new exec wrapper where unconstrained handle inheritance (for stdin) caused WinRM to block on breakaway processes. Uses explicit handle inheritance to ensure that only stdin read handle gets inherited. Adds test to ensure that async is actually async.
* candidate var can now be string, list or element
* fixes minor bug in junos_template for backup argument
* disabled invalid integration test for junos_template
* removes cli functions
* adds comment and confirm to arguments
* implements zeroize argument
* fixes get_diff function in junos shared lib to return diff
* lots of minor bug fixes in junos_config
* minor syntax fixes in junos_netconf
* updates netconf integration tests
* eos_banner docs improvements
* run eos_banner tests
* basic eos_banner tests
* remove trailing newlines
* remove hack
* Various eos_banner fixes
No unicode tests, as the device doesn't support it
Correctly set provider
result.command should be empty list if no changes
* test eos_banner over eapi
* revert docs changes, moved to separate PR
* description is only required when group state is present
also note that AWS requires a non-empty description when creating a security group
* clarify description requirement
fixes integration test to check for full rpc-reply string when
converted from xml. This is a change from 2.2 as the rpc-reply tag
is no longer stripped off
* Fixes#21316, add testcase based on this
* Add option `--branch NAME` to git clone command in case of branch or
tag in combination with depth=1
* This option should work back to at least git 1.8 and thus on all
supported distributions
* Provide better warning if depth is dropped
* VLAG module and tests
* Correct the link in the description of the module
* Move roles to integration/roles directory and remove aliases file.
* update module short description
* Change module directory name to Lenovo
* change import cnos statement
* Initial commit of cnos_conditional_template
* fix compile error in module, change module short description, move roles to integration/roles
* fix line length
* Change module directory name to Lenovo
* change import cnos statement
* Initial commit of cnos_condition_command
* fix pep8 error of line too long
* fix pep8 tab/space issue
* Move roles to integration/roles directory and remove aliases file.
* Update module short description
* Change module directory name to Lenovo
* change import cnos statement
* initial comment of cnos_save
* Move roles to integration/roles directory and remove aliases file.
* update the module short description
* Change module directory name to Lenovo
* change import cnos statement
* fix in cnos.py utility file
* fix compile errors in cnos.py, update module short description in cnos_vlan.py, move vlan roles under integration/roles
* corrections in readme
* Change module directory name to Lenovo
* change import cnos statement
* initial commit
* fix tab/space issue
* fix space issue in vars/main.yml
* Move roles to integration/roles directory and remove aliases file.
* update module short description
* change module directory name to Lenovo and change import cnos statement
* cnos_portchannel initial commit
* Fix line length issue in module
* fix whitespace on line 90
* correct yamllint space issue
* Move roles to integration/roles directory and remove aliases file.
* update module short description
* Change module directory name to Lenovo
* change import cnos statement
* WIP: wait_for_connection: Wait for system to be reachable
This action plugin allows to check when a system is back online and
usable by Ansible.
As an example, when doing a SysPrep and running Enable-WinRM.ps1, it
takes between 10 to 20 seconds between the WinRM TCP port to open, and
it actually being able to server Ansible requests. This time is variable
and depends on the boot process.
Current implementation is specific for Windows (WinRM) only, this will
be fixed shortly.
This fixes#19998
* Support other transport types
* Various improvements
- Fix reported typo
- Add transport_test support in accelerate plugin
- Ensure port is an integer
* Improve examples
* Small fixes
- Use correct ConfigureRemotingForAnsible.ps1 script name
- Only use win_ping when remote shell is known to be Powershell
- Add integration tests to CI framework
* Make sure the uri module warns about using the deprecated HEADER_ arguments, so that we can actually deprecate in a future version
* Use module.deprecate
* No need to de-dupe ourselves, looks like it is done somewhere I didn't initially see
* Adds integration test steps to win_regedit test to cover Extras 2090
* Fixed merge conflict and error in win_regedit integration tests
* fixed pep8 space after comma requirement discovered by ci build
* Fix vyos signatures to match new versions
* Fix test cases referring to 'updates' instead of 'commands'
* I think this is an artifact of `connection: network_cli`?
* module will now check protocol running state before return
* warns on config argument no longer being needed
* lots of integration test cases updated
* updates unit and integration test cases for state check
fixes#21908
eos_config module wasn't respecting config block path (parents). This
patch fixes that problem. Also fixes a number of integration tests
cases
fixes#21903
Fix 'task name is not templated in retry callback'
Add a task_name property to TaskResult that knows to
check in TaskResult._task_fields.
Add integration test for v2_retry_runner callback
Fixes#18236
When parsing the configuration, the wrong indent level was set so the
configurations could not be properly compared. Also noted that
defaults: yes needed to be added to the integration tests to make the
test cases valid, which is also included in this patch
fixes#21828
* First set of CNOS network modules
Two modules: cnos_vlan and cnos_command. First 2 of 17 CNOS modules
* Revert "First set of CNOS network modules"
This reverts commit 63da68b2a857e622b449f514050eed25fbcf0a35.
* cnos_vlan module
First of 17 CNOS modules
* Modify playbook for cnos_vlan module testing
* fix issue found by shippable in cnos.py
* cnos_backup module
* Fix Python3.5 compilation error
* remove cnos_backup module from this PR
* add aliases file so that testing of the cnos_vlan module will be skipped.
* Use empty aliases file
* utility scripts under module_utils changed to BSD license.
* clean up PEP8 issues, add cnos_vlan tests
* remove blank lines from yml files
* remove more blank lines
* remove even more blank lines
* Revert "remove even more blank lines"
This reverts commit bb3998661ce387260340bca39d5f146f5ec54f44.
* re-submit last commit
* fix blank lines in this file under parsing
* fix yamllint errors in doc section of cnos_vlan
* Fix errors reported by Sanity check using validate-modules
* change test playbook to include tasks for all modules
* fix additional pep8 errors
* more fixes, plus fix for python3
* fix for empty options section, removed unnecessary documentation
* fix up documentation section
* Correct options capitalization
* fix example section, add options
* Fix example section of the documentation
I have from time to time a need of random password without
wanting to write them down (one example is mailman list creation,
that requires a password to be given to be sent to the list owner).
But using /dev/null do not return null, but the empty string, which
doesn't generate a password at all and so do not achieve my use case.
hashlib hashes operate on byte strings. When given a text string on
Python3, hashlib backtraces. When given a text string on Python2,
hashlib will backtrace if the string contains non-ascii characters.
Encode the text string to utf-8 prior to hashing to avoid this problem.
Fixes#21452
* known_hosts: support --diff
* known_hosts: support --diff also without --check
* Add unit tests and fix incorrect diff in one corner case
Tests are good!
* Refactor for readability
* Python 3 compat
* More Python 3 compat
* Add an integration test for known_hosts
* Handle ssh-keygen -HF returning non-zero exit code
AFAICT this is a bug in ssh-keygen in some newer OpenSSH versions
(>= 6.4 probably; see commit dd9d5cc670):
when you invoke ssh-keygen with -H and -F <host> options, it always
returns exit code 1. This is because in ssh-keygen.c there's a function
do_known_hosts() which calls
exit (find_host && !ctx.found_key);
at the end, and find_host is 1 (because we passed -F on the command line),
but ctx.found_key is always 0. Why is found_key always 0? Because the
callback passed to hostkeys_foreach(), which is known_hosts_hash(),
never bothers to set found_key to 1.
* This test does not need root
* Avoid ssh-ed25519 keys in sample known_hosts file
Older versions of OpenSSH do not like them and ssh-keygen -HF
aborts with an error when it sees such keys:
line 5 invalid key: example.net...
/root/ansible_testing/known_hosts is not a valid known_hosts file.
* Fix Python 3 errors
Specifically, the default mode of tempfile.NamedTemporaryFile is 'w+b',
which means Python 3 wants us to write bytes objects to it -- but the
keys we have are all unicode strings.
Network devices in the lab have by default an indentation of 1 for
sub-sections, and so does the netcfg.NetworkConfig constructor
indent param.
This will fix reported issue 21055, and make the integration tests
to pass.
Fixes#21055
- centralized skipping
- also fixed module name broken by previous refactor
- let action modules handle async processing
- moved async into base action class's module exec
- action plugins can now run final action as async
- actually skip copy if base skips
- fixed normal for new paths
- ensure internal stat is never async
- default poll to 10 as per docs
- added hint for callback fix on poll
- restructured late tmp, now a pipeline query
- moving action handler to connection as networking does
- fixed network assumption invocation is always passed
- centralized key cleanup, normalized internal var
- _supress_tmpdir_delete now in _ansible_xxx and gets removed from results
- delay internal key removal till after we use em
- nicer tmp removing, using existing methods
- moved cleanup tmp flag to mking tmp func
* added docs for vault and made trigger shorter: !vault
* added single var valuting
* Update playbooks_vault.rst
Edit pass for spelling and grammar. Ship it!
* Update playbooks_vault.rst
Typo fixes.
* Make the module_utils path configurable
* Add a config value to define the path site module_utils files
* Handle module_utils that do not have source as an error
* Make an integration test for module_utils envvar working
* Add documentation for the ANSIBLE_MODULE_UTILS config option/envvar
* Add it to the sample ansible.cfg
* Add it to intro_configuration.
* Also modify intro_configuration to place envvars on equal footing with
the config options (will need to document the envvar names in the
future)
* Also add the ANSIBLE_LIBRARY use case from
https://github.com/ansible/ansible/issues/15432 so we can close out
that bug.
* win_shortcut: Add missing $check_mode definition
For some reason this entry was missing, possible a merge-conflict gone
wrong :-(
* Added integration tests and bugfix
Add missing changes.
This version just gets the relevant paths from PluginLoader and then
uses the existing imp.find_plugin() calls in the AnsiballZ code to load
the proper module_utils.
Modify PluginLoader to optionally omit subdirectories (module_utils
needs to operate on top level dirs, not on subdirs because it has
a hierarchical namespace whereas all other plugins use a flat
namespace).
Rename snippet* variables to module_utils*
Add a small number of unittests for recursive_finder
Add a larger number of integration tests to demonstrate that
module_utils is working.
Whitelist module-style shebang in test target library dirs
Prefix module_data variable with b_ to be clear that it holds bytes data
* Use vars for user, group and locale suffix.
* Switch tests from `es_MX` to `es_ES`.
* Enable postgresql tests on freebsd.
* Work-arounds for service restart on freebsd.
* Fix git remote url change detection.
When comparing paths, unfrackgitpath must be called on both sides
of the comparison. Otherwise comparisons involving symlinks will
return incorrect results.
* Re-enable git test on OS X.
* Return None from unfrackgitpath when path is None.
Per a change in jinja2 2.9, local variables no longer are prefixed
with l_, so this updates AnsibleJ2Vars to pull in all locals (while
excluding some) regardless of name.
Fixes#20063
(cherry picked from commit 4d49b317929b86e1fc1b0cbace825ff73b372dc7)
* Fix `iterators` test for osx.
* Enable `git` test for osx.
* Enable `get_url` test for osx and freebsd.
* Remove unused SNI_URI from uri test.
* Use ansible hosted SNI host oustside docker.
* Enable `uri` test for osx and freebsd.
* Enable `gathering_facts` test for osx.
* Skip single task on osx until test is fixed.
* Add jinja2 groupby filter override to cast namedtuple to tuple. Fixes#20098
* Address some of the requested changes
* Quoting
* Print the python path and version
* Be less explicitly verbose, rely on implicit verbosity
* Adding support for Amazon ECR
This patch adds a new module named ecr, which can create, update or
destroy Amazon EC2 Container Registries. It also handles the management
of ECR policies.
* ecs_ecr: addressed review feeback
* Renaming ecr to ecs_ecr
* Fixed docs
* Removed bad doc about empty string handling
* Added example of `delete_policy`
* Removed `policy_text` option; switched policy to `json` type so
it can accept string or dict
* Added support for specifying registry_id
* Added explicit else after returned if clauses
* Added `force_set_policy` option
* Improved `set_repository_policy` error handling
* Fixed policy comparisons when AWS doesn't keep the ordering stable
* Moved `boto_exception` into the module
* Conditional include on ansible_network_os
* copy & paste error
* More tests
* More tests
* junos tests (based on vyos)
* remove excessive whitespace
* Pass in ansible_network_os
* net_command for ios
* consistent debug
* wrapp line
* ansible-test changes made in another PR
* ansible-test changes made in another PR
map + extract is the usual way to use it but map isn't available on
older versions of jinja2 that we still work with. Test extract even on
those versions.
* Only add Content-Type if not specified in headers. Fixes#20046
* Update documentation to indicate body_format will not override Content-Type if specified in headers
When using AWS we have to use the full domain name in the inventory file, which
we rather than the short name. This change avoids that ending up being
set in the tests.
* Cleanup git tests
* Split git tests in seperate files
* Remove use of repo_depth_url
* Use native yaml
* Remove unnecessary remote/local clones
* Fix newlines for yamllint
* If the hash is valid (full-length) but doesn't exist, git returns 128 instead of 1.
* Ensure git doesn't use hardlinks for shallow clones
* Reenable yum install root tests
No need for sos to test installroot. Something with less deps works
just as well.
* Fix yum installroot.
Fix module import to use fail_json when the modules aren't installed.
Remove wildcard imports
* Lsat task is supposed to remove sos so make that happen
* Add --installroot to YUM and DNF modules, issue #11310
This continues ansible-modules-core#1558, and
ansible-modules-core#1669
Allow specifying installroot for the yum and dnf modules
to install and remove packages in a location other than /.
* Remove empty aliases
* Simpler installroot set default logic
* Refactoring: split readkeys() into readfile() and parsekeys()
* Refactoring: split writekeys() into writefile() and serialize()
* authorized_key: support --diff
* Refactoring: remove no-longer used readkeys()/writekeys()
* Integration test for authorized_key in check mode
The gce_tag module can support updating tags on multiple instances via an instance_pattern field. Full Python regex is supported in the instance_pattern field.
'instance_pattern' and 'instance_name' are mutually exclusive and one must be specified.
The integration test for the gce_tag module has been updated to support the instance_pattern parameter. Unit tests have been added to test the list-manipulation functionality.
Run the integration test with:
TEST_FLAGS='--tags "test_gce_tag"' make gce
Run the unit tests with:
python test/units/modules/cloud/google/test_gce_tag.py
An inner single-quote pair breaks out of the outer single-quote
pair. Rather than escaping the inner quotes to protect against
this, just use the fact that `str()` is equivalent to `""`.
* Fix UnboundLocalError remote_head in git
Fixes#5505
The use of remote_head was a leftover of #4562.
remote_head is not necessary, since the repo is unchanged anyway and
after is set correctly.
Further changes:
* Set changed=True and msg once local_mods are detected and reset.
* Remove need_fetch that is always True (due to previous if) to improve
clarity
* Don't exit early for local_mods but run submodules update and
switch_version
* Add test for git with local modifications
* Use native yaml for apache2 test
* Test removal of default modules with force
a2enmod on debian has `-f`, but not on SUSE (runs there without force).
Therefore don't test that option on SUSE.
The docs already specify that the option is intended for Debian systems
only.
* Fix regression in jinja2 include search path
Since commit 3c39bb5, the 'ansible_search_path' variable is used to set
jinja2's search path for {% include %} directives. However, this path is
the the proper one because our templates live in 'templates' subdirs in
our search path.
This is a regression because previously, our include search path would
include the dirname of the currently interpreted file, which worked most
of the time.
fixes#18526
* Fix template lookup search path
Improve fix in commit c96c853 so that the search path contain both
template-suffixed paths as well as original paths.
ref PR #18617
* Add integration test for template lookups
Tests regression at #18526
This test fails on current devel branch and succeeds on PR #18617
* Add test cases for VyOS commands that don't honor paging settings
Testing for issue fixed in PR #18546
* Add provider line and fix indentation
For the way we invoke the tests we need to specify the `provider:`
Also fix the indentation on `register:`
This change corrects problems reported by the `yamllint` linter.
Since key duplication problems were removed in 4d48711, this commit
mainly fixes trailing spaces and extra empty lines at beginning/end of
files.
* Added test for sequenced-name instance generation (num_instances)
* Added param-check tags to tests that only do argument checking
Should be merged AFTER ansible/ansible-modules-core#4276
- Correct directory name in test/README.md
- Move code-smell tests to test/sanity/code-smell
- Update code-smell.sh to use new script paths
- Add test/integration/target-prefixes.win for ansible-test
- Move module unit tests to match module directory layout
* Network Test Documentation
Will need improving over time, though this ensure that everything that was in `ansible/test-network-modules` is in `ansible/ansible`
* Update README.md
* Inventory file
* Network module prefixes
In ansible-test we should skip tests for these modules, they will be
tested via another process.
* Update target-prefixes.network
If the facts returned by setup included strings that
had double quotes in them, the asserts in test_gathering_facts.yml
would fail with errors like:
"The conditional check '\"[{u'mounts': {u'options':
u'rw,context=\"system_u:\"'}}]\" != \"UNDEF_HW\"' failed. The error was:
template error while templating string: expected token 'end of statement
block', got 'system_u'. String: {% if \"[{u'mounts': {u'options':
u'rw,context=\"system_u:\"'}}]\" != \"UNDEF_HW\" %} True {% else %}
False {% endif %}"
For one example, if mount facts returned an 'options' field that
included double quoated selinux context ids, the test would fail.
Fix is removing the double quoting in the assert 'that:' lines,
and removing the unneeded double curly brackets.
* Add tag verification test (ansible-modules-core PR 2654)
* Fix typo
* Use smaller repo for testing, add dependency control
* Test is gpg exists before running git signing tasks
* Correct the test conditionals so that gpg1 is tested
Implement tag and skip_tag handling in the CLI() class. Change tag and
skip_tag command line options to be accepted multiple times on the CLI
and add them together rather than overwrite.
* Make it configurable whether to merge or overwrite multiple --tags arguments
* Make the base CLI class an abstractbaseclass so we can implement
functionality in parse() but still make subclasses implement it.
* Deprecate the overwrite feature of --tags with a message that the
default will change in 2.4 and go away in 2.5.
* Add documentation for merge_multiple_cli_flags
* Fix galaxy search so its tags argument does not conflict with generic tags
* Unit tests and more integration tests for tags
* Turn mount test back on
* Mount tests need PRIVILEGED so turn that back on
Revert "Revert "Set PRIVILEGED=true for non_destructive tests. (#17733)" (#17738)"
This reverts commit dc0fb1c212.
* Add a needs_privileged tag so that we can skip mount tests on centos6
Some containers timeout on shippable tests when run with privileged.
Unfortunately, some tests require that in order to run. Tagging those
allows us to skip those tests on the platforms that timeout when we get
ready to run the integration test in shippable.
* Centos6 times out with PRIVILEGED set so remove that (will disable the mount tests on centos6)
* Remove false start
* Add tests for the mount module
* Switch from unmounted to absent...
the code for mounting always modifies fstab so we need to always modify
fstab to cleanup as well.
* Fix comments and copyright
Includes:
* Tests to create, delete, resize, recreate and configure an Autoscaler
* Tests for parameters only. Use TEST_FLAGS="--tags 'param-check'" to execute only these tests.
Requires:
* An instance template to be configured in your project.
* Fixes to the controller text model
* Change command line args to text type
* Make display replace undecodable bytes with replacement chars. This
is only a problem on pyhton3 where surrogates can enter into the msg
but sys.stdout doesn't know how to handle them.
* Remove a deprecated playbook syntax in unicode.yml
* Fix up run_cmd to change its parameters to byte string at appropriate times.
The test_async test target was updated to accommodate changes in
output buffering behavior in python 3. This change in behavior
may need to be addressed in the future.
* New features for include_vars
include_vars.py now allows you to include an entire directory and its nested directories of variable files.
Added Features..
* Ignore by default *.md, *.py, and *.pyc
* Ignore any list of files.
* Only include files nested by depth (default=unlimited)
* Match only files matching (valid regex)
* Sort files alphabetically and load in that order.
* Sort directories alphabetically and load in that order.
```
- include_vars: 'vars/all.yml'
- name: include all.yml
include_vars:
file: 'vars/all.yml'
- name: include all yml files in vars/all and all nested directories
include_vars:
dir: 'vars/all'
- name: include all yml files in vars/all and all nested directories and save the output in test.
include_vars:
dir: 'vars/all'
name: test
- name: include all yml files in vars/services
include_vars:
dir: 'vars/services'
depth: 1
- name: include only bastion.yml files
include_vars:
dir: 'vars'
files_matching: 'bastion.yml'
- name: include only all yml files exception bastion.yml
include_vars:
dir: 'vars'
ignore_files: 'bastion.yml'
```
* Added whitelist for file extensisions (yaml, yml, json)
* Removed unit tests in favor of integration tests
While trying to fix the test suite on python3, I noticed
this test fail due to to_json adding more whitespace in
python3 than in python2. So -w should ignored those
differences.
Fixes for these are either rewriting to get rid of the need for the
functions or using six.moves to get equivalent functions for both
python2 and python3
Make !vault-encrypted create a AnsibleVaultUnicode
yaml object that can be used as a regular string object.
This allows a playbook to include a encrypted vault
blob for the value of a yaml variable. A 'secret_password'
variable can have it's value encrypted instead of having
to vault encrypt an entire vars file.
Add __ENCRYPTED__ to the vault yaml types so
template.Template can treat it similar
to __UNSAFE__ flags.
vault.VaultLib api changes:
- Split VaultLib.encrypt to encrypt and encrypt_bytestring
- VaultLib.encrypt() previously accepted the plaintext data
as either a byte string or a unicode string.
Doing the right thing based on the input type would fail
on py3 if given a arg of type 'bytes'. To simplify the
API, vaultlib.encrypt() now assumes input plaintext is a
py2 unicode or py3 str. It will encode to utf-8 then call
the new encrypt_bytestring(). The new methods are less
ambiguous.
- moved VaultLib.is_encrypted logic to vault module scope
and split to is_encrypted() and is_encrypted_file().
Add a test/unit/mock/yaml_helper.py
It has some helpers for testing parsing/yaml
Integration tests added as roles test_vault and test_vault_embedded
* actions/unarchive: fix unarchive from remote url
Currently unarchive from remote url does not work because the core
unarchive module was updated to support 'remote_src' [1], but the
unarchive action plugin was not updated for this. This causes failures
because the action plugin assumes it needs to copy a file to the
remote server, but in the case of downloading a file from a remote
url a local file does not exist, so an error occurs when the file is
not found.
[1] https://github.com/ansible/ansible-modules-core/commit/467516e
* test_unarchive: fix test with wrong remote_src use
The non-ascii filenames test had improperly set remote_src=yes even
though it was actually copying the file from the local machine (i.e.
the file did not already exist remotely). This test was passing
until the remote_src behavior of unarchive was fixed in 276550f.
Rather than repeatedly searching for tasks by uuid via iterating over
all known blocks, cache the tasks when they are added to the PlayIterator
so the lookup becomes a simple key check in a dict.
* Query lookup plugin
* Add license and docstrings
* Add python3-ish imports
* Change query plugin type from lookup to filter
* Switch from dq to jsonpath_rw
* Add integration test for query filter
* Rename query filter to json_query
* Add jsonpath-rw
* Rename query filter to json_query
* Switch query implementation from jsonpath-rw to jmespath
When a task result has an empty results list, the
list should be ignored when determining the results
of `_check_key`. Here the empty list is treated the
same as a non-existent list.
This fixes a bug that manifests itself with squashed
items - namely the task result contains the correct
value for the key, but an empty results list. The
empty results list was treated as zero failures
when deciding which handler to call - so the task
show as a success in the output, but is deemed to
have failed when deciding whether to continue.
This also demonstrates a mismatch between task
result processing and play iteration.
A test is also added for this case, but it would not
have caught the bug - because the bug is really in
the display, and not the success/failure of the
task (visually the test is more accurate).
Fixesansible/ansible-modules-core#4214
Binary module tests now download pre-compiled binaries from S3
instead of downloading go and compiling the modules.
Files downloaded form SourceForge are now downloaded from S3.
* Enabled more tests in test_service on systems with systemd.
* Fix inconsistency in cleaning up test service. The conditions for service setup and service cleanup were different.
* Add check mode run for service enable.
Enable color output from integration tests in Docker containers:
* In run_tests.sh when output is attached to a terminal.
* In shippable/integration.sh using force mode (can be disabled).
Also fix blocks tests to work with or without color output
* add check_mode option for tasks
includes example testcases for the template module
* extend check_mode option
* replace always_run, see also proposal rename_always_run
* rename always_run where used and add deprecation warning
* add some documentation
* have check_mode overwrite always_run
* use unique template name to prevent conflicts
test_check_mode was right before, but failed due to using the same filename as other roles
* still mention always_run in the docs
* set deprecation of always_run to version 2.4
* fix rst style
* expand documentation on per-task check mode
I suspect this problem was masked previously as older versions of pywinrm
where significantly slower, allowing more time for the windows installer service to complete.
6eefc11c converted task.loop_control into an object, but while the other
callers were updated to use .loop_var instead of .get('loop_var'), this
site was overlooked.
This can be reproduced by including with loop_control a file that does
set_fact; a simple regression test along these lines is included.
* Lookup unencrypted password must not include salt
* Integration test lookup: remove previous directory
* Test that lookup password doesn't return salt
* Lookup password: test behavior with empty encrypt parameter
Closes#16189
* add git shallow fetch test
covers https://github.com/ansible/ansible-modules-core/issues/3782
updating a repo with depth=1 fails silently if version==HEAD
* disable git shallow tests for old git versions
Older git versions don't treat the --depth option correctly.
While the git module tried to work around this and introduced subtle
bugs, ansible/ansible-modules-core#3794 falls back to full checkouts.
Don't run the tests then.
* reduce async sleep time in test
* make zypper test less destructive (don't break following uses of zypper)
* fix ca cert on suse
* fix/enable postgres/mysql on opensuseleap
* fix mysql test for mysql versions 5.7.6 and newer
* skip sni_host check on ubuntu1604
* add HTTPTESTER flag for test_uri
ubuntu 16.04 uses dash which drops env variables containing a dot
we work around this by adding an explicit env variable to enable httptester
* alias module is very basic and removing it leads to the suse default
config failing
* future improvements might test different modules and the effect of
them being removed
* When var name is the same as var content, try to template it before reporting that var is not defined.
Add asserts in test_var_blending to check this special corner case.
* Fix integration tests when using debug with list or dict.
* extend zypper integration tests
* fix caching issue of local RPMS on openSUSE 42.1 Leap
* add tests for simultaneous install/remove via prefixes +-
* test fail cases (rm patch or URL)
* test patch install (succes, unchanged second run, fail on wrong name)
* add test for pattern install
* include options refresh and priority introduced in ansible/ansible-modules-extras#1990
* test repo update both based on name and URL
* run on all suse flavors
* test overwriting repos by url or name
* add integration test for apache2_module
* fix assert syntax
* check os_family for this test
* add comment why only debian and suse are tested
* move apache2 test to destructive
* install apache2 package
- get_real_file will decrypt vault encrypted files and return a path to
a temporary file.
- cleanup_real_file will remove a temporary file created previously with
get_real_file
* Ziploader proof of concept (jimi-c)
* Cleanups to proof of concept ziploader branch:
* python3 compatible base64 encoding
* zipfile compression (still need to enable toggling this off for
systems without zlib support in python)
* Allow non-wildcard imports (still need to make this recusrsive so that
we can have module_utils code that imports other module_utils code.)
* Better tracebacks: module filename is kept and module_utils directory
is kept so that tracebacks show the real filenames that the errors
appear in.
* Make sure we import modules that are used into the module_utils files that they are used in.
* Set ansible version in a more pythonic way for ziploader than we were doing in module replacer
* Make it possible to set the module compression as an inventory var
This may be necessary on systems where python has been compiled without
zlib compression.
* Refactoring of module_common code:
* module replacer only replaces values that make sense for that type of
file (example: don't attempt to replace python imports if we're in
a powershell module).
* Implement configurable shebang support for ziploader wrapper
* Implement client-side constants (for SELINUX_SPECIAL_FS and SYSLOG)
via environment variable.
* Remove strip_comments param as we're never going to use it (ruins line
numbering)
* Don't repeat ourselves about detecting REPLACER
* Add an easy way to debug
* Port test-module to the ziploader-aware modify_module()
* strip comments and blank lines from the wrapper so we send less over the wire.
* Comments cleanup
* Remember to output write the module line itself in powershell modules
* for line in lines strips the newlines so we have to add them back in
